CoreWeave is a 3,000-person data center company powering some of the largest AI workloads in the world. Behind the infrastructure, their internal teams are building a new generation of enterprise agents to automate everything from parts procurement to employee onboarding.
Taj Singh leads this effort. His "three steps to agent fluency" framework is driving AI adoption across 13 business verticals. Step one: prompt fluency. Step two: basic democratized agents. Step three: deeply actioning agents that deliver high ROI. It's that third step where things got complicated.
“Agent building took us two days after two weeks of requirements. But four months of testing. I'm still not done.”
Taj Singh, Head of IT Strategy and Transformation, CoreWeave
Agents that actually do things
CoreWeave's agents are not chatbots. When a power strip fails in a data center, a JIRA ticket triggers an agent that queries a custom parts database, evaluates 50+ vendors on delivery history and ratings, selects the best match, sends requests for quotes, compares responses, and places the purchase order. Every step involves real decisions with real consequences.
Their HR onboarding agent is even more complex. Called the "adaptive ramp up coach," it guides new hires through their first weeks at the company. But it's completely non-linear. Every day is different. Every person's path is different. The agent pulls from Confluence, Slack history, Zoom recordings, and internal wikis to deliver hyper-personalized guidance. There are no fixed steps, just a main goal with 20 sub-goals that can be achieved in any order.
Across the company, 35+ agents like these are being built on Workato, Glean, and LangGraph.
The testing bottleneck
The parts ordering agent took two days to build. Testing took four months. And it still wasn't done. The very first test case failed: the agent looked up vendors and returned zero results. Engineers had never done this query before. Every subsequent step surfaced new edge cases. Wrong vendor selected. Part numbers that changed every six months. Amp ratings that required physics-level reasoning to validate compatibility.
The marketing agent fared worse. First test: 3% of generated emails were accurate. The team was figuring out how testing should even work while simultaneously trying to test the agents. There was no framework, no CI/CD, no intake process. Just engineers manually running through scenarios and hoping to catch failures.
CoreWeave tried building observability and issue tracking in-house. It didn't work. The problem wasn't a lack of engineering talent. It was that agent testing is fundamentally different from software testing. When every run can take a different path, you can't write deterministic test cases.
“This is worse than software engineering. These flows are not like sending an email. Every step had a failure.”
Taj Singh, Head of IT Strategy and Transformation, CoreWeave
What Sentrial changed
Signal detection for non-deterministic agents
CoreWeave's biggest challenge was testing agents where the correct output changes every time. Sentrial's custom classifiers train on previous session logs, so each agent gets signal detection tuned to its specific failure modes. The parts agent gets flagged for wrong vendor selection. The HR agent gets flagged for skipping critical onboarding steps. The marketing agent gets flagged for inaccurate personalization. Instead of manually reviewing every run, the team only looks at what matters.
Regression testing across prompt and workflow changes
Every time CoreWeave modifies a prompt or adjusts a workflow step, Sentrial runs the existing test suite and surfaces any regressions. If the success rate drops or a signal spikes, the team knows immediately, not four months later.
A/B experiments across platforms
CoreWeave is evaluating whether to build agents on Workato or LangGraph. Sentrial's A/B experiments let them run the same test cases on both implementations and compare success rate, latency, cost, and accuracy side by side. Instead of gut feel, they get a quantified decision.
Production monitoring from day one
Once agents ship, the same signals and classifiers that caught issues in testing continue running in production. No separate monitoring setup. No new tools to learn. The team went from having zero production visibility to full session replay and automatic issue detection with no additional work.
The results
Agent testing cycles
4 months per agent
→Under 2 weeks
Production monitoring
Built in-house, failed
→Automatic, zero additional setup
Issue discovery
Manual review of every run
→Signal detection surfaces only what matters
Platform decisions
Gut feel and guesswork
→Quantified A/B experiments across Workato and LangGraph
The engineering team that spent months stuck in testing loops is now shipping agents in weeks. The new LangGraph-focused hires can iterate on agent architecture without worrying about building a testing framework from scratch. And the board reports that Taj sends every week now have real data behind them.
“We have a testing framework, we have a launching framework, we have CI/CD around agents — yet Sentrial is 10x better than them all at actually monitoring and making our agents work better.”
Taj Singh, Head of IT Strategy and Transformation, CoreWeave
Integration across platforms
CoreWeave's agents run on multiple platforms. For LangGraph agents, integration is a few lines of code. For Workato workflows, Sentrial plugs in through HTTP tracking calls added to existing workflow steps. No refactoring, no new infrastructure. One monitoring layer across every agent, regardless of where it runs.
Stop building testing frameworks. Start shipping agents.
CoreWeave went from 4-month testing cycles to under 2 weeks. See how Sentrial can do the same for your enterprise agents.