← Case Studies
EnterpriseData Centers

How CoreWeave cut agent testing from 4 months to under 2 weeks

With Taj Singh, Head of IT Strategy and Transformation at CoreWeave

8xFaster agent testing cycles, from 4 months down to under 2 weeks
35+Enterprise agents being built across procurement, HR, marketing, and sales
3,000Employees at CoreWeave, 58% already using AI regularly

CoreWeave is a 3,000-person data center company powering some of the largest AI workloads in the world. Behind the infrastructure, their internal teams are building a new generation of enterprise agents to automate everything from parts procurement to employee onboarding.

Taj Singh leads this effort. His "three steps to agent fluency" framework is driving AI adoption across 13 business verticals. Step one: prompt fluency. Step two: basic democratized agents. Step three: deeply actioning agents that deliver high ROI. It's that third step where things got complicated.

“Agent building took us two days after two weeks of requirements. But four months of testing. I'm still not done.”

Taj Singh, Head of IT Strategy and Transformation, CoreWeave

01

Agents that actually do things

CoreWeave's agents are not chatbots. When a power strip fails in a data center, a JIRA ticket triggers an agent that queries a custom parts database, evaluates 50+ vendors on delivery history and ratings, selects the best match, sends requests for quotes, compares responses, and places the purchase order. Every step involves real decisions with real consequences.

Their HR onboarding agent is even more complex. Called the "adaptive ramp up coach," it guides new hires through their first weeks at the company. But it's completely non-linear. Every day is different. Every person's path is different. The agent pulls from Confluence, Slack history, Zoom recordings, and internal wikis to deliver hyper-personalized guidance. There are no fixed steps, just a main goal with 20 sub-goals that can be achieved in any order.

Across the company, 35+ agents like these are being built on Workato, Glean, and LangGraph.

02

The testing bottleneck

The parts ordering agent took two days to build. Testing took four months. And it still wasn't done. The very first test case failed: the agent looked up vendors and returned zero results. Engineers had never done this query before. Every subsequent step surfaced new edge cases. Wrong vendor selected. Part numbers that changed every six months. Amp ratings that required physics-level reasoning to validate compatibility.

The marketing agent fared worse. First test: 3% of generated emails were accurate. The team was figuring out how testing should even work while simultaneously trying to test the agents. There was no framework, no CI/CD, no intake process. Just engineers manually running through scenarios and hoping to catch failures.

CoreWeave tried building observability and issue tracking in-house. It didn't work. The problem wasn't a lack of engineering talent. It was that agent testing is fundamentally different from software testing. When every run can take a different path, you can't write deterministic test cases.

“This is worse than software engineering. These flows are not like sending an email. Every step had a failure.”

Taj Singh, Head of IT Strategy and Transformation, CoreWeave

03

What Sentrial changed

Signal detection for non-deterministic agents

CoreWeave's biggest challenge was testing agents where the correct output changes every time. Sentrial's custom classifiers train on previous session logs, so each agent gets signal detection tuned to its specific failure modes. The parts agent gets flagged for wrong vendor selection. The HR agent gets flagged for skipping critical onboarding steps. The marketing agent gets flagged for inaccurate personalization. Instead of manually reviewing every run, the team only looks at what matters.

Regression testing across prompt and workflow changes

Every time CoreWeave modifies a prompt or adjusts a workflow step, Sentrial runs the existing test suite and surfaces any regressions. If the success rate drops or a signal spikes, the team knows immediately, not four months later.

A/B experiments across platforms

CoreWeave is evaluating whether to build agents on Workato or LangGraph. Sentrial's A/B experiments let them run the same test cases on both implementations and compare success rate, latency, cost, and accuracy side by side. Instead of gut feel, they get a quantified decision.

Production monitoring from day one

Once agents ship, the same signals and classifiers that caught issues in testing continue running in production. No separate monitoring setup. No new tools to learn. The team went from having zero production visibility to full session replay and automatic issue detection with no additional work.

04

The results

Agent testing cycles

4 months per agent

Under 2 weeks

Production monitoring

Built in-house, failed

Automatic, zero additional setup

Issue discovery

Manual review of every run

Signal detection surfaces only what matters

Platform decisions

Gut feel and guesswork

Quantified A/B experiments across Workato and LangGraph

The engineering team that spent months stuck in testing loops is now shipping agents in weeks. The new LangGraph-focused hires can iterate on agent architecture without worrying about building a testing framework from scratch. And the board reports that Taj sends every week now have real data behind them.

“We have a testing framework, we have a launching framework, we have CI/CD around agents — yet Sentrial is 10x better than them all at actually monitoring and making our agents work better.”

Taj Singh, Head of IT Strategy and Transformation, CoreWeave

Integration across platforms

CoreWeave's agents run on multiple platforms. For LangGraph agents, integration is a few lines of code. For Workato workflows, Sentrial plugs in through HTTP tracking calls added to existing workflow steps. No refactoring, no new infrastructure. One monitoring layer across every agent, regardless of where it runs.

Stop building testing frameworks. Start shipping agents.

CoreWeave went from 4-month testing cycles to under 2 weeks. See how Sentrial can do the same for your enterprise agents.