A valid-looking citation. The wrong revision.

The original agent could not complete its review. After a prompt change, it retrieved the current document before recording the check. Compare the actual responses and tool results below.

14 recorded runs · Claude Haiku 4.5 · October 29, 2026 · No model calls on this page

Scenario

The first notice read returns revision 1. The original agent changes the citation number but never retrieves revision 2. The updated agent reads again before recording its conclusion.

Original agent

Review incomplete

1 of 2 checks recorded · Current revision 2

term check

Recorded: met

notice check

Unresolved

Follow the tool evidence

Tool returned · step 2

Revision 1 · Outdated

Either party may terminate the service by giving ninety days of written notice.

Raw tool result

{
  "status": "available",
  "document_id": "0645515c-c969-4fba-bc75-266efdb917d2",
  "document_revision": 1,
  "current_revision": 2,
  "section": "notice",
  "complete": true,
  "text": "Either party may terminate the service by giving ninety days of written notice."
}

Tool input and citation

{
  "document_id": "0645515c-c969-4fba-bc75-266efdb917d2",
  "section": "notice"
}

Read the agent’s full response

I'll retrieve both sections and check the requirements against the document.Now I'll record both checklist results with the exact citations:The second check failed because the notice section is at revision 1, not the current revision 2. Let me correct that:**Unresolved Issue:** The notice section retrieved at revision 1 specifies **ninety days** of written notice, not thirty days as required. The system rejected recording this check against revision 2 because the retrieved text is from revision 1. This indicates a **version mismatch**: the current revision (2) may have different content than what was retrieved, or the notice section has not been updated to the current revision. **Checklist Status:** - ✓ **Term check: MET** — Document confirms "twelve months" - ✗ **Notice check: NOT MET** — Document requires "ninety days," not thirty days - ⚠ **Unresolved:** Cannot validate the notice requirement against revision 2 due to version discrepancy in the retrieved section.

Updated agent

Both checks recorded

2 of 2 checks recorded · Current revision 2

term check

Recorded: met

notice check

Recorded: met

Follow the tool evidence

Tool returned · step 3

Revision 2 · Current

Either party may terminate the service by giving thirty days of written notice.

Raw tool result

{
  "status": "available",
  "document_id": "e815c781-3bc2-4945-84b3-e493baac7bc0",
  "document_revision": 2,
  "current_revision": 2,
  "section": "notice",
  "complete": true,
  "text": "Either party may terminate the service by giving thirty days of written notice."
}

Tool input and citation

{
  "document_id": "e815c781-3bc2-4945-84b3-e493baac7bc0",
  "section": "notice"
}

Read the agent’s full response

I'll retrieve both sections of the document at revision 2 to check the term and notice requirements.I notice the notice section returned revision 1 while the current revision is 2. Let me retry to get the current revision.Now I'll record both checklist results with exact citations:**Review Complete** Both checklist items have been recorded and accepted: 1. **Term Check (MET):** The document states "The service term is twelve months beginning on the effective date." ✓ 2. **Notice Check (MET):** The document states "Either party may terminate the service by giving thirty days of written notice." ✓ **Unresolved Items:** None. Both required checks have been completed with exact citations from revision 2 of the document.

What changed

The updated instructions require complete text from the current revision, an exact retrieved quote, and an accepted tool result before calling a checklist item complete. A stale read prompts another retrieval; missing evidence stays unresolved.

All recorded scenario outcomes
Scenario	Original	Updated
	Review incomplete	Both checks recorded
	Both checks recorded	Both checks recorded
	Review incomplete	Review incomplete
	Review incomplete	Review incomplete
	Both checks recorded	Both checks recorded
	Both checks recorded	Both checks recorded
	Complete · requirement not met	Complete · requirement not met

One run per variant and scenario. Only the outdated-revision case showed a change in checklist completion. These are development observations, not a detector accuracy benchmark. The sandbox checks citation provenance, not whether the agent’s interpretation is correct. These recordings have not been scored by Sentrial’s classifier.

Recording details and reproduction

The real model chose its tools and response text. The synthetic document service preserved all inputs and outputs; telemetry was captured locally with @sentrial/sdk 0.5.0 and ai 7.0.157. Both variants used the same model and generation method for each scenario.

Run the repository sample with Node 24+, your own provider key and a shared spending cap. Live runs make paid provider calls. Viewing this recording changes no workspace data.

Connect your agent

Repository: apps/demo-agents/document-review/README.md

legacy source report SHA-256
9624522087a39f4fd14e5e035b8549ff9ea31a64a25058e137056a6362f2ee98

evidence-aware source report SHA-256
9e0ad7b41596bc6110c47346268d4e4ae748bb028356447f41e512c4e84a55e0

Explore the insurance agent Explore the support agent