What this engagement is evidence of
Citation first output in a regulated setting, inside the client's own cloud tenancy, with confidence routing to a named human reviewer and an immutable record behind every claim. This is the evaluation and guardrails problem at its strictest, because the reader is a partner signing their name to it.
Big 4 audit practices live under unrelenting regulatory pressure. Every engagement demands exhaustive documentation - risk matrices, compliance cross-references, workpapers, and partner-ready summaries - before a single judgment call can be made. At EY, that documentation work was falling on senior auditors who should have been spending their hours on the high-stakes reasoning that clients pay for. The problem wasn't a shortage of talent. It was that talent was buried under assembly work.
Every prior attempt at AI in audit had failed for the same reason: hallucinated outputs that auditors couldn't trust or cite. Our core design principle was citation-first - every claim the system makes must link directly to a source document, regulatory reference, or data record. We spent the first two weeks not building software, but mapping exactly which document types lived where, who was authorised to access what, and what a 'good enough to hand to a partner' draft actually looked like. We then designed a human-in-the-loop architecture where AI handles assembly and cross-referencing, and every human touch is on judgment - not data retrieval.
The system is a LangGraph-based multi-agent pipeline with four specialised agents: a Data Retrieval Agent that queries all 8 source systems via secure API connectors, a Regulatory Mapping Agent that cross-references extracted data against IFRS, SOX, and local GAAP frameworks stored in a structured knowledge base, a Draft Generation Agent that writes sections of the risk assessment with inline citations, and a Review Routing Agent that scores output confidence and routes low-confidence sections directly to a named partner queue. Everything is logged to an immutable audit trail in Azure Blob Storage.
Data connectors
Secure REST connectors to all 8 source systems - ERPs, legacy workpaper tools, and the firm's proprietary client data platform - with field-level access control per engagement team.
Regulatory knowledge base
Structured vector store of IFRS, SOX, and 12 local GAAP frameworks with versioning, so the system always cites the correct year of the standard and flags when a referenced standard has been updated since last engagement.
Citation engine
Every sentence in a generated draft carries a citation ID linking back to a specific source document and page. Reviewers click any sentence to see its evidence chain instantly.
Confidence routing
Any output section scoring below the defined confidence threshold is automatically routed to the relevant human reviewer with the agent's uncertainty reasoning pre-populated - so partners see exactly why the AI wasn't sure.
Audit trail
Every agent decision, API call, and human override is written to an immutable log in Azure Blob Storage. The entire decision history for a client report can be exported as a single compliance record.
The numbers only tell part of the story. The bigger shift was cultural: auditors started treating AI output as a first draft they refined, not a risk they managed around.
72% reduction in report preparation time. Zero hallucination incidents. 50+ concurrent engagements handled across 6 countries with full audit trails.
We thought the hard part would be getting accurate AI outputs. The real challenge was building a citation system trustworthy enough that a senior partner would stake their sign-off on it. Every architectural decision - from the knowledge base schema to the confidence thresholds - was driven by that requirement.
The first version of the system was technically solid but adoption was slow. Auditors didn't distrust the technology - they distrusted change during the highest-stakes period of their year. We ran structured onboarding sessions, kept human override frictionless, and made the first two months a 'co-pilot mode' where auditors could see AI drafts alongside their manual work. Adoption accelerated sharply once they had direct comparison evidence.
We built a suite of 200+ test cases before shipping to production - covering edge cases in regulatory cross-referencing, ambiguous jurisdiction overlaps, and known failure modes from previous AI attempts at EY. This harness now runs on every model update, giving the engineering team and EY's risk team a shared quality gate.
“This system handles the most time-consuming part of our audit cycle. What used to take two senior staff three days now takes hours - with full traceability. Our partners now spend their time on the work that actually requires their judgment.”
Senior Audit Director, EY
Model versions are deliberately not listed. The architecture treats the provider as a configuration decision, and a version printed on a page is out of date within the quarter.
engineering / other engagements
Send the task, the data it runs on and the systems it has to touch. You get a written read from an engineer within a working day.