Six Layers of Testing: How QA Made an AI Assistant Safe for a Leading National Medical Association

Quality assurance became the gating factor in launching an AI assistant for a leading national medical association, built to support 40,000+ surgeons and residents with high‑volume queries across renewals, registration, CME, and career pathways. In a high‑stakes healthcare context, the core risk was hallucination—confident, incorrect answers that erode trust and create liability. With knowledge fragmented across SharePoint, internal databases accessed via MCP connectors, and content scraped from the website, and interactions requiring complex multi‑turn reasoning at a projected annual scale of 10 million conversations, traditional QA methods fell short, leaving the association without the evidence needed to deploy safely.

Validating the system required addressing failure modes that standard QA cannot detect:

• Hallucinations: Confident but incorrect responses with no reliable detection in conventional testing
• Retrieval accuracy: Answers dependent on pulling the right content from fragmented, unverified sources
• Multi‑turn reliability: Maintaining accuracy across real, messy conversations—no single queries
• Responsible AI: Measuring fairness, safety, and behavior on sensitive topics
• Security resilience: Testing against prompt injection and jailbreak attacks
• Performance at scale: Ensuring accuracy and latency hold under peak volumes

The scale, complexity, and compliance sensitivity made traditional QA inadequate—requiring a purpose-built, AI-specific validation approach.

Evoke Technologies designed and executed a specialist Generative AI Quality Assurance program for the assistant before go-live. The program was structured around a single objective: provide measurable evidence — not assurances—that the AI was safe, accurate, and reliable enough for production use with 40,000+ medical professionals.

Six-Layer Testing Framework

The assistant was tested across six independent quality dimensions, each targeting a distinct failure mode in Generative AI systems:

• Retrieval quality — verifies that the correct source content is surfaced for every query.
• Hallucination detection — confirms answers are grounded in approved content, not fabricated.
• Context and multi-turn integrity — validates accuracy across long, multi-step conversations.
• Responsible AI — assesses fairness, safety, and behavior on sensitive topics.
• Adversarial and security — tests resistance to prompt injection and jailbreak attempts.
• Performance and scale — measures reliability and response quality under peak load.

Purpose-Built AI Evaluation Methods

Testing was conducted using AI evaluation frameworks designed specifically for Generative AI, measuring retrieval accuracy, faithfulness, and answer relevance. Manual or spreadsheet-based QA was deliberately avoided because it cannot evaluate probabilistic AI outputs with the consistency required in a regulated environment.

Layered Testing Isolated the Real Source of Errors

By testing each layer of the AI pipeline independently, the program produced a critical finding: most incorrect answers originated in the retrieval layer rather than the language model. The AI was generating accurate responses from the inputs it received — but those inputs were often the wrong source documents. Correcting the retrieval logic significantly reduced hallucinations. This level of diagnosis is not possible with end-to-end testing alone, and it prevents the costly effort spent tuning the wrong component.

With hallucinations diagnosed at their true source, six quality dimensions validated end-to-end, and an AI assistant safely serving up to 10 million annual visits seamlessly handled under peak load, Evoke demonstrates that specialist Gen AI Quality Assurance is what moves an AI product from demo-ready to boardroom-trusted — turning trust from a hope into measurable evidence.