Critical-Error Auto-Fail Flagger

The problem this kills

Random sampling is great for the average - and useless for the disaster. You review 2% of tickets, and the one where an agent gave a customer wrong account information, skipped an identity check, or breached a compliance rule is sitting quietly in the 98% you never looked at. In finance, healthcare, and security support, that single miss is the one that gets escalated, fined, or splashed across a complaint.

A critical error is not "could have phrased it better." It's the kind of failure that fails the whole interaction on its own, regardless of how polished the rest was. Those are exactly the tickets you can't afford to leave to chance.

This tool is the safety net under your random sampling. It reads every resolved ticket, looks for the specific failures your team has defined as fatal, and pulls the suspects to the top for a human to judge - so the worst misses get caught even when the dice don't land on them.

What you'll build

A small, private web app for your QA team:

• Load a batch of resolved tickets from a CSV or Google Sheet export - no integration required to start.
• AI screens each ticket against your critical-error definitions and flags potential auto-fails, each one tagged with the rule it matched and the exact quote or evidence that triggered it.
• A QA lead reviews the flags in a clean queue and confirms or dismisses each one - the AI never decides; it only nominates.
• Confirmed fails are recorded as auto-fails, routed to remediation (coaching, re-training, escalation), and the agent and ticket are logged.
• A trend dashboard shows critical errors over time, by type, by agent, and by team - so you can see whether the worst misses are getting rarer.
• Email alerts notify the right people the moment a critical error is confirmed.

What's inside the Implementation Plan

• A discovery interview that runs first. Before it builds anything, the plan has the AI agent interview you about your support operation: your current QA process, where your tickets live, how your fields are named, your real volumes, your exact critical-error rules, and your messy edge cases. It reflects a short tailored spec back to you for a thumbs-up - so you get a tool shaped to your business, not a generic template you have to bend to fit.
• A step-by-step build, each step ending in a ready-to-paste prompt for your AI coding agent.
• A clear definition of done so you know exactly when it's finished.
• A "No API yet?" fallback so the whole thing is buildable today: import a Sheet/CSV and export a clean CSV of flagged critical-error candidates in the columns your system expects.

The governance it includes (this is the point)

This isn't a toy script - it's built like an internal tool you can trust:

• Login so only your team can open it.
• Row-level security so each organization only ever sees its own tickets and flags.
• A complete audit trail - who flagged, who confirmed or dismissed, what evidence, and when.
• A hard human-in-the-loop gate - the AI drafts the flag with its reasoning, but nothing becomes an official auto-fail until a QA lead approves it. This is what protects your agents from false accusations.
• Duplicate guards keyed on ticket ID so the same ticket can never be flagged or recorded twice.

Who it's for

QA leads and support-quality managers in high-stakes environments - finance, healthcare, security, regulated services - where one critical miss matters more than a hundred small ones, and where you need to prove the worst errors are being caught and remediated.

You've got this - paste the first prompt and let the plan interview you.