21 — AI Evaluation and Regression Checklist#

Verify that AI-enabled applications have representative, measurable, repeatable evaluation of task quality and safety, and that changes are held to defined regression and release standards. Scores from a benchmark, a model judge, or a dashboard alone do not prove real-world correctness or absence of harm.

Scope: Backend / API-specific. This file is one part of the 21_AI_Evaluation_and_Regression.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

• Project / product:
• Release / version / environment:
• Scope and applicable jurisdictions / markets:
• Evaluated tasks, models, agents, retrieval, and user populations:
• Golden dataset / evaluator / model and prompt versions:
• Task, safety, latency, cost, and regression thresholds:
• Evaluator / domain / risk approvers:
• Review date:
• Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

1. Copy this shared checklist into project-specific evaluation records. Leave the repository's reusable checkbox items unchecked.
2. Define task outcomes, severe failures, baseline performance, and release thresholds before inspecting candidate results. Use separate measures for task success, policy/safety, retrieval, downstream tool actions, user experience, and operational cost where applicable.
3. Mark [x] only when verified from versioned dataset manifests, reproducible runs, validated scorers, qualified human reviews, CI outcomes, or a documented approver decision. Keep evidence, failures, exceptions, and genuine N/A reasons outside this template, indexed by EVAL ID.
4. Compare under equivalent settings; account for stochastic generation, sample size, evaluator bias, and real-world coverage limitations. A high aggregate score must never hide a severe individual failure.
5. Apply ## API Checks and ## Frontend Checks independently for supported interfaces. Document inapplicable items in project review records.
6. Re-run appropriate evaluations after model, prompt, retrieval, dataset, guardrail, API, tool, or frontend changes. Coordinate model selection, data curation, governance, safety/red teaming, and monitoring in their separate checklists.

API Checks#

• EVAL-097 Expose evaluation job creation, status, result retrieval, and deletion only to authorized roles and scopes where an eval API exists.
• EVAL-098 Separate staging, evaluation, and live production model endpoints and prevent sandbox cases from changing real customer records.
• EVAL-099 Version API and grader input/output contracts for test cases, responses, scores, and artifact references.
• EVAL-100 Verify stored evaluation runs identify exact model, prompt, retrieval, tool, safety policy, and dataset revisions.
• EVAL-101 Validate all evaluation input fields, tool fixtures, requested models, and maximum output or context sizes.
• EVAL-102 Use strictly scoped service credentials when invoking third-party judge models or provider evaluation endpoints.
• EVAL-103 Ensure evaluation traffic cannot bypass standard tenant isolation, data rights, or secret handling policies.
• EVAL-104 Test that external graders receive only approved sanitized content and required contextual fields.
• EVAL-105 Use safe mock or idempotent tools to assess agent side effects without executing real payments, deletions, or messages.
• EVAL-106 Distinguish model refusals, malformed outputs, policy blocks, tool failures, timeouts, and evaluator infrastructure errors.
• EVAL-107 Validate structured output and tool-call assertions independently of model-generated explanations.
• EVAL-108 Keep input data, reference labels, predictions, grades, and evaluator reasoning separate so expected answers cannot contaminate generation.
• EVAL-109 Limit retries, concurrency, total job duration, per-model requests, and spend for bulk evaluation runs.
• EVAL-110 Check job cancellation and partial evaluation behavior preserves accurate denominators and does not produce false passes.
• EVAL-111 Protect result exports, trace logs, raw prompts, grader comments, and sensitive test fixtures with access and retention controls.
• EVAL-112 Verify comparability of production and evaluation API versions, rate limits, streaming, and context-window behavior.
• EVAL-113 Emit safe correlation identifiers for run, test case, model response, tool activity, and judge verdict.
• EVAL-114 Test APIs with missing or malformed grader outputs and fail or escalate safely rather than recording zero-risk success.
• EVAL-115 Enforce release gates on server-validated evaluation results rather than editable dashboard summaries.
• EVAL-116 Test representative API contract and model-quality regressions together before promoting the release.

Release gate#

• The exact release has approved task definitions, meaningful primary metrics, severe-failure conditions, risk guardrails, and predeclared acceptance thresholds.
• Golden and held-out evaluation cases are representative of the intended deployment, versioned, permissioned, contamination-checked, and reviewed for major coverage gaps.
• Deterministic assertions and model-based graders are fit for purpose; human rubrics and automated judge scores have suitable calibration and auditability.
• Candidate and approved baseline results are compared with reproducible settings, adequate repeatability, and explicit statistical or practical uncertainty where needed.
• All release-blocking privacy, safety, authorization, and consequential task failures are resolved or have the specifically required authorized disposition; averaging does not waive critical failures.
• API, retrieval, agent/tool, and frontend end-to-end paths meet the relevant quality, contract, usability, accessibility, latency, and cost requirements.
• CI/controlled release gates, anomaly and regression review, safe rollback, production observation, and accountable incident response are ready.
• A named release approver has reviewed run artifacts, per-case regressions, coverage limitations, accepted risks, and the precise model/data/prompt/guardrail versions promoted.

Domain-specific approval rule: Block sign-off when the evaluation suite is materially unrepresentative or contaminated, scoring is unvalidated for decisions it controls, a severe user-impacting or unauthorized-action failure is unresolved, or an important metric regresses beyond its approved threshold without qualified acceptance. Passing these checks is not proof of complete AI safety, impartiality, or compliance with any particular jurisdiction.

Trusted references and scope#

The following official frameworks, provider documentation, and tool-maintainer references informed these project-specific verification questions. They are not universal requirements or substitutes for independent evaluation design.

• NIST AI RMF Playbook — Measure — risk-based evaluation methods, uncertainty, deployment conditions, and measurement gaps.
• NIST AI RMF Generative AI Profile (AI 600-1) — generative AI risk identification, evaluation, and trustworthiness considerations.
• OpenAI API — Evaluation best practices — task-specific evals, representative cases, model variability, calibration, and development-driven regression.
• OpenAI API — Graders — deterministic and model-based grading, human reference scores, grader limitations; note platform deprecation.
• Microsoft Foundry — Evaluation overview — evaluation workflow, datasets, response generation, quality/safety metrics, and production traces.
• Microsoft Foundry — Built-in Evaluators — task completion, retrieval, groundedness, safety, and textual similarity metrics.
• Microsoft Foundry — RAG Evaluators — separate retrieval-process and grounded-answer system evaluation.
• Google Cloud — Gen AI evaluation service overview — adaptive task rubrics, version/model comparisons, production sampling, and synthetic test cases.
• Amazon Bedrock — Automatic model evaluation jobs — task-specific dataset jobs, settings, built-in metrics, and evaluation artifacts.
• Amazon Bedrock — Human model evaluation jobs — human review teams, task-specific instructions, qualitative comparisons, and grading controls.
• Amazon Bedrock — LLM-as-a-judge evaluation jobs — judge-model evaluation configuration, IAM, datasets, and permissions.
• Ragas — Available evaluation metrics — RAG faithfulness/retrieval and agent task/tool metrics where this library is chosen.

Applicability: NIST AI RMF and its GenAI profile are voluntary risk frameworks, not certifications. As of November 2026, the OpenAI Evals platform and associated graders are being retired (read-only after October 31; scheduled closure November 30); avoid building new long-lived release gates dependent on that service and verify the latest migration guidance. Microsoft, Google, and AWS evaluation features, model judges, and scoring thresholds are platform-specific and may have version, regional, or preview limitations. Ragas metrics are implementation choices, not a universal scientific gold standard. Protect evaluation data and check legal rights before sharing it with third-party judges.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

• Test focus: held-out cases, severe failures, judge calibration, and CI quality thresholds; include API and service behavior where applicable.
• Cross-stack: promptfoo.
• Python: DeepEval + Ragas + pytest; for pytest-based suites, run python -m pytest -q.
• TypeScript: promptfoo + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
• JavaScript: promptfoo + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
• Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

• agentic-eval — Check end-to-end agent outcome regressions (github/awesome-copilot, 41,073 repo stars).
• eval-driven-dev — Test evaluation-based development gates (github/awesome-copilot, 41,073 repo stars).
• llm-evaluation — Review datasets, graders, and quality thresholds (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the backend / API scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.