awesomechecklists
llms.txtGitHub ↗
/

07 — AI Model and Dataset Cards Checklist#

Verify that model and dataset documentation accurately communicates intended use, capabilities, limitations, provenance, evaluation evidence, risks, and change history. A card is a living disclosure and engineering record—not proof of regulatory compliance, model safety, lawful data rights, or benchmark reproducibility.

Scope: AI-specific. This file is one part of the 07_AI_Model_and_Dataset_Cards.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Model, dataset, provider and artifact versions:
  • Card audience, publication location, and visibility rules:
  • Documentation, evaluation, provenance and risk owners:
  • Reviewer / owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Copy the reusable checklist into project-specific review records. Leave every checkbox in this repository unchecked.
  2. Evaluate the model card and dataset card separately where both exist; for a third-party model or dataset, verify documented provider facts independently when feasible and mark undisclosed facts as unknown instead of guessing.
  3. Mark [x] only after checking versioned model or dataset artifacts, lineage manifests, evaluation results, licenses and rights reviews, source documents, publication controls, or a recorded owner decision. Keep evidence, accepted exceptions, failed checks and justified N/A outside this shared template, keyed by CARDS ID.
  4. Apply ## API Checks and ## Frontend Checks where applicable, noting that an internal-only or offline dataset may not expose one or both surfaces.
  5. Decide explicitly what is appropriate to publish: technical summaries and meaningful limitations may be public, while source data, confidential tests, protected characteristics and security-sensitive artifacts may require restricted internal records.
  6. Re-review cards when source rights, model versions, usage boundaries, dataset versions, evaluation outcomes, safety policies, deployment conditions or known incidents change. Use the dedicated data curation, model evaluation, governance and lifecycle checklists for deeper controls.

Intended uses and limitations#

  • CARDS-001 Describe each model and dataset in plain language, including its purpose, scope, owner, current status, and version.
  • CARDS-002 Identify intended direct users, downstream integrators, affected individuals, and the expected operating environment.
  • CARDS-003 Distinguish supported tasks from adjacent tasks that have not been evaluated or approved.
  • CARDS-004 State out-of-scope uses, prohibited uses, and predictable misuse scenarios with actionable examples.
  • CARDS-005 Identify high-impact domains and circumstances where independent human review or stronger controls are necessary.
  • CARDS-006 Describe supported languages, scripts, modalities, regions, input sizes, and interface constraints using actual tested coverage.
  • CARDS-007 Record known task-specific limitations such as stale knowledge, hallucinations, missing classes, and weak long-context behavior where relevant.
  • CARDS-008 Explain assumptions about input quality, data distributions, prompt structure, context sources, and required external services.
  • CARDS-009 State confidence or uncertainty limits without portraying uncalibrated scores as reliable probabilities.
  • CARDS-010 Document how outputs can affect people who are not direct users, including groups with limited opportunity to contest errors.
  • CARDS-011 Describe reasonably foreseeable safety, security, privacy, and discrimination risks relevant to the documented deployment.
  • CARDS-012 Explain mitigations, fallback procedures, and circumstances when users must abstain, escalate, or verify independently.
  • CARDS-013 Distinguish model capabilities from safeguards imposed by the surrounding application, access controls, and human workflow.
  • CARDS-014 List supported deployment environments, resource constraints, inference dependencies, and known incompatibilities.
  • CARDS-015 Document any restrictions arising from licenses, provider terms, consent, data processing, or applicable market requirements.
  • CARDS-016 Clarify whether the card covers a base model, fine-tuned variant, quantized checkpoint, hosted snapshot, or complete AI system.
  • CARDS-017 Provide examples of appropriate inputs and expected behavior without including confidential or disallowed personal data.
  • CARDS-018 State important conditions where publicly reported benchmark scores may not generalize to production workloads.
  • CARDS-019 Explain the potential for performance changes due to model-provider updates, retrieval freshness, tools, or user population drift.
  • CARDS-020 Identify who can approve exceptions and where users or integrators can report errors or suspected harmful behavior.
  • CARDS-021 Separate evidence-based statements, provider assertions, assumptions, and unknown information in the card.
  • CARDS-022 Review whether the intended-use guidance is comprehensible to non-expert stakeholders and accurately limits implied guarantees.

Data and training provenance#

  • CARDS-023 Identify model developer, provider, distributor, architecture family, checkpoint, release date, and accountable maintainer.
  • CARDS-024 Record base-model identity and lineage for adapters, merges, distillation, fine-tuning, quantization, or post-training variants.
  • CARDS-025 Link approved training, validation, and evaluation datasets to exact versioned manifests or immutable references where available.
  • CARDS-026 Summarize training data sources, collection periods, geographic and language coverage, and intended representativeness.
  • CARDS-027 Document major source omissions, selection biases, exclusions, and known gaps in corpus coverage.
  • CARDS-028 State dataset creators, stewards, publishers, acquisition channels, and the purpose for which data was collected.
  • CARDS-029 Identify source license terms and relevant restrictions on access, redistribution, commercial use, training, and derived artifacts.
  • CARDS-030 Record applicable notice, consent, contractual, and privacy reviews for identifiable or sensitive source data.
  • CARDS-031 Document whether and how personal, protected, copyrighted, or proprietary content is filtered or handled in source material.
  • CARDS-032 Describe annotation approach, label schema, annotator or expert qualifications, and adjudication methods where applicable.
  • CARDS-033 Report significant data-cleaning, deduplication, filtering, augmentation, and synthetic-generation steps.
  • CARDS-034 Track transformation code revisions, extraction tools, feature schemas, tokenizers, and relevant preprocessing versions.
  • CARDS-035 Explain training and validation split strategies and potential overlap or contamination limits.
  • CARDS-036 Link model training runs to architecture, important hyperparameters, training regime, compute settings, and reproducibility artifacts.
  • CARDS-037 Record when training details are unavailable for third-party closed models instead of inferring undisclosed facts.
  • CARDS-038 Document who verified supplied source or vendor claims and which provenance elements remain unconfirmed.
  • CARDS-039 Track origin of multimodal inputs, such as images, audio, video, and sensor data, where relevant to permissions and bias.
  • CARDS-040 Describe expected handling of withdrawn, corrected, expired, or disputed source data and its downstream derivatives.
  • CARDS-041 Publish data volume, modality distribution, label balance, and other useful composition statistics at a suitably safe aggregation level.
  • CARDS-042 Maintain traceable identifiers between model-card data claims, dataset-card entries, manifests, and underlying approved evidence.
  • CARDS-043 Verify published data examples and metadata do not expose protected records, hidden labels, or nonpublic evaluation cases.
  • CARDS-044 Explain differences between data that was used for training, retrieval at runtime, preference tuning, and evaluation.
  • CARDS-045 Describe data retention and restricted access to raw corpora, generated labels, model artifacts, and dataset exports.
  • CARDS-046 Document remaining provenance or licensing uncertainty as a material limitation with an owner and review status.

Evaluation and fairness disclosure#

  • CARDS-047 Specify each evaluation's task, metric definition, scoring direction, acceptance threshold, and tested model or dataset version.
  • CARDS-048 Describe benchmark dataset names, versions, sample sizes, sampling method, and permitted use.
  • CARDS-049 Document evaluation prompts, relevant system configuration, decoding settings, tools, retrieval sources, and test conditions.
  • CARDS-050 Distinguish model-only scores from end-to-end application results and from provider-reported benchmark claims.
  • CARDS-051 Show baseline or comparator scores where available so results can be interpreted in context.
  • CARDS-052 Report meaningful uncertainty, repeatability, confidence intervals, or limitations for variable and small-sample results.
  • CARDS-053 Disclose important severe errors and failure modes even if the average score meets an acceptance threshold.
  • CARDS-054 Describe subgroup performance or relevant differences across languages, cohorts, locations, and accessibility contexts when lawfully and validly evaluated.
  • CARDS-055 State when subgroup measurements were not performed or are unreliable due to limited sample sizes or inadequate labels.
  • CARDS-056 Report the rationale and limitations for collecting or inferring any sensitive attributes used in fairness analysis.
  • CARDS-057 Explain harmful-output, refusal, overblocking, privacy, prompt-injection, or tool-authorization tests appropriate to the actual capabilities.
  • CARDS-058 Document robustness results for malformed, ambiguous, rare, out-of-distribution, and adversarial examples where tested.
  • CARDS-059 Describe false-positive, false-negative, calibration, or abstention performance when those distinctions affect real-world risk.
  • CARDS-060 Include known dataset annotation uncertainties, class imbalance, missingness, and potential evaluation contamination.
  • CARDS-061 Assess whether data and metrics represent the deployed population rather than relying only on public leaderboard rankings.
  • CARDS-062 Record latency, memory, throughput, energy or cost observations only with relevant hardware and serving conditions.
  • CARDS-063 Explain how evaluation results differ across deployment variants, regions, provider snapshots, or quantized configurations.
  • CARDS-064 Label qualitative human-assessment findings with rubric, reviewer process, and potential evaluator bias.
  • CARDS-065 Link findings to authorized, versioned reports without exposing restricted test prompts or personal data.
  • CARDS-066 State mitigation status, accepted residual risk, and planned additional evaluation for unresolved limitations.
  • CARDS-067 Require review by responsible risk or domain specialists when the documented model affects consequential decisions.
  • CARDS-068 Update the disclosed evidence after material regressions or verified incidents rather than leaving contradicted claims in place.

Documentation and revision history#

  • CARDS-069 Store the canonical card in a discoverable repository, registry, or documentation system with a named maintenance owner.
  • CARDS-070 Bind each released card to an immutable model or dataset version rather than an untracked latest alias.
  • CARDS-071 Include creation, last-review, publication, effective, and supersession dates as relevant.
  • CARDS-072 State the audience and decide which content is public versus access-restricted by policy and contractual rights.
  • CARDS-073 Use a maintained card structure that separates overview, intended use, data, evaluation, risks, limitations, and contact channels.
  • CARDS-074 Populate required metadata such as language, license, task type, base model, or dataset references when the publishing platform supports them.
  • CARDS-075 Validate machine-readable metadata schema and links before publication, including YAML or JSON-LD syntax where applicable.
  • CARDS-076 Record the document source, reviewer, approval status, and exact release build to which the card applies.
  • CARDS-077 Keep technical claims aligned with evaluated configurations, recorded deployment versions, and reproducible experiment manifests.
  • CARDS-078 Define a change log showing what changed, why, who reviewed it, and when the new version became effective.
  • CARDS-079 Review cards after changes to model weights, tuning, data sources, collection rules, labels, supported uses, or policies.
  • CARDS-080 Add incident-driven warnings or revised limitations without waiting for a full model retrain where immediate disclosure is appropriate.
  • CARDS-081 Ensure changes to dataset versions and documentation propagate to linked model cards and published downstream records.
  • CARDS-082 Mark retired, withdrawn, deprecated, or replaced models and datasets clearly while preserving authorized archival history.
  • CARDS-083 Publish migration guidance or updated links when a model, endpoint, dataset, or evaluation protocol is superseded.
  • CARDS-084 Check accessibility, readable tables, descriptive links, and relevant localization for documentation used by external audiences.
  • CARDS-085 Verify that public download cards and discovery listings do not imply licenses, capabilities, or safety approvals not actually established.
  • CARDS-086 Document an intake process for correction requests, misuse reports, reproducibility questions, and provenance disputes.
  • CARDS-087 Provide appropriate limitations or unknown-status notation rather than silently omitting unavailable evidence.
  • CARDS-088 Retain previous approved card versions so reviewers can reconstruct what users were told at earlier release dates.
  • CARDS-089 Define a periodic review cadence proportionate to model change frequency, use-case risk, and data freshness.
  • CARDS-090 Have a qualified owner review the release-ready card for factual consistency, privacy, security, and lawful publication.

Release gate#

  • The released model and/or dataset has an identifiable owner, exact artifact version and approved current card with intended uses, out-of-scope uses and supported constraints.
  • Source, training and dataset provenance, relevant licensing or rights restrictions, transformations and open lineage gaps are documented without exposing protected records.
  • The card presents version-matched evaluation evidence, tested populations or modalities, material performance differences and known severe failure cases or uncertainty.
  • Meaningful safety, privacy, bias, accessibility and downstream misuse limitations have responsible mitigation or escalation guidance proportionate to risk.
  • Published metadata, references, model identifiers and dataset links have passed schema, access-control, consistency and link checks where relevant.
  • API and frontend surfaces expose appropriate, version-matched documentation, and correctly handle restricted, withdrawn or superseded cards.
  • The maintenance owner, change-triggered review, correction intake, revision history, publication controls and responsible reviewers are in place.
  • Significant missing rights, contradictory or misleading capability claims, unsupported high-impact uses or unresolved disclosure risks are blocked or formally dispositioned by qualified approvers.

Domain-specific approval rule: Block documentation approval if material claims cannot be reconciled to the actual model or dataset, a critical known limitation is concealed, a public card exposes protected information, or rights and usage restrictions remain unreviewed for the intended release. Publication does not certify a model, validate every provider claim or grant dataset-use rights.

Trusted references and scope#

The following primary specifications, publishing guides and responsible-AI references support practical model and dataset documentation. Use the parts applicable to the artifact and publication platform.

Applicability: Hugging Face model and dataset card metadata conventions apply when using that platform; an equivalent internal registry and documentation process can be appropriate for restricted artifacts. MLCommons Croissant 1.1 provides an optional machine-readable dataset metadata specification, including provenance, rather than a universal requirement. W3C PROV documents lineage structures but does not establish that training data was lawfully obtained. NIST AI RMF is voluntary guidance. Model Card Toolkit and vendor transparency notes are implementation examples, not evidence that an individual model is complete, unbiased, safe or certified. Never infer undisclosed third-party training details; document the uncertainty.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: required metadata, model/dataset versions, limitations, and documentation links; include model and dataset behaviors where applicable.
  • Cross-stack: markdownlint-cli2 + lychee.
  • Python: check-jsonschema + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: Ajv + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: Ajv + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • doc-coauthoring — Review completeness and clarity of published cards (anthropics/skills, 193,495 repo stars).
  • evaluation-methodology — Check metrics, limitations, and evaluation disclosures (wshobson/agents, 42,437 repo stars).
  • dataset-curation — Trace dataset sources, assumptions, and splits (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the AI-specific scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.