awesomechecklists
llms.txtGitHub ↗
/

19 — Multimodal AI Quality Checklist#

Verify that AI features using images, audio, speech, video, documents, or combinations of modalities produce fit-for-purpose, safe, accessible, and properly grounded results. An AI model's claim to support a modality does not establish accuracy for a specific task, language, device, input quality, or deployment configuration.

Scope: Frontend-specific. This file is one part of the 19_Multimodal_AI_Quality.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Supported input / output modalities and intended tasks:
  • Model/provider, media preprocessing, and evaluation versions:
  • Quality, accessibility, safety, privacy, latency, and cost targets:
  • Reviewer / owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Create a project-specific copy of this reusable checklist; leave all checkboxes in the repository template unchecked.
  2. Scope checks to the actual supported modalities, formats, audiences, languages, and product workflows. Image-only, speech-only, video, and cross-modal systems require different test fixtures and quality measures.
  3. Verify applicable checks using representative approved test media, independent reference judgments, actual deployed endpoints, preprocessing configurations, end-to-end user tests, privacy/security reviews, and accessibility assessments. Record failed checks, evidence, risk exceptions, and justified N/A outside this shared template using the MULTI ID.
  4. Keep ## API Checks and ## Frontend Checks separate, documenting inapplicability when a surface does not exist.
  5. Separate extraction/transcription, reasoning, generation, transport, playback, and presentation errors. A usable transcript, credible image, or plausible video summary does not prove downstream factual or safety correctness.
  6. Repeat representative multimodal and accessibility evaluations after changes to model versions, media transforms, codecs, cropping, frame sampling, provider capabilities, streaming logic, policy, or user interfaces. Coordinate deeper AI Evaluation, Red Teaming, Data Curation, and Accessibility reviews with those dedicated templates.

Frontend Checks#

  • MULTI-115 Clearly indicate which image, audio, video, and file types can be used and their actual current size and duration limits.
  • MULTI-116 Give users accessible upload, recording, preview, replacement, and removal controls before submission.
  • MULTI-117 Explain what media will be processed or shared with an external AI provider and request any required consent.
  • MULTI-118 Show meaningful recording, uploading, transcribing, generating, buffering, and completed states.
  • MULTI-119 Provide accessible and understandable error messages for malformed, unsupported, too-large, or disconnected media.
  • MULTI-120 Allow users to correct transcription errors and clarify visual interpretation before consequential external actions.
  • MULTI-121 Mark partial captions, provisional transcripts, and generated previews as unverified until finalized where appropriate.
  • MULTI-122 Provide required captions, text transcripts, audio descriptions, and alternative access to media content.
  • MULTI-123 Test keyboard, focus, screen reader, caption controls, and audio player behavior across supported platforms.
  • MULTI-124 Ensure generated alt text and captions are accurate enough for their purpose or provide a correction mechanism.
  • MULTI-125 Keep audio and visible transcript synchronized when possible and explain limitations where exact alignment is unavailable.
  • MULTI-126 Implement usable microphone and camera permission-denied states and alternatives to voice-only interactions.
  • MULTI-127 Handle interruption, pause, cancellation, playback stop, and session reconnection without duplicated or misleading completion.
  • MULTI-128 Display visual source thumbnails or source-time references only when the user is authorized to see them.
  • MULTI-129 Prevent unsafe rendering of user-supplied SVG/HTML or model-generated markup, scripts, and media links.
  • MULTI-130 Test responsive image, video, subtitles, long transcripts, and audio controls under text zoom and mobile layouts.
  • MULTI-131 Review multilingual, right-to-left, accented speech, and assistive-device behavior for the supported user population.
  • MULTI-132 Disclose AI-generated image, voice, or synthetic video when required and avoid implying it is human-authored or verified.
  • MULTI-133 Protect local previews, cached media, transcription history, and analytics against account switching and unauthorized sharing.
  • MULTI-134 Collect privacy-conscious user feedback on inaccurate recognition, missing captions, harmful outputs, and inaccessible workflows.

Release gate#

  • Every advertised modality, combination, language, file format, and input limit is supported by the exact deployed model/API/client configuration and correctly documented.
  • Representative image/video, audio/speech, and supported cross-modal tasks pass predeclared factuality, fidelity, temporal alignment, and user-task quality criteria.
  • Low-quality input, unsupported metadata, missing evidence, ambiguous media, streaming failures, and incomplete results trigger tested safe fallback, abstention, or clarification behavior.
  • Important privacy, consent, source-use rights, synthetic-media disclosures, and data lifecycle requirements have appropriate qualified approval.
  • Unsafe media content and embedded instructions cannot bypass independent access controls, moderation, tool authorization, or protected data boundaries.
  • Required text alternatives, transcripts, captions, visual descriptions, accessible controls, and supported assistive-technology journeys have been checked for the actual content and applicable WCAG requirements.
  • Production-equivalent API and frontend tests verify format validation, size limits, media handling, cancellation, latency/cost budgets, and safe completion or failure states.
  • Accountable owners approve relevant versioned evaluation results, known modality gaps, remaining high-impact risks, rollback paths, and the exact release.

Domain-specific approval rule: Block release where critical media interpretation or synthesis is materially misleading, protected recordings or images can be exposed, unsafe embedded media instructions enable unauthorized actions, or necessary accessible alternatives are missing for a supported consequential workflow without approved mitigation. Passing individual modality benchmarks does not prove correct cross-modal reasoning or regulatory compliance.

Trusted references and scope#

The primary technical documentation and accessibility standards below inform project-specific verification. These sources cover different vendors and media modalities; use only the standards and provider features applicable to the deployed system.

Applicability: OpenAI vision, audio, transcription, and file-input capabilities differ by model, endpoint, and revision; verify current formats, quotas, metadata, and any deprecation dates instead of copying them across models. As of this source review, OpenAI documentation lists a February 26, 2027 shutdown for certain legacy transcription models and a January 6, 2027 shutdown for several legacy TTS models, so new deployments should check current alternatives. Amazon Bedrock multimodal indexing and retrieval are also workflow- and provider-specific; some modes do not search speech or support mixed text/image queries. WCAG 2.2 obligations depend on media type, live/prerecorded status, conformance level, and jurisdiction. NIST AI RMF guidance is voluntary and not a safety certification. Do not use automated visual or audio inference as unreviewed proof for consequential identity or health decisions.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: image/audio/video fixtures, timing, captioning, invalid media, and accessibility; include browser and client behavior where applicable.
  • Cross-stack: FFmpeg + Playwright.
  • Python: Pillow + librosa + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: Playwright + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: Playwright + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • caption-qa — Assess captions and media accessibility (github/awesome-copilot, 41,073 repo stars).
  • remotion-captions — Check video caption workflows when applicable (remotion-dev/skills, 6,895 repo stars).
  • webapp-testing — Exercise media upload, rendering, and failures (anthropics/skills, 193,495 repo stars).

How to check: In a project-specific copy, have the agent audit the frontend scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.