awesomechecklists
llms.txtGitHub ↗
/

33 — Analytics and Product Metrics Checklist#

Verify that product metrics answer meaningful decisions and are built on well-defined events, trustworthy data, privacy-aware collection, and sound experimentation. This template does not certify that a particular analytics vendor's reports or causal claims are correct.

Scope: Backend / API-specific. This file is one part of the 33_Analytics_and_ProductMetrics.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Critical product metrics and metric owners:
  • Analytics platforms / tracking plan version:
  • Consent and data-processing review owner:
  • Reviewer / owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Copy this shared checklist into a project-specific review; leave all repository checkboxes unchecked.
  2. Mark [x] only after verifying an item using approved metric definitions, a tracking plan, validation tests, reconciliation, representative reports, an experiment design, or authorized privacy decisions. Record failed items and valid N/A rationales in project notes/issues keyed by ANL ID rather than adding evidence tables here.
  3. Tailor measurement to actual product goals and lawful data collection. Not every service uses advertising attribution, browser analytics, monetization funnels, experiments, or GA4.
  4. Review ## API Checks and ## Frontend Checks independently when applicable; document N/A and why for missing product surfaces in project records.
  5. Revalidate after event-schema, consent, attribution, instrumentation, pipeline, identity, or metric-definition changes. Complete detailed Security, Legal, Data Governance, and Experimentation specialist reviews separately where needed.

Event taxonomy and implementation#

  • ANL-027 Separate an attempted action from a successful server-confirmed outcome so conversion metrics do not count failed attempts.
  • ANL-040 Test client and server event paths for offline use, delayed uploads, process restarts, and transient network failures.
  • ANL-038 Add event-schema and required-parameter checks to automated tests or controlled staging validation where feasible.
  • ANL-041 Monitor event ingestion, schema violations, duplicates, missing fields, and unexpected changes in volume after releases.
  • ANL-042 Version significant event schema changes and coordinate downstream dashboards, warehouses, and consumer migrations.

Data quality and reporting#

  • ANL-046 Compare client and server event counts for key journeys and investigate significant discrepancies.
  • ANL-065 Verify cost, cardinality, storage, and retention characteristics of analytics ingestion and reporting infrastructure.
  • ANL-047 Monitor source collection, ingestion, transformation, and dashboard freshness with documented acceptable delays.
  • ANL-048 Detect data gaps, schema drift, unexpected nulls, duplicate event IDs, and anomalous traffic before publishing key metrics.
  • ANL-064 Create regression tests for key transformations, aggregates, and conversion funnels affected by code or schema changes.
  • ANL-067 Inventory each analytics collector, SDK, pixel, cookie, server-side destination, and downstream data recipient.
  • ANL-072 Confirm declines, withdrawals, and preference changes propagate to applicable tags, SDKs, server-side collectors, and destinations.

Experimentation and decision quality#

  • ANL-089 Determine sample-size and runtime assumptions before launch, including the minimum detectable effect where applicable.
  • ANL-099 Check for novelty effects, seasonality, network effects, or interference between users when they could bias interpretation.
  • ANL-100 Review contradictory signals between the primary metric and guardrails before recommending deployment.

API Checks#

  • ANL-105 Define which business events must be emitted by authoritative server-side API operations rather than browser-only interactions.
  • ANL-106 Include stable event or transaction identifiers for deduplication of API analytics events where needed.
  • ANL-107 Emit completed transaction events only after the relevant durable business operation succeeds.
  • ANL-108 Ensure retries, timeouts, and idempotent API writes cannot accidentally inflate order, signup, or conversion totals.
  • ANL-109 Document how API failures, canceled operations, and partial outcomes differ from successful conversions.
  • ANL-110 Use consistent user/account identifiers and consent flags across server and client analytics within authorized bounds.
  • ANL-111 Validate analytics event schemas, required fields, and permitted values before dispatch to external collectors.
  • ANL-112 Keep analytics API secrets and provider credentials on trusted server infrastructure rather than in client code.
  • ANL-113 Set limits, retry budgets, and backpressure for telemetry dispatch so analytics failures cannot exhaust primary API capacity.
  • ANL-114 Do not block or roll back customer-critical API operations solely because an optional analytics sink is unavailable.
  • ANL-115 Define event delivery guarantees, deduplication, replay, and late-arrival behavior for asynchronous collection where used.
  • ANL-116 Propagate relevant deployment version, environment, and correlation metadata without attaching sensitive request bodies.
  • ANL-117 Confirm server-side events respect applicable consent or collection restrictions rather than assuming server origin removes those duties.
  • ANL-118 Test measurement endpoints, payload limits, and validation rules against the selected analytics vendor where applicable.
  • ANL-119 Monitor ingestion failure, rejection, sampling, queue lag, and invalid-event rates for critical API-origin events.
  • ANL-120 Reconcile important server-emitted metrics with source-of-truth database transactions and finance records.
  • ANL-121 Check that analytics exports, enrichment, and external destinations enforce appropriate access and data-retention boundaries.
  • ANL-122 Document API version changes that alter event names or business semantics and notify downstream metric owners.
  • ANL-123 Keep event timestamps, transaction ordering, and timezone conversions consistent across distributed API services.

Release gate#

  • Launch-critical business and user outcome metrics have stable definitions, owners, targets or baselines, and documented measurement limitations.
  • Critical events have an approved tracking plan, unique trigger semantics, appropriate identity and attribution rules, and validated representative client/server payloads.
  • Key conversion, transactional, or financial metrics reconcile with the authoritative system to an agreed tolerance, or material gaps have an accepted remediation plan.
  • Applicable consent, notices, data minimization, privacy controls, recipient access, and retention requirements have been reviewed and approved by the appropriate owner.
  • Analytics ingestion and dashboards are checked for freshness, missing events, duplicates, inappropriate test traffic, and known export or modeling limits.
  • Any launch-critical experiment has a documented hypothesis, decision metrics, randomization/health checks, and approved stop or rollout rules.
  • Responsible API/frontend owners confirm that relevant user journeys are measured correctly without disrupting functionality or collecting prohibited sensitive data.
  • Unresolved defects that could materially mislead product decisions, harm users, or breach applicable data-processing requirements have an owner, escalation path, and explicit authorized disposition.

Domain-specific approval rule: Do not approve measurement as release-ready when critical decisions depend on undefined or demonstrably incorrect metrics, required consent/collection rules are unaddressed, server-side business events systematically double count, or invalid experiment allocation is used to claim causality. Analytics review is not a substitute for legal/privacy compliance review or independent financial audit.

Trusted references and scope#

These checks translate source guidance into testable measurement and decision controls. Vendor-specific technical details apply only when that platform is used; these references are not a universal compliance standard.

Applicability: GA4 event names, API payload limits, and BigQuery export semantics are relevant only to GA4 implementations. UK tracking obligations depend on applicable law and current regulatory guidance, including changes made in 2026; seek qualified legal advice for jurisdiction-specific decisions. Online controlled experiments should be used only when methodologically appropriate. Consent-driven gaps, identity uncertainty, ad blockers, and sampling can affect measurement completeness and must not be ignored.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: event naming, consent, schemas, experiments, and reporting integrity; include API and service behavior where applicable.
  • Cross-stack: GA4 DebugView.
  • Python: Pandera + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: Zod + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: Ajv + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • analytics — Review event data and measurement plans (coreyhaines31/marketingskills, 59,899 repo stars).
  • ab-testing — Assess experiment controls and results (coreyhaines31/marketingskills, 59,899 repo stars).
  • kpi-dashboard-design — Check metric definitions and dashboards (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the backend / API scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.