27 — Performance and Scalability Checklist#

Verify that priority user experiences and services meet documented latency, throughput, capacity, and efficiency objectives at realistic demand. Separate browser metrics from backend metrics; avoid treating a single benchmark as production capacity proof.

Scope: Backend / API-specific. This file is one part of the 27_Performance_and_Scalability.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

• Project / product:
• Release / version / environment:
• Scope and applicable jurisdictions / markets:
• Target workloads, user cohorts, and devices:
• Performance budgets / SLO references:
• Reviewer / owner:
• Review date:
• Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

1. Copy this template into project-specific review records; leave the shared template unchecked.
2. Mark [x] only after verifying a check through a reproducible test, representative field measurement, resource profile, capacity model, or documented decision. Record failed items and N/A reasons outside this template, keyed by PERF ID.
3. Use realistic traffic mixes, data sizes, network/device conditions, and explicit percentiles. Document environment limitations and do not use synthetic tests alone to claim production performance.
4. Review API and frontend performance separately where relevant. An API-only service or static page may make some checks inapplicable; document why in project records.
5. Revisit budgets and tests after workload growth, architecture changes, large frontend changes, or major dependency updates. Coordinate functional correctness, security, reliability, accessibility, and cost reviews with their dedicated checklists.

Performance budgets and measurements#

• PERF-004 Identify production and representative test populations, workloads, regions, devices, network conditions, and data volumes.

Backend throughput and capacity#

• PERF-024 Include cold-start and warm-start scenarios when runtime initialization or autoscaling affects latency.

• PERF-025 Measure saturation of CPU, memory, disk I/O, network, thread pools, and connection pools where relevant.

• PERF-026 Evaluate query execution plans and measured query timings for high-cost database reads and writes.

• PERF-030 Test critical cache invalidation or stale-data conditions where optimization can affect correctness.

• PERF-031 Measure queue depth, consumer lag, job duration, and retry volume for asynchronous workloads.

• PERF-033 Verify expensive work is canceled or bounded when clients disconnect or request deadlines expire.

• PERF-017 Model the mix of reads, writes, background work, and high-cost operations observed or forecast in real workloads.

• PERF-018 Include authenticated roles, tenant sizes, object sizes, and representative query parameters in load profiles.

• PERF-019 Execute a smoke test of the performance harness before conducting larger runs.

• PERF-020 Test expected steady-state load and compare response times, throughput, and errors with agreed thresholds.

• PERF-021 Test anticipated peak traffic and sudden bursts, including connection establishment and authentication overhead.

• PERF-022 Run stress tests to identify the point at which latency, errors, or resource saturation exceed acceptable limits.

• PERF-023 Run soak/endurance tests long enough to expose memory leaks, connection leaks, fragmentation, or accumulating queues.

• PERF-027 Check indexing, query selectivity, pagination, and avoidance of unnecessary repeated queries in high-volume paths.

• PERF-028 Measure transaction lock contention, deadlocks, and concurrency effects for write-heavy workflows.

• PERF-029 Confirm caches have suitable keys, eviction/expiry rules, hit rates, and behavior during cold starts or failures.

• PERF-032 Exercise timeouts, bounded retries, backpressure, rate limiting, and load shedding under dependency saturation.

• PERF-034 Test graceful degradation and recovery after overload without indefinite backlog growth.

• PERF-035 Measure performance of external dependencies and enforce contractual or permission limits during tests.

• PERF-036 Compare production capacity forecasts with observed usage and prior load-test results.

Client performance#

• PERF-041 Record page-load request count, transferred bytes, decoded resource size, and major blocking dependencies.
• PERF-042 Measure time to first byte and the contribution of redirects, DNS, connection setup, and backend work to initial loading.

Scaling and efficiency#

• PERF-063 Model the cost per request, transaction, active user, or job at expected and peak demand.

• PERF-053 Estimate capacity needs using peak forecasts, observed growth, seasonal demand, and planned launches.

• PERF-054 Document scaling units, bottleneck resources, quota limits, and the conditions triggering additional capacity.

• PERF-055 Validate autoscaling thresholds and scale-out/scale-in behavior under realistic traffic changes when autoscaling is used.

• PERF-056 Account for provisioning and warm-up delays so scaling begins early enough to protect user experience.

• PERF-057 Check that stateless services can add or remove instances without losing essential user or workflow state.

• PERF-058 Test load-balancer distribution, connection pooling, and uneven traffic/hot-key conditions.

• PERF-059 Maintain headroom for expected failures or maintenance consistent with the service's availability objectives.

• PERF-060 Evaluate regional latency and content delivery paths where users are geographically distributed.

• PERF-061 Check that caches and CDNs improve latency without serving unauthorized, private, or misleadingly stale content.

• PERF-062 Verify data partitioning, replication, and storage scaling strategies for projected dataset growth if applicable.

• PERF-064 Compare right-sizing, caching, batching, compression, and concurrency tuning using measured outcomes.

• PERF-065 Ensure background processing cannot starve customer-facing requests or saturate shared resources.

• PERF-066 Test how system behavior changes when a dependency is throttled or unavailable, including backlog recovery.

• PERF-067 Assess sustainable resource use, including idle overprovisioning and unnecessary repeated computation.

• PERF-068 Define alerts for capacity exhaustion, budget violations, unusual resource growth, and deteriorating tail latency.

• PERF-069 Review and update performance forecasts after release or major changes to user demand or architecture.

API Checks#

• PERF-070 Document traffic profiles and budgets for each critical API endpoint, operation class, and consumer.
• PERF-071 Measure p50, p95, and p99 latency together with requests per second and status-code distributions under representative load.
• PERF-072 Test authorization and authenticated workloads instead of benchmarking only unauthenticated happy paths.
• PERF-073 Include realistic payload sizes, nested objects, query combinations, and large-response cases in API benchmarks.
• PERF-074 Validate that pagination, filtering, and sorting prevent unbounded response sizes and excessive database work.
• PERF-075 Measure effects of serialization, compression, schema validation, and gateway middleware on response times.
• PERF-076 Identify N+1 requests, excessive fan-out, and chained downstream calls on latency-critical operations.
• PERF-077 Enforce and verify request deadlines, upstream timeouts, concurrency limits, and cancellation propagation.
• PERF-078 Test rate limits, quotas, backpressure, and appropriate retry guidance under controlled overload.
• PERF-079 Confirm that retries and idempotency do not amplify server load or duplicate expensive actions.
• PERF-080 Benchmark bulk APIs, asynchronous jobs, and streaming endpoints using appropriate throughput and completion metrics.
• PERF-081 Check connection pooling, keep-alive behavior, and TLS overhead where they materially affect workload capacity.
• PERF-082 Verify API cache semantics, private response isolation, cache-key correctness, and safe invalidation.
• PERF-083 Test sustained traffic to detect queue growth, memory retention, and connection exhaustion.
• PERF-084 Ensure performance tests exclude non-owned third-party services unless permission and safe limits have been agreed.
• PERF-085 Observe endpoint-level latency, error rates, saturation, and dependencies during staged releases.
• PERF-086 Verify public API contract changes do not silently alter pagination limits, timeout behavior, or payload-size expectations.
• PERF-087 Confirm heavy requests and expensive queries cannot exhaust shared capacity at the expense of other consumers.

Release gate#

• Performance budgets, workload assumptions, target cohorts, and accountable owners are defined for the release's critical journeys and endpoints.
• Representative baseline and relevant peak-load tests meet documented latency, throughput, error-rate, and saturation criteria, with acknowledged test-environment limitations.
• Major backend or database bottlenecks, cache behavior, and overload/recovery paths are measured and either addressed or accepted with authorized risk ownership.
• Applicable frontend routes have measured Core Web Vitals or equivalent user-centric measures, and critical interaction regressions have been reviewed.
• Expected peak/growth demand has a documented capacity plan, scaling constraints, monitoring signals, and safe response to saturation.
• No unresolved critical performance failure jeopardizes core tasks or overload recovery without an explicit, time-bound acceptance and mitigation plan.

Domain-specific approval rule: Block performance sign-off when material latency or capacity targets are missing, critical workflows breach agreed thresholds without approved mitigation, overload causes uncontrolled failure, or regressions remain unowned. Benchmarks and Web Vitals are context-dependent; meeting a public threshold is not a universal performance guarantee.

Trusted references and scope#

These actionable checks are informed by the following primary engineering and standards sources; they do not reproduce the source publications or imply certification.

• AWS Well-Architected Framework — Performance Efficiency Pillar — workload selection, load testing, resource efficiency, and continual improvement.
• AWS Well-Architected — Performance process and culture — measurement, automated performance tests, and performance tuning.
• Google SRE — Production Services Best Practices — forecasting, measured capacity, and overload behavior.
• Google SRE — Handling Overload — backpressure, throttling, and overload defenses.
• Grafana k6 — Load Testing Websites — realistic traffic profiles, backend versus browser testing, and test scope.
• Grafana k6 — Automated Performance Testing — smoke, average-load, stress, spike, and soak test planning.
• web.dev — Web Vitals — LCP, INP, CLS, 75th-percentile thresholds, and field measurement.
• web.dev — Learn Performance — loading, code splitting, image loading, and client responsiveness.
• OpenTelemetry — Metrics — request latency, throughput, resource utilization, and telemetry cardinality.
• PostgreSQL — Using EXPLAIN — query planning and database bottleneck analysis when PostgreSQL is used.
• MDN — HTTP Cache-Control — browser and shared cache semantics, freshness, and private responses.

Applicability: Apply web-specific metrics only to applicable web experiences. Current commonly used Core Web Vitals 'good' thresholds are LCP at or below 2.5 seconds, INP at or below 200 milliseconds, and CLS at or below 0.1, assessed at the 75th percentile of real-user page loads for mobile and desktop separately; verify current official guidance when reviewing. PostgreSQL, cloud-native autoscaling, CDN, and asynchronous queue checks apply only if the architecture uses them. Never perform high-volume tests against infrastructure or third-party services without authorization.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

• Test focus: throughput, latency percentiles, browser responsiveness, and resource budgets; include API and service behavior where applicable.
• Cross-stack: k6 + Lighthouse.
• Python: Locust + pytest-benchmark + pytest; for pytest-based suites, run python -m pytest -q.
• TypeScript: Playwright + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
• JavaScript: autocannon + Lighthouse CI + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
• Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

• performance-optimization — Review profiling, budgets, and bottlenecks (addyosmani/agent-skills, 116,378 repo stars).
• web-perf — Audit web performance when deployed on the web (cloudflare/skills, 3,349 repo stars).
• python-performance-optimization — Examine Python service hotspots where applicable (wshobson/agents, 42,437 repo stars).

How to check: In a project-specific copy, have the agent audit the backend / API scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.