awesomechecklists
llms.txtGitHub ↗
/

11 — AI Agents and Tool Orchestration Checklist#

Verify that AI agents achieve bounded tasks through permitted tools, safe handoffs, controlled state, and meaningful human supervision. The model may propose plans and actions, but independent application controls must enforce access, approvals, budgets, and actual execution.

Scope: AI-specific. This file is one part of the 11_AI_Agents_and_Tool_Orchestration.md checklist. Apply the domain-wide release gate across all applicable scope files; each checklist ID remains unique.

Review details#

  • Project / product:
  • Release / version / environment:
  • Scope and applicable jurisdictions / markets:
  • Agent tasks, delegated roles, and tool inventory:
  • Model / runtime / connector versions and execution environment:
  • Authorized users, approvals, permissions, and limits:
  • Review / security / operations owner:
  • Review date:
  • Decision: Pending / Approved / Approved with documented exceptions / Blocked

How to use#

  1. Copy this shared template into project-specific review records; leave all repository checkboxes unchecked.
  2. Define the agent's goal, scope, tool authority, user delegation, human-approval policy, and stop conditions before running tests or granting production access.
  3. Mark [x] only after verifying workflow diagrams, tool permissions, adversarial and failure-mode tests, approval traces, safe retry behavior, monitored execution limits, or qualified risk decisions. Track failures, evidence, accepted exceptions, and justified N/A in project records/issues keyed by AGENT ID rather than adding an evidence table here.
  4. Review ## API Checks and ## Frontend Checks separately where those surfaces exist; document N/A and why for headless or disconnected deployments.
  5. Do not assume particular SDK approval, session, or MCP protocol features apply to other frameworks. Treat model messages and tool outputs as untrusted; enforce permissions, schemas, and side-effect safeguards in the calling application or infrastructure.
  6. Reassess after changing tools, providers, delegated identities, workflow goals, model or policy versions, execution sandboxes, approval rules, or user-facing actions. Coordinate detailed red-team, governance, monitoring, and regression review with the other AI-specific templates.

Agent goals and workflow#

  • AGENT-001 Define the business task, intended user, expected outcome, and explicit stop conditions for each autonomous or semi-autonomous workflow.
  • AGENT-002 Determine whether a deterministic workflow, direct model call, or simple tool sequence would meet the need with lower risk than an agent.
  • AGENT-003 Document the boundaries between model-directed decisions and hard-coded workflow transitions.
  • AGENT-004 Identify an accountable owner for agent behavior, tool authority, service operations, and release approval.
  • AGENT-005 Map the permitted workflow states, transitions, termination paths, and externally visible side effects.
  • AGENT-006 Define what constitutes a successfully completed task, a partial result, an acceptable refusal, and an unrecoverable failure.
  • AGENT-007 List foreseeable misuse, ambiguous requests, unrealistic goals, and tasks the agent must not pursue.
  • AGENT-008 Document requirements for clarification, abstention, user confirmation, or qualified human escalation.
  • AGENT-009 Distinguish advisory outputs from actions that change external systems, send communications, or transfer value.
  • AGENT-010 Bound the agent's task scope to the authorized user's intent, target resources, and stated environment.
  • AGENT-011 Define which steps can safely execute automatically and which require deterministic checks or human intervention.
  • AGENT-012 Assign measurable thresholds for task completion, correctness, severe-error rate, and quality of the final artifact.
  • AGENT-013 Document model, instructions, tool definitions, memory behavior, orchestration library, and execution environment versions.
  • AGENT-014 Test the agent on representative multi-step tasks, including missing prerequisites and unexpected tool responses.
  • AGENT-015 Evaluate whether workflow decomposition into subtasks improves results without introducing harmful context or authority leakage.
  • AGENT-016 Define permitted specialist-agent roles and ensure their responsibilities are distinct and reviewable.
  • AGENT-017 Identify high-risk decisions, irreversible actions, and sensitive data transfers along the workflow.
  • AGENT-018 Document the sources of truth and validation required before an agent reports an external action as completed.
  • AGENT-019 Define how the agent detects a blocked task and stops instead of exploring unrelated targets or acquiring new permissions.
  • AGENT-020 Assess user harm, cost, latency, availability, and operational complexity created by autonomous execution.
  • AGENT-021 Provide a safe fallback to human or non-agent processing for unavailable models or prohibited tasks where necessary.
  • AGENT-022 Reassess agent suitability after changes to models, tool capabilities, connected accounts, customer populations, or workflow scope.

Tools, permissions, and boundaries#

  • AGENT-023 Inventory every callable tool and connector, including owner, purpose, provider, version, data sensitivity, and possible side effects.
  • AGENT-024 Classify tools by read-only, data-changing, externally communicating, privileged, and destructive capabilities.
  • AGENT-025 Allowlist the tools exposed to each agent role and workflow rather than giving every agent access to every integration.
  • AGENT-026 Validate tool names, argument types, required fields, ranges, resource identifiers, and payload sizes against server-owned schemas.
  • AGENT-027 Enforce object-, tenant-, account-, and operation-level authorization in the tool implementation regardless of the model's instructions.
  • AGENT-028 Bind delegated access to the authenticated initiating user or an explicitly approved service identity and purpose.
  • AGENT-029 Use narrowly scoped and expiring credentials when supported; prohibit unnecessary shared administrative tokens.
  • AGENT-030 Validate token issuer, audience, scopes, expiry, and resource binding for applicable OAuth or MCP-connected services.
  • AGENT-031 Prevent an agent from discovering, registering, or invoking unapproved tools through a model-generated URL or description.
  • AGENT-032 Review connector endpoints and server identities before trusting dynamically discovered tool metadata.
  • AGENT-033 Treat retrieved documents, tool results, webpage text, and other agents' messages as untrusted data rather than new instructions.
  • AGENT-034 Test prompt injection that tries to call higher-privilege tools, reveal secrets, or redirect the original user goal.
  • AGENT-035 Constrain filesystem, shell, browser, code-execution, and network tools to approved sandboxes, paths, hosts, and resources.
  • AGENT-036 Enforce egress and execution limits outside the model for agents that can make network calls or run code.
  • AGENT-037 Use separate credentials and isolation for development, testing, staging, and production tool endpoints.
  • AGENT-038 Prevent agents from creating or forwarding credentials or accessing secrets outside the permitted task.
  • AGENT-039 Validate external tool results and reject malformed, oversized, incompatible, or untrusted outputs before downstream use.
  • AGENT-040 Ensure model-generated SQL, scripts, requests, and commands cannot bypass independent policy, parameterization, or sandbox checks.
  • AGENT-041 Review cross-tool data transfers for privacy, licensing, data residency, and user permissions.
  • AGENT-042 Test confused-deputy scenarios where an agent has broader authority than the initiating user.
  • AGENT-043 Require distinct permissions and approvals for export, deletion, money movement, publishing, or other consequential actions.
  • AGENT-044 Implement tool-specific timeouts, resource ceilings, concurrency limits, and throttling to prevent runaway consumption.
  • AGENT-045 Check that denial or revocation of tool access takes effect in active and resumed workflows.
  • AGENT-046 Keep useful tool-call audit records with actor, action, target, policy decision, and result while redacting protected content.
  • AGENT-047 Review tool inventory and permissions after provider changes, connector upgrades, policy changes, and new agent capabilities.
  • AGENT-048 Do not treat tool descriptions, system prompts, output filters, or human-friendly warnings as substitutes for enforceable access controls.

Handoffs, state, and execution limits#

  • AGENT-049 Define a documented routing policy for choosing between agents, specialists, and deterministic workflow steps.
  • AGENT-050 Restrict each handoff to approved destinations and validate any model-supplied destination or routing metadata.
  • AGENT-051 Reevaluate authorization before handoff side effects or delegated tool actions occur.
  • AGENT-052 Filter transferred conversation history and context to the minimum information the receiving agent is authorized to see.
  • AGENT-053 Test that tenant identity, user permissions, and consent boundaries survive multi-agent routing and nested agents.
  • AGENT-054 Distinguish model-generated handoff metadata from trusted server-side state and dependencies.
  • AGENT-055 Preserve stable workflow/run IDs, parent-child task IDs, and correlation identifiers across tool calls and handoffs.
  • AGENT-056 Define explicit maximum turns, tool calls, nested handoffs, wall-clock duration, token usage, and spend per run.
  • AGENT-057 Detect recursion, repetitive tool loops, oscillating handoffs, and lack of task progress before limits are exhausted.
  • AGENT-058 Ensure cancellation propagates to pending model calls, queued tool work, child agents, and other resources where feasible.
  • AGENT-059 Test the treatment of in-flight side effects when a run is canceled, times out, or loses its initiating session.
  • AGENT-060 Persist only necessary state and ensure session memories and checkpoints have documented retention and access policies.
  • AGENT-061 Keep serialized agent state and pending approvals in trusted storage with ownership, integrity, and replay protections.
  • AGENT-062 Verify that restoring a checkpoint does not grant permissions that were revoked after it was created.
  • AGENT-063 Document how concurrent agents coordinate shared state and prevent races, duplicate writes, or contradictory updates.
  • AGENT-064 Use operation identifiers and deduplication when retries or resumed workflows could replay non-idempotent tool calls.
  • AGENT-065 Test network interruptions, worker restarts, queue delays, and partial tool failures for recoverable continuation.
  • AGENT-066 Define what happens when an intermediate agent has completed work but the final synthesis or response fails.
  • AGENT-067 Prevent stale memory, wrong-user history, or previous-task instructions from contaminating a new task.
  • AGENT-068 Enforce input/output bounds and validation on messages sent between agents, including untrusted structured content.
  • AGENT-069 Record whether completed tool actions are confirmed from the authoritative service or only inferred by a model.
  • AGENT-070 Define a policy for partial completion, compensating actions, rollback, and escalation when workflows span systems without a shared transaction.
  • AGENT-071 Monitor per-run step count, handoffs, resource usage, failure patterns, retries, and unusual tool sequences.
  • AGENT-072 Test unsupported tasks and unreachable targets to ensure the agent stops within scope rather than improvising unsafe alternatives.

Human approval and failure recovery#

  • AGENT-073 Define the tool actions and risk conditions that require human approval before execution.
  • AGENT-074 Present an authenticated reviewer with the exact proposed action, target, affected records, and material consequences.
  • AGENT-075 Bind each approval to a specific run, pending tool call, immutable argument set, and authorized decision maker.
  • AGENT-076 Reject altered, expired, duplicated, forged, or unbound approval responses before any privileged action occurs.
  • AGENT-077 Revalidate current permissions and action arguments immediately before executing a previously approved tool call.
  • AGENT-078 Limit approval lifetime and scope; do not silently reuse blanket approval across different tools, accounts, or privilege contexts.
  • AGENT-079 Provide an explicit rejection path and ensure rejected actions never execute later because of stale state.
  • AGENT-080 Define the behavior when an approval request times out, the reviewer disconnects, or the user cancels the task.
  • AGENT-081 Keep pending approvals and approval records in trusted server-side storage rather than trusting client-supplied snapshots.
  • AGENT-082 Test approvals arising from nested or handed-off agents rather than only top-level calls.
  • AGENT-083 Require independent confirmation for destructive changes, external publication, payments, high-impact decisions, and other risk-defined actions.
  • AGENT-084 Ensure reviewers can inspect enough trusted context to challenge model assumptions without exposing unauthorized data.
  • AGENT-085 Show a clear difference between an approval request, an approved request, an action in progress, and a confirmed action.
  • AGENT-086 Design safe recovery when the application crashes after a side effect but before persisting completed run state.
  • AGENT-087 Use idempotency keys, authoritative status checks, and reconciliation for retried actions; do not assume approvals imply exactly-once execution.
  • AGENT-088 Document compensating actions and escalation contacts for partial failures that cannot be rolled back atomically.
  • AGENT-089 Provide a tested emergency stop that disables agent execution and revokes tools or credentials as appropriate.
  • AGENT-090 Test that emergency stops constrain already queued and independently running child tasks.
  • AGENT-091 Record approval, denial, execution, partial failure, and remediation events with safe identifiers for audit and incident review.
  • AGENT-092 Train authorized reviewers on automation bias, tool effects, and the limits of model-proposed explanations.
  • AGENT-093 Provide an escalation path for ambiguous, disputed, or consequential outcomes requiring human judgment.
  • AGENT-094 Validate the complete approval/recovery flow in staging with failures, replay attempts, denied requests, and interrupted resumes.

Release gate#

  • The agent's business purpose, task boundaries, permitted tools, authorized user population, run limits, and responsible owners are documented and reviewed.
  • Tool and connector inventory, least-privilege scopes, tenant boundaries, schema validation, and external execution/network restrictions have been tested against unauthorized access and tool misuse.
  • Representative end-to-end tasks meet agreed outcome and severe-error criteria, including blocked objectives, malformed tool data, and adversarial indirect instructions.
  • Routing and handoff tests confirm that specialist agents cannot gain extra privileges, improperly disclose context, or bypass normal approval and resource limits.
  • High-impact tool calls require authenticated, action-specific human approval bound to exact parameters; rejected, expired, changed, or replayed approvals do not execute.
  • Cancellation, shutdown, crash recovery, duplicate delivery, and partial external side effects have tested safe outcomes or a documented compensating/escalation procedure.
  • API and frontend consumers can distinguish proposed actions from confirmed execution and handle pauses, refusals, errors, and partial completion safely.
  • Monitors, alerts, incident response, execution traces, cost and throughput limits, and emergency tool revocation are operational with named responders.
  • Material unresolved authority, user-harm, privacy, prompt-injection, or side-effect risks are resolved or have an explicitly authorized and time-bound disposition.

Domain-specific approval rule: Block sign-off if an agent can reach unauthorized data or tools, autonomously cross approved scope, execute material unapproved side effects, resume with forged authority, or replay irreversible actions without accepted controls. Passing the checklist does not imply that agents are infallible, that actions are exactly once, or that a specific security or regulatory certification has been achieved.

Trusted references and scope#

The following primary and authoritative sources inform these practical review checks. Their SDK-specific details should be mapped to the actual framework and deployment environment rather than assumed to apply universally.

Applicability: MCP tool discovery is not permission to invoke every available function. Its November 2025 authorization specification is optional for some transports and implementations, and newer drafts may differ. OpenAI and Microsoft agent approvals must be implemented with server-owned decisions and current permission checks; resuming a previously approved run does not ensure exactly-once external side effects. OpenTelemetry conventions describe instrumentation but do not justify storing protected prompts or tool payloads. OWASP risk lists and NIST AI RMF practices support threat modeling and governance rather than certifying deployment safety. Anthropic's 2024 orchestration article carries an explicit notice that tooling has evolved.

Source review date: 2026-11-12.

Suggested testing tools#

Choose tools appropriate to the project's stack and risks; automation supports evidence but does not approve the checklist.

  • Test focus: tool permissions, budgets, retries, state handoffs, and high-risk approval gates; include model and dataset behaviors where applicable.
  • Cross-stack: promptfoo + OpenTelemetry.
  • Python: LangGraph testing + pytest; for pytest-based suites, run python -m pytest -q.
  • TypeScript: promptfoo + Vitest; for Vitest-based suites, run npx vitest run (or npx playwright test for E2E).
  • JavaScript: promptfoo + Jest; for Jest-based suites, run npx jest --ci (or npx playwright test for E2E).
  • Evidence: Save tool versions, configuration, test environment, fixtures, results, and links in project-specific review records using the existing checklist IDs. Expert review remains necessary where relevant.

Suggested AI skills for verification#

These are optional agent review aids, not automated passes or release approval. Each linked repository had at least 1,000 GitHub stars when checked on 2026-11-12; counts are per repository, not per skill, and do not indicate security or correctness.

  • agent-owasp-compliance — Review agentic threat and tool-abuse scenarios (github/awesome-copilot, 41,073 repo stars).
  • workflow-orchestration-patterns — Check agent state, retries, and handoffs (wshobson/agents, 42,437 repo stars).
  • agentic-eval — Test bounded tool outcomes and regressions (github/awesome-copilot, 41,073 repo stars).

How to check: In a project-specific copy, have the agent audit the AI-specific scope and map findings to this file's existing checklist IDs, with code/configuration references, reproducible test results, failed or inapplicable cases, and uncertainties. Independently verify findings; keep shared template boxes unchecked and retain human approval requirements.

Before use: Read the skill's SKILL.md, referenced scripts, license, and requested tool permissions. Only execute vetted skills with appropriate access in a controlled environment.

Important warning: reference only#

⚠️ Reference only — not a definitive or exhaustive checklist. This template is a starting point for review, not the ultimate authority or a guarantee of project safety, quality, regulatory compliance, accessibility, security, or launch readiness.

Use project-specific judgment and independent verification. Requirements and risks evolve and differ by project, technology, and jurisdiction. Adapt the checks, consult current authoritative sources, perform relevant tests, and obtain qualified specialist review or approval when appropriate. Checking every box does not replace these steps.

Reference, not a certification. Check applicability, verify important claims against current primary sources, and seek expert review where necessary. Suggest an improvement.