ENGINEERING CONTROL SELF-AUDIT

AI Agent Production Readiness Checklist

A free 50-item self-audit across five engineering control dimensions, with no signup. Record verified controls, review outstanding release dependencies, and copy a local Markdown issue draft. This self-assessment is not a release approval.

Scope: designed for agents that call external tools or handle sensitive data, especially in containerized deployments. Check an item only after verifying its control objective in your architecture; commands and numeric thresholds are examples, not universal requirements. If a control does not apply, leave it unchecked and document the reason separately; this checklist has no N/A state.

Release Dependencies: Review Not Started

Verified 0 of 50 controls. Check verified guards to identify dependencies for engineering follow-up.

Filter:

Showing 50 items

Jump: §1 Permissions §2 Sandboxes §3 Anti-Injection §4 Circuit Breakers §5 Human & Rollback

📦 FREE PUBLIC STARTER • PYTHON & DEEPEVAL

Looking to implement these 50 controls in your codebase?

Clone the public starter: deterministic circuit breakers, tool allowlists, recognized path-argument checks, and CI regression tests with explicit limits.

Explore Starter Kit →

Section 01

Permission, Tool & Credential Scoping

Hard constraints on what tools and data the model is physically permitted to interact with.

[P0 Blocking] Ban Unbounded Shell Execution: No generic bash, sh, cmd, or exec tools exposed to the agent.

[P0 Blocking] Ban Arbitrary SQL Execution: No raw SQL execution tools. All database interactions use pre-parameterized, typed endpoints.

[P0 Blocking] Physical Database Credential Separation: Read tools connect to read-only replicas; write tools authenticate through an isolated gateway with zero admin rights.

[P0 Blocking] Static Parameter Schemas: All tool definitions enforce strict JSON Schema validation (enums, strict types, length bounds) before execution.

[P0 Blocking] Ephemeral Credential Injection: Zero static API keys or cloud tokens hardcoded in prompts or code; use short-lived STS / IAM tokens.

[P1 High] Environment Secret Scrubbing: Process environment variables (.env, AWS_SECRET_ACCESS_KEY) are scrubbed before launching tool processes.

[P1 High] Idempotency Keys: All state-mutating tool invocations require deterministic idempotency keys to prevent duplicate actions on retry.

[P1 High] Deny-by-Default Tool Policy: Agents possess zero tools by default; tools are explicitly granted per session based on user permissions.

[P1 High] Parameter Allowlisting: Tool dispatchers strictly reject payloads containing unrecognized or extraneous properties.

[P2 Medium] Rate & Concurrency Caps: Hard limit on the frequency of tool calls per user and per tenant to prevent resource exhaustion.

Section 02

Sandbox, Egress & Environment Containment

Isolating execution environments to contain host escapes and data exfiltration.

[P0 Blocking] Ephemeral Runtime Environments: Tools run inside single-task containers or microVMs (e.g. Firecracker/gVisor) destroyed immediately upon completion.

[P0 Blocking] Default-Deny Network Egress: Sandbox outbound networking is blocked by default; zero direct connection to arbitrary public IP addresses.

[P0 Blocking] Strict DNS Egress Allowlisting: Outbound traffic is restricted via firewall to a minimal whitelist of verified internal services and package registries.

[P0 Blocking] Block Cloud Metadata Endpoints: Host metadata service (169.254.169.254) is blocked at the network interface to prevent IAM role theft.

[P1 High] Read-Only Root Filesystem: Worker container root filesystem mounted with --read-only flag; writable storage restricted to ephemeral memory mounts.

[P1 High] Bounded Memory Scratchpad: Bound temporary writable storage and prevent execution where supported. For containerized workers, tmpfs with noexec and a 50MB quota is one example; size the limit from peak workload and verify quota enforcement and cleanup.

[P1 High] Non-Root Process Execution: Agent and tool processes run without root privileges or sudo. A high-numbered UID is common in containers, but verify the effective identity and blocked privilege escalation rather than relying on a specific UID.

[P1 High] Linux Capability Stripping: All default Linux capabilities dropped (cap-drop=ALL) with strict seccomp profile enabled.

[P2 Medium] Zero Cross-Session Persistence: No files, caches, or environment state retained across tasks or shared between users.

[P2 Medium] Host Path Mount Isolation: Absolute prohibition against mounting developer home directories, .ssh, .gitconfig, or root folders.

Section 03

Ingress Sanitization & Anti-Injection Defenses

Neutralizing indirect prompt injections and protecting model context.

[P0 Blocking] Physical Instruction / Data Separation: External data is wrapped in structured envelopes with random nonces and never concatenated into system prompts.

[P0 Blocking] Automated Secret & Credential Redaction: Inbound prompts and retrieved context pass through high-speed regex/NLP redactors to strip PII and credentials.

[P0 Blocking] Parser Output Sanitization: Web scraper and PDF parser output stripped of hidden control characters, markdown link exploits, and script tags.

[P0 Blocking] System Prompt Immutability: The agent cannot alter its own system prompt, tool definitions, or safety parameters through tool outputs.

[P1 High] Maximum Context Window Thresholds: Hard limits on ingested context length to prevent context-stuffing denial of service and memory overflow.

[P1 High] Tool Metadata Poisoning Defense: Tool names and schema descriptions are statically defined at build time; dynamic user input in tool schemas is banned.

[P1 High] Authenticated Ingestion Webhooks: Inbound webhooks require cryptographic signature verification (HMAC SHA256) before reaching agent context.

[P1 High] Structured LLM Output Enforcement: Model responses must conform to strict JSON schemas; malformed responses trigger deterministic re-prompting.

[P2 Medium] Automated Adversarial CI Regression: CI runs automated test suites for indirect prompt injection and jailbreak resistance on every prompt/model change.

[P2 Medium] Output Safety Sanitization: Outbound text sent to customers is checked for leaked system instructions, internal IDs, and raw stack traces.

Section 04

State Machine, Quotas & Circuit Breakers

Preventing recursive loops, infinite retries, and runaway compute bills.

[P0 Blocking] Deterministic Step Ceiling: An external supervisor enforces a finite reasoning/tool-step limit. Eight steps is an example; choose and test the limit against legitimate task length and cost budgets.

[P0 Blocking] Hard Dollar-Cost Circuit Breaker: Real-time cumulative token expenditure tracking with immediate session termination upon reaching dollar threshold.

[P0 Blocking] Infinite Loop Dampener: Detect repeated tool calls without state progression and stop unbounded loops. Two identical calls is an example trigger; choose and test a threshold that allows legitimate retries.

[P1 High] Downstream Tool Timeout Caps: Enforce per-tool deadlines so stalled calls cannot occupy workers indefinitely. Five seconds is an example; set and test each deadline against tool latency and service SLOs.

[P1 High] Bounded Exponential Backoff: Bound retries on transient errors with backoff and jitter. Three retries is an example, not a global limit; verify the budget against rate limits, idempotency and end-to-end deadlines.

[P1 High] Graceful Degradation Protocol: Failure of a mutative write tool cleanly degrades into a read-only notification without recursive re-planning loops.

[P1 High] Session Inactivity Expiry: Abandoned or frozen agent sessions automatically terminate and release resources after a defined idle timeout.

[P2 Medium] Watchdog Memory Monitor: External watchdog process monitors worker memory consumption and terminates runaway processes before host OOM.

[P2 Medium] Dead-Letter Queue (DLQ): Failed or aborted agent tasks are published to a dead-letter queue for offline engineering triage.

[P2 Medium] Token Consumption Velocity Alerts: Alert on abnormal token-consumption velocity against a measured baseline. A 3x rise within 60 seconds is an example; tune and test sensitivity for your traffic and response time.

Section 05

Human Oversight, Rollback & Observability

Ensuring state reversibility, cryptographic auditability, and executive accountability.

[P0 Blocking] Two-Phase Commit on Mutative Actions: High-consequence operations generate an uncommitted Action Proposal requiring explicit secondary confirmation.

[P0 Blocking] Cryptographic Human Sign-Off: Execution of mutative actions requires a time-bound, cryptographically signed approval token from an authorized operator.

[P0 Blocking] Comprehensive Diffs for Approvers: Human approval interfaces present human-readable before-and-after diffs, impacted records, and estimated blast radius.

[P0 Blocking] Automated State Rollback / Compensation: Every mutative tool call has an automated rollback mechanism (DB transaction rollback, git revert, or compensating API call).

[P0 Blocking] Immutable Structured Event Tracing: All input prompts, raw completions, tool invocations, and stdout/stderr blocks exported to an append-only OpenTelemetry stream.

[P1 High] Full Session Replay Capability: The system can re-run and inspect any historical execution trace step-by-step for post-incident security forensics.

[P1 High] Real-Time Anomaly Alerting: Operational alerts trigger on unexpected tool errors, authorization rejections, or high-privilege tool invocations.

[P1 High] Named Engineering Release Owner: A specific technical leader (VP Eng, CTO, or Principal Architect) is explicitly accountable for the production deployment.

[P2 Medium] Pre-Release Domain Edge-Case Suite: Test domain-specific failure and timeout scenarios in staging before release. Twenty cases is an example, not a coverage target; select cases from your actual failure modes and record results.

[P2 Medium] Formal Launch Decision Record: Documented launch readiness assessment, release dependencies, and designated sign-off archived in engineering records.

Need an Independent Pre-Release Launch Gate?

Don't deploy high-consequence agents on unverified prompt safety. BartsAI provides 1-week independent reviews that stress-test your tool boundaries, calculate your blast radius, and deliver an actionable architecture review identifying critical release dependencies and fallback controls.

Request a Launch Gate Review Try the Architecture Self-Assessment →

Progress is automatically saved in this browser's localStorage as checked item codes and an update timestamp; these selections are not uploaded. Progress is valid for 7 days, renewed when saved or restored. Expired records are removed the next time you open the tool, or you can clear them using "Clear local progress".