What Actually Breaks When AI Agents Reach Production

A demo proves the happy path. Production exposes permissions, retries, cost, bad inputs, and unclear ownership.

An agent combines model output with tools that read data or create side effects. That can be useful, but it also joins nondeterministic decisions to APIs, queues, databases, credentials, and people. Buyers should review the entire workflow, not only the quality of one response.

A demo proves the happy path

A prepared demo usually has a clean input, available dependencies, valid credentials, and one expected route. Production adds missing fields, conflicting instructions, expired tokens, slow services, duplicate events, changed schemas, and users with different access.

Ask the agency to demonstrate the failure paths: a tool times out after another tool already wrote data; an approval is denied; a record belongs to another tenant; a dependency returns malformed content; or the model cannot choose a valid next step. A production design needs a safe state for each case.

Permissions spread

Each new tool expands what the workflow can read or change. A prompt is not a permission boundary. Access must be enforced by the application and target system for the specific user, tenant, record, and operation.

Separate read access from write access. Avoid a shared credential with blanket authority. Require a person to approve consequential actions with the final arguments visible, and make revocation possible without redeploying the whole agent.

Retries multiply cost

One business task may trigger several model calls and tool calls. A timeout can make the orchestrator try again even when the first call completed. The result can be repeated charges, duplicate messages, conflicting updates, or a loop that continues without useful progress.

Every retry needs a limit, and every side effect needs protection against duplicate execution. Ask for idempotency keys, deadlines, retry budgets, backoff, loop limits, and a manual queue for work that cannot finish safely. Cost controls should apply to the whole task, not only one model request.

The log should connect the original request to every model call, tool call, approval, retry, and final state. Without that trace, the team cannot explain a bill or repair a partial failure.

Bad inputs become actions

Customer messages, documents, web pages, database fields, and tool results can all contain mistakes or hostile instructions. Untrusted text is data, not an instruction to follow. The workflow should separate system rules from retrieved content and keep the model from granting itself new capabilities.

Validate tool arguments against allowed schemas, values, destinations, and business rules. Validate results before another tool uses them. A structured output can be well formed and still be wrong, unauthorized, or unsafe for the next action.

• Use allowlists for tools, destinations, and consequential operations.
• Reject or quarantine inputs that cannot be validated.
• Show reviewers the source, proposed action, and final arguments.
• Test indirect instructions hidden in documents and tool responses.
• Keep a non-agent fallback for work that reaches a boundary.

Someone still owns the failure

A production workflow needs a named business owner and a named technical owner. They decide what counts as success, which failures require an incident, when to disable the system, and who resolves records left in an uncertain state.

The operating plan should define task-level evaluations, escalation thresholds, cost and latency limits, retained evidence, dependency changes, and a shutdown path. Monitoring is useful only when a person is responsible for acting on it.

Ask the agency to answer these questions before launch:

• Which production inputs and failure cases are in the evaluation set?
• What is the maximum authority, call count, duration, and cost for one task?
• How are partial writes, duplicate actions, and unavailable dependencies repaired?
• Which events page a person, and who owns the response?
• Can your team inspect the trace and disable each tool without the agency?
• What changes when a model, prompt, tool schema, or policy is updated?

The buying decision is not whether the demo looks autonomous. It is whether the workflow has limited authority, bounded retries, validated inputs, recoverable failure states, and an owner who can stop it.