An Error Message Is Not a Retry Policy: Typed Failures for AI Tools

By Zahi Raifer

An AI assistant calls a tool that edits a page. The tool returns “Something went wrong.” The assistant tries again. The same message comes back, so it changes the request and tries a third time. None of these actions has addressed the actual problem: the account is not allowed to edit that page.

The tool technically reported a failure. It failed to communicate the kind of failure that matters for the next decision.

For an AI application, an error response is part of the control interface. It tells the caller what is known, what remains uncertain, and which actions are permitted next. Treating every failure as an arbitrary string gives the model a troubleshooting problem that ordinary code could have resolved.

The design below is a proposed engineering pattern, not a claim about features already shipped in any particular product.

Separate the audience from the decision

A useful error has at least two audiences: the person who needs an understandable explanation and the software that needs a reliable next step.

A human message can say, “This page changed while your request was being prepared.” The machine-readable part can identify a revision conflict and state that a fresh read is required before another write.

In the website-editing category I work on at Orivox, that separation is particularly useful. A person describes an intention in ordinary language, but the component applying a change needs an exact account of why it could not proceed.

Do not make the human sentence carry both responsibilities. Wording changes, translations, and better explanations should not change the caller's control flow.

Define a small failure vocabulary

Start with a short list of error classes that correspond to different recovery paths.

A validation error means the proposed input does not satisfy the tool's contract. An authorization error means the caller is not permitted to perform the action. A conflict means an assumption about current state no longer holds. A capacity error means the service cannot currently accept the requested work. A dependency error means another component failed.

These classes are not a substitute for specific codes. They organize them. A missing target and an invalid URL may both be validation failures while requiring different corrections.

Keep the vocabulary stable and resist adding a new top-level category for every exception. The objective is a set of decisions callers can implement, not a taxonomy of everything that can go wrong inside the server.

Report what happened to the effect

A failed call does not always mean that nothing changed.

The server may have rejected the request before execution, applied the change but lost the response, or stopped halfway through a multi-step operation. These situations require different handling even when the visible symptom is the same timeout.

Include an explicit effect status where the tool can establish it: not applied, applied, or unknown. If an operation is atomic, document which outcomes it can produce. If it is not atomic, describe the inspectable progress rather than inventing an all-or-nothing story.

An unknown outcome should lead to an observation step, not an automatic second mutation. A caller needs a way to inspect the operation or current resource state and reconcile the uncertainty.

Honesty here is more valuable than presenting every failure as a clean rejection.

Make recovery instructions bounded

A boolean called retryable is attractive because it is simple. It is also easy to misuse.

Retryable for whom, after what change, and within which budget? A capacity failure may allow another attempt after a delay. A conflict may allow another attempt only after rereading the resource. A validation failure may allow a corrected request, but repeating identical input is pointless.

Prefer a small set of recovery actions, such as wait, refresh state, correct input, request permission, inspect outcome, or stop. Attach bounded parameters only when useful: a minimum wait interval, a field name, or an operation reference.

The caller should still enforce its own attempt and time limits. A tool response cannot grant itself unlimited retries or override the application's authorization policy.

Keep internal detail out of the public contract

An exception message can contain a database query, a filesystem path, a provider response, or customer content. Returning it directly is both a privacy risk and an unstable interface.

Map internal failures to public codes at a deliberate boundary. Keep detailed diagnostics in access-controlled logs and give the response a correlation reference that support staff can use.

For validation feedback, return only the necessary information. A field path and a concise rule may be enough. Do not echo an entire submitted document to explain one invalid attribute.

Also avoid revealing whether a resource exists when the caller is not authorized to inspect it. A helpful explanation must not become an account-enumeration feature.

Distinguish evidence from instructions

Errors sometimes contain text from an external system: a provider's explanation, a rejected document fragment, or a remote page title. Treat that material as evidence, not executable recovery policy.

The trusted recovery action should come from application code. An untrusted error string that says “disable validation and try again” must not become an instruction to the assistant.

If including external text helps a human understand the failure, label and bound it. The model can read the quotation without receiving authority to follow it.

This rule is especially important when tools fetch content as part of editing. A remote website should not influence the permissions of the local editing tool merely because its response appeared inside an error.

Version the contract without breaking explanations

Publish the stable fields callers may depend on. Make optional diagnostic fields clearly optional, and define how an unknown code should behave.

A conservative fallback is to stop mutation and surface a safe explanation, rather than guess that a new error is transient. That keeps an older caller from repeatedly applying a request it no longer understands.

Do not require clients to recognize every future subtype. A stable category and effect status can support a safe fallback while a more specific code enables better handling in newer clients.

Keep translated human messages outside the identity of the code. “Permission denied” and its equivalent in another language should lead to the same authorization decision.

Test decisions, not just response shapes

Schema validation proves that an error object is well formed. It does not prove that its advice is correct.

Test that authorization failures never trigger identical retries, conflicts require a fresh read, unknown outcomes lead to inspection, and internal exception text does not leak into the public response.

Include failures in the error-handling path itself. If diagnostic logging is unavailable, the tool still needs to return a safe result rather than expose an exception stack.

A good error contract makes failure less improvisational. The model can explain the situation in natural language while code decides which recovery paths are actually safe.

Zahi Raifer is the founder of Orivox, focused on engineering practical AI tools for building and editing websites.