From Model Call to Reliable AI Workflow

The first version of an AI feature is often a prompt, a model call, and a string returned to the user. That is an excellent prototype. It is not yet a production system.

Production changes the question from “Can the model produce a good answer?” to “Can the complete system produce an acceptable outcome, under known constraints, repeatedly enough to operate?” The complete system includes input policy, context, tools, model choice, output validation, evaluation, latency, cost, observability, and recovery.

The right architecture is usually the least autonomous design that satisfies the task.

Begin with the outcome contract#

Define the job without mentioning a model. For a hypothetical support-triage feature:

Given a new ticket and approved account context, classify the ticket, propose a routing destination, and draft a response. Never change account state. Escalate when evidence is insufficient or the request concerns billing disputes.

This contract identifies:

• Inputs the system may use.
• Outputs it must produce.
• Actions it must not take.
• Conditions requiring human judgment.
• A result that can be evaluated.

“Build an agent that handles support” identifies an implementation before it identifies safe behavior.

Level zero: one constrained model call#

Start with a single request when the task is self-contained and mistakes are reversible.

const response = await model.generate({
    system: TRIAGE_POLICY,
    input: ticket,
    responseSchema: TriageResultSchema,
});

const result = TriageResultSchema.parse(response.output);

Structured output turns malformed responses into explicit failures. It does not prove semantic correctness. The schema can ensure that route is one of support, billing, or security; it cannot ensure the chosen route is justified.

Before adding components, measure this baseline against representative examples. A simple call may already be the best architecture.

Add context only when the task needs it#

There are three common sources of context:

1. Static instructions and examples included with every request.
2. Retrieved documents selected for the current input.
3. Live state obtained through an authorized tool.

Retrieval is appropriate when answers depend on a corpus too large, volatile, or private to place in every prompt. A retrieval pipeline introduces its own failure modes:

• The relevant document is absent from the index.
• Chunking separates a statement from its qualifier.
• Ranking selects plausible but irrelevant text.
• Permissions are lost during indexing or filtering.
• Stale documents outrank current policy.

Evaluate retrieval separately from generation. If the required evidence never reaches the model, prompt changes cannot fix the system.

Prefer deterministic workflows for known paths#

Many “agent” use cases are actually workflows. The possible steps are known; a model contributes classification or generation inside controlled code paths.

flowchart LR
    I[Ticket] --> V[Validate and redact]
    V --> C[Classify]
    C -->|billing dispute| H[Human queue]
    C -->|supported category| R[Retrieve policy]
    R --> D[Draft response]
    D --> G[Policy and schema gates]
    G -->|pass| P[Present for approval]
    G -->|fail| H

The application decides which steps are possible and which transitions are allowed. The model handles bounded reasoning within a step. This is easier to test, observe, and budget than an open-ended loop.

Useful workflow patterns include:

• Prompt chaining: one step produces structured input for the next.
• Routing: classify input and select a specialized path.
• Parallelization: run independent analysis concurrently.
• Evaluator-optimizer: generate, evaluate against explicit criteria, and revise within a bounded loop.
• Orchestrator-workers: dynamically divide a complex task while constraining worker capabilities.

Every extra call adds latency, cost, and another opportunity for inconsistent state. Add a step because evaluation shows it improves an outcome, not because the diagram looks more sophisticated.

Use an agent only when the path is genuinely unknown#

An agent chooses its next action based on intermediate state. That flexibility is valuable when a fixed workflow cannot enumerate the necessary steps—for example, investigating an unfamiliar codebase or gathering evidence across heterogeneous systems.

It also introduces compounding risk. A small probability of a poor decision at each step becomes more significant over a long trajectory.

Use this decision sequence:

flowchart TD
    A[Is one model call sufficient?] -->|yes| B[Use one call]
    A -->|no| C[Are the steps known in advance?]
    C -->|yes| D[Use a deterministic workflow]
    C -->|no| E[Must the system choose tools or steps dynamically?]
    E -->|no| D
    E -->|yes| F[Are actions reversible and bounded?]
    F -->|no| G[Add human approval or redesign]
    F -->|yes| H[Use a bounded agent loop]

A bounded loop needs:

• Maximum steps and wall-clock duration.
• Tool allowlists and argument validation.
• Per-run cost and token budgets.
• Cancellation and idempotent resumption.
• A clear terminal condition.
• Human approval for consequential actions.

Tools are authority, not merely functions#

A tool description tells the model what can be attempted. The server-side implementation determines what is actually authorized.

Design each tool with narrow semantics:

type RefundPreviewInput = {
    orderId: string;
};

type RefundPreviewResult = {
    eligible: boolean;
    maximumAmount: number;
    reasonCodes: string[];
};

refund_preview is safer and easier to evaluate than a generic execute_sql or call_internal_api. Separate read and write tools. Validate arguments. Derive user identity from trusted authorization context, never from a model-provided field. Return concise, structured results.

Do not rely on the prompt to enforce permissions. Prompts guide behavior; application code enforces authority.

Treat model output as untrusted data#

The system should validate every boundary:

• Schema validation for structured output.
• URL and identifier allowlists.
• Output encoding before rendering.
• Citation checks when claims must be grounded.
• Policy checks before side effects.
• Human approval for irreversible or high-impact operations.

Retrieval content and tool results can contain instructions. Keep trusted system policy separate from untrusted data, label boundaries, and avoid granting an instruction found in a document the authority of application code.

Build evaluation before optimization#

An evaluation suite is a versioned set of tasks, expected properties, graders, and results. Begin with failures the product actually cares about.

For support triage, include:

• Straightforward routing examples.
• Ambiguous tickets that should escalate.
• Conflicting or outdated retrieved policies.
• Attempts to override system instructions.
• Missing account context.
• Tool timeouts and malformed tool results.
• Long inputs and multilingual inputs within product scope.

Grade dimensions separately:

Use deterministic graders when possible: schemas, exact labels, unit tests, database state, and policy rules. Model-based graders can help judge nuance but should be calibrated against human review and should not be the only source of truth.

For agents, evaluate the outcome and the trajectory. A booking agent saying “done” is not success if no reservation exists. A correct outcome reached through repeated unauthorized attempts is also not acceptable.

Observe a trace, not a blob of text#

Log a structured trace for every generation:

• Trace and request identifiers.
• Application, prompt, model, and tool versions.
• Redacted input classification—not raw secrets by default.
• Retrieval queries and selected document IDs.
• Tool names, validated arguments, durations, and outcomes.
• Token usage, latency, and estimated cost.
• Validation and policy-gate results.
• Final disposition: success, fallback, escalation, cancellation, or failure.

Avoid placing private prompts, access tokens, or full customer data in general-purpose logs. Observability must follow the same data-minimization rules as the feature.

Control latency and cost as architecture constraints#

A workflow with six sequential model calls cannot meet a two-second objective if each call takes one second. Set budgets before adding steps:

end-to-end p95 latency: 4 seconds
maximum model calls:    3
maximum tool calls:     4
maximum run cost:       $0.03
maximum agent steps:    8

Then decide where to spend them:

• Parallelize independent retrieval or checks.
• Route simple cases to smaller models.
• Cache stable deterministic transformations.
• Summarize tool output before it enters context.
• Stop when sufficient evidence exists.
• Fall back or escalate before exhausting the entire budget.

Cost per request is less useful than cost per successful, policy-compliant outcome. A cheaper model that causes more retries or escalations may be more expensive operationally.

Design escalation as a first-class result#

Human escalation is not an exception handler. It is a valid terminal state.

Provide the reviewer with:

• The original task.
• Relevant evidence and citations.
• Proposed output or action.
• Reasons for uncertainty or policy conflict.
• A compact trace of tools already used.

Do not force a model to guess merely because the interface expects a string. A typed result can represent uncertainty:

type WorkflowOutcome =
    | { status: "completed"; draft: string; evidenceIds: string[] }
    | { status: "needs_review"; reason: string; evidenceIds: string[] }
    | { status: "failed"; code: string; retryable: boolean };

A production progression#

Build in this order and advance only when evaluation identifies a limitation:

1. One constrained call with structured output.
2. Retrieval or examples for missing context.
3. A deterministic workflow for known multi-step behavior.
4. Narrow, authorized tools for live state or actions.
5. Evaluation, tracing, budgets, fallbacks, and escalation.
6. A bounded agent only for tasks with genuinely dynamic paths.

The hard part of AI engineering is not making a model sound capable. It is constructing a system that knows what it may do, can demonstrate what it did, and has a safe answer when it should not continue.

Operate changes as versioned experiments#

An AI workflow changes when its model, prompt, retrieval corpus, tool contract, policy, or post-processing code changes. Treat those inputs as a versioned release unit. A trace should identify the complete unit so a regression can be attributed and an earlier behavior reproduced. Recording only the model name is insufficient when a prompt template or retrieval filter changed simultaneously.

Before promotion, run the candidate against a fixed evaluation set and a recent sample representing current traffic. Compare task-specific quality, refusal and escalation behavior, tool errors, latency, and cost. Aggregate scores are useful gates, but inspect individual regressions: a small average improvement can hide a severe failure on a critical slice. Evaluation sets also need maintenance because a static set gradually stops representing changing inputs.

Release to a stable cohort and keep the previous workflow available for rollback. Shadow evaluation can compare outputs without exposing the candidate response, although tools with side effects must be stubbed or disabled. Define the decision rule and maximum exposure before a live experiment starts; otherwise normal variance invites post-hoc conclusions.

Provider failure requires an explicit response. A fallback model may preserve availability while violating quality, latency, context-window, or data-residency assumptions. Validate it as a separate workflow rather than treating providers as interchangeable endpoints. For some tasks, a clear temporary failure or human route is safer than silent degradation.

Finally, define deletion criteria. Once a candidate meets its observation window, rollback risk is acceptable, and active traces no longer depend on the old path, remove obsolete prompts, adapters, and evaluation branches. Reliability includes controlling the complexity surrounding the model.

Sources and further reading#

• Anthropic: Building effective agents
• Anthropic: Demystifying evals for AI agents
• NIST AI Risk Management Framework
• OWASP Top 10 for LLM Applications
• OpenAI: Evaluation best practices