Anatomy of an AI agent tool-calling loop

Overview

A tool-calling loop solves a precise systems problem: a language model can generate text but cannot, by itself, read a database, send an HTTP request, inspect a file, run a calculation, or commit a state change. The loop turns model output into a constrained request for an external capability, executes that request in ordinary software, then returns the result to the model as another input. The agent is therefore not the model alone; it is the model, the conversation state, the tool registry, the validator, the executor, and the policy layer that decides which calls are allowed.

The boundary matters because natural-language generation is probabilistic while tool execution is deterministic or at least externally observable. A correct design separates intent selection from authority. The model may propose a tool name and arguments, but the host process validates the arguments against a schema, checks authorization, applies timeouts and idempotency rules, and records what happened. The loop is useful when the task requires fresh state, precise computation, controlled side effects, or decomposition into steps that can be observed and corrected.

How it works

At the systems level, a typical loop has six stages. First, the host sends the model a task, prior messages, and a machine-readable catalog of tools. Second, the model returns either final text or a structured tool-call proposal containing a tool identifier, a call identifier, and arguments. Third, the host parses and validates the arguments. Fourth, the executor invokes the local function, command, database query, or HTTP endpoint. Fifth, the host appends a tool-result message, usually tied to the original call identifier. Sixth, the model consumes the observation and either stops or proposes another call.

The control plane and data plane should be treated separately. The control plane contains schemas, tool names, policies, iteration limits, cancellation, tracing, and authorization. The data plane contains tool inputs and outputs, which may be large, untrusted, or confidential. Streaming adds another path: model deltas may arrive over Server-Sent Events as defined by the WHATWG HTML Living Standard’s EventSource processing model, but partial deltas should not be executed until a complete, validated call is assembled. Parallel tool calls are possible only when the host can prove that their side effects do not conflict or has compensating controls.

What the standard says

There is no single W3C, WHATWG, or IETF standard that defines an “AI agent” or mandates a universal tool-call message shape. Current implementations compose existing public specifications. JSON is defined by IETF RFC 8259; it requires object member names to be strings and states that member names within an object should be unique, because duplicate-name behavior differs across parsers. JSON Schema Draft 2020-12 defines how an instance is validated against a schema, including object properties, required fields, enumerations, numeric bounds, and vocabularies. The schema validates data; it does not prove that a proposed action is safe.

When tools are HTTP operations, OpenAPI Specification 3.1 is the current OpenAPI version aligned with modern JSON Schema use; OpenAPI 3.0 used a more limited schema dialect and should not be treated as equivalent. An OpenAPI Operation Object can describe parameters, request bodies, responses, and an operationId, which many runtimes map to a tool name. HTTP method semantics come from RFC 9110: GET, HEAD, OPTIONS, and TRACE are defined as safe; PUT and DELETE are idempotent; POST is not guaranteed to be either. Error reporting can use RFC 9457 Problem Details for HTTP APIs, which obsoletes RFC 7807. Rate limiting can be exposed with the RateLimit fields in RFC 9564 and retry timing with Retry-After from RFC 9110.

Trade-offs

The main measurable trade-off is extra work per step. Each loop iteration adds at least one model inference plus any external tool latency, and the tool catalog consumes context tokens before the task itself is considered. Tool outputs also consume context unless summarized, filtered, or stored outside the prompt. These quantities are measurable without inventing new thresholds: number of iterations, prompt tokens, completion tokens, tool-call count, tool latency, end-to-end latency, validation-failure count, HTTP status distribution, and bytes returned by tools.

Stronger structure reduces invalid calls but can increase repair turns. A strict JSON Schema with required fields and additional or unevaluated property controls catches malformed arguments before execution, but the model may need another turn to correct them. Permissive schemas allow progress but move ambiguity into the executor. Parallel execution can reduce wall-clock time for independent reads, while sequential execution is safer for dependent or state-changing actions. The HTTP distinctions in RFC 9110 are practical here: retrying an idempotent PUT after a transport failure has different risk than retrying a POST that may already have caused a side effect.

Failure modes

Common failures fall into a few diagnosable classes. Schema failures include invalid JSON, missing required fields, wrong enum values, and stringified numbers where numbers are required. Tool-selection failures include choosing a tool that cannot answer the question or calling tools in the wrong order. Execution failures include timeouts, authorization errors, HTTP 4xx and 5xx responses, and rate-limit responses such as 429 with Retry-After or RateLimit fields. Reasoning failures include infinite loops, ignoring an observation, or treating untrusted tool output as an instruction.

Diagnosis depends on preserving the boundary between proposal, validation, execution, and observation. Logs should record the model response, parsed call, schema validation result, normalized tool arguments, tool status, elapsed time, and the call identifier that links the result to the request. Distributed traces can propagate W3C Trace Context fields, traceparent and tracestate, through model gateway, orchestrator, and tool executor; OpenTelemetry’s public specification defines spans, attributes, events, and status for recording this path. For HTTP tools, RFC 9457 problem details give machine-readable type, title, status, detail, and instance fields, which are more useful than opaque error strings.

A minimal implementation

A correct minimal implementation is a bounded state machine, not an unconstrained recursive chat. It defines a small tool registry, gives each tool a stable name and JSON Schema Draft 2020-12 input schema, and stores an executor function outside the model. For every model turn, the host asks for either a final answer or a tool call. If a call appears, the host parses JSON with a standards-compliant parser, rejects duplicate or ambiguous shapes according to local policy, validates the arguments, checks that the tool is allowed in the current context, executes with a timeout, and appends a structured result. The loop stops on final output, validation failure that cannot be repaired, cancellation, or a configured iteration cap.

In pseudocode, the shape is: initialize messages and tools; while steps remain, call the model with messages plus tool schemas; if the response is final, return it; otherwise locate the named tool; validate arguments against its schema; if validation fails, append a tool-error observation and continue or stop; execute the tool with least authority; append {tool_call_id, status, result_or_problem}; repeat. The minimal design deliberately omits autonomous privilege escalation, hidden state changes, and free-form shell access. Those additions are not properties of the tool-calling loop itself; they are separate authority decisions that require their own specifications, audit records, and failure handling.