Preparing your control plane for stateless tool APIs and per-request identity

Stateless tool APIs remove convenient server-side memory, but they do not remove the need for control. Preparing your control plane for stateless tool APIs and per-request identity means making every request independently authorized, explicitly contextualized, observable, and safe to retry.

For platform teams running specialist agents and MCP-connected tools, this is a practical architecture change rather than a switch on one API flag. A request may arrive without durable provider-side continuation state, execute tools in an isolated runtime, and need a short-lived identity that is valid only for that workload. The control plane must therefore own the policies, context envelope, lifecycle records, and recovery behavior that stateful integrations previously obscured.

Direct answer: Build for stateless tool APIs by externalizing durable workflow state, attaching a request-scoped authorization context to every tool call, using short-lived workload credentials where available, and recording a trace that links the user, agent, tools, policy decision, and outputs. Treat store: false as an API state-setting, not as a data-retention guarantee.

Why stateless tool APIs change the control plane

A stateful agent integration can quietly depend on provider-managed conversation history, runtime variables, and remembered tool progress. With stateless tool APIs, those dependencies become visible design responsibilities. The system can still support multi-step work, but it must decide where continuation data lives, who can read it, how long it is retained, and how a failed request is resumed.

OpenAI’s Responses API explicitly supports stateless continuation through store: false. That is useful when a control plane wants to avoid relying on stored response state. It should not be interpreted as a complete privacy or compliance mode: OpenAI’s current guidance distinguishes store=false from Zero Data Retention (ZDR), and notes that some features may still store application state even when ZDR is enabled.

This distinction matters because state has several layers:

• Request state

  is the prompt, selected tools, policy context, and credentials needed to execute one call.

• Conversation and workflow state

  is the durable record required to continue work across requests, retries, approvers, or agents.

• Runtime state

  includes temporary variables and files available while code or tools execute.

• Provider retention state

  concerns how the API platform handles eligible request and response data.

• Control-plane audit state

  records what was requested, authorized, executed, and returned.

These layers should not be collapsed into a single “stateless” label. A workflow can use a stateless model API while the organization deliberately retains a minimal encrypted workflow ledger. Conversely, an API request can set store: false while the overall product still has state in other services. Clear boundaries make retention reviews and incident investigations more credible.

Isolated execution narrows the implicit trust boundary

OpenAI documents Programmatic Tool Calling in the Responses API as running in a fresh, isolated V8 runtime. There is no persistent JavaScript state between executions, no direct network access, and only the tools enabled in the request. This is an important constraint: code cannot safely assume that a prior invocation left behind a variable, a connection, or an ambient network path.

For a control plane, that is an advantage only if tool enablement is intentional. The request becomes the enforcement point for the available capability set. Rather than presenting every agent with a broad catalog and relying on instructions to limit behavior, resolve the smallest acceptable tool set before the call is made.

Design a per-request identity model before adding more tools

Per-request identity is the ability to answer, for each execution, not merely which API key called a service, but which user, workload, agent, tenant, policy version, and delegated authority caused a tool action. It provides a cleaner basis for authorization than sharing long-lived credentials across a fleet of agents.

Start by separating identities that are often conflated:

1. Human identity:

   the employee, customer, or service user initiating work.

2. Workload identity:

   the application, service, or deployment performing the request.

3. Agent identity:

   the configured specialist or agent role selected by the control plane.

4. Tool identity:

   the principal accepted by a downstream system, such as an MCP server or private API.

5. Delegation context:

   the bounded reason, scope, tenant, and expiry that connect those identities for one action.

The control plane should carry these as a signed or otherwise integrity-protected internal context, rather than asking the model to preserve them in prose. A model can receive a concise policy summary needed for reasoning, but authorization-relevant attributes should remain machine-readable and be evaluated by policy code.

Use mTLS as a verification layer, not as the whole identity system

OpenAI now documents mutual TLS (mTLS) for verifying certificate identity before authorizing an API request. The documentation also makes the boundary clear: mTLS does not replace API keys, service-account credentials, or workload-identity access tokens. Certificate verification proves an accepted client certificate is present; it does not automatically supply every authorization decision your application needs.

In OpenAI’s X.509 workload identity federation flow, a certificate is used to obtain a short-lived bearer token. The request then sends that token together with an accepted client certificate to the API mTLS endpoint. This pattern is valuable for control-plane design because it combines transport-level client verification with a time-bounded authorization artifact.

A practical request identity envelope can include the following fields:

• A generated execution ID and idempotency key.

• The authenticated human or upstream service subject.

• Tenant, project, and environment identifiers.

• The selected agent and approved tool capability set.

• A policy decision ID and policy version.

• A short expiry time, audience, and permitted operation scope.

• A trace or correlation ID shared across the agent and downstream tools.

Do not make every downstream service parse a large end-user token if a narrower delegated token is sufficient. Mint or exchange for a credential that has the smallest useful audience and scope. This reduces the blast radius of a misrouted request and makes logs easier to interpret.

Externalize workflow state without recreating uncontrolled memory

The goal is not to eliminate all state. The goal is to retain only the state the workflow genuinely needs, in a system your organization controls. A stateless API request should be reconstructible from an execution record and a deliberately selected context package, not from an undocumented mix of old chats, runtime leftovers, and shared credentials.

OpenAI’s tracing documentation defines a session as an agent’s conversation and work together, with each turn containing model responses, tool calls, and delegated subagent work. That model is useful even when the underlying request path is stateless: a session can be your business-level organizing concept, while each turn is independently assembled, authorized, and executed.

Keep a workflow ledger and a context assembler

A durable workflow ledger should record facts, not become an unbounded transcript dump. At minimum, keep the execution state needed to determine whether work is pending, completed, failed, awaiting approval, or safe to retry. Store references to protected source records where possible, instead of duplicating full sensitive payloads into every agent trace.

The context assembler reads the ledger and constructs a request-specific package. It chooses the relevant prior outputs, approved artifacts, task status, user preferences, and tool results. It should also apply tenant boundaries and redaction rules before the model receives context.

This separation produces useful operational behavior:

• A retry can reuse the same execution ID and policy decision while receiving a new short-lived credential.

• A new agent can take over a paused workflow without inheriting an old runtime’s hidden variables.

• An approver can inspect the proposed action and authorize the next turn without access to unrelated conversation data.

• A retention policy can delete or minimize workflow artifacts without relying on an API state setting to do that work.

Be careful not to overcorrect by serializing every token, tool payload, and intermediate thought into your own database. That creates cost, access-control, and retention burdens. Preserve the minimum evidence needed for reconstruction, audit, debugging, and user-visible continuity; make deeper diagnostic capture an explicit, access-controlled mode.

Know when managed state is the better fit

Stateless requests are not automatically preferable. OpenAI’s 2026 engineering guidance describes a good agent workflow as a tight loop: the model proposes actions, the platform runs them, and results feed the next step. The same guidance describes hosted containers with persistent runtime context and compaction for long-running agent work. That can be a better operational fit when a task genuinely needs sustained execution context.

The decision is therefore workload-specific. Use stateless request assembly when isolation, explicit handoffs, and control-plane portability are primary. Consider managed agent state or persistent runtime context when the platform’s managed loop and long-lived execution context solve a real workflow problem more safely than rebuilding the same behavior externally. In either case, preserve clear authorization, observability, and retention boundaries.

Prepare continuation paths for HTTP and WebSocket mode

Continuation is where stateless designs most often fail in production. A successful first request does not guarantee a later request can continue the same task, especially after a worker restart, socket loss, deployment, or regional routing change.

OpenAI documents that WebSocket mode preserves the same previous_response_id chaining semantics as HTTP mode. It keeps recent continuation state only in memory on the active socket, which makes it compatible with store=false and ZDR. But the trade-off is explicit: if a previous_response_id is not in the in-memory cache while store=false is used, there is no persisted fallback for continuation.

That behavior leads to a straightforward design rule: treat a response ID as a continuation optimization, not as the sole durable source of workflow truth. Your control plane should be able to assemble a new request from its own workflow ledger when the ephemeral path is unavailable.

1. Persist the business task ID, execution phase, selected agent, tool receipts, and approved outputs after meaningful transitions.

2. Keep the active socket and response-chain metadata as ephemeral transport state with a clear owner and timeout.

3. On reconnect failure or cache miss, transition the task into a reconstruction path rather than repeatedly attempting an unavailable chain.

4. Rebuild only the minimum context needed for the next action, then create a newly authorized request.

5. Record that reconstruction occurred so reviewers can distinguish a normal continuation from a recovery path.

For short interactive work, a WebSocket path can reduce the need to stitch recent continuation state externally. For durable operations, do not confuse in-memory socket continuity with a recovery strategy. A control plane serving enterprise users should make the failover behavior visible in its workflow status model.

Authorize tools as capabilities, including MCP and private APIs

Tool selection is an authorization decision. The control plane should decide which tools are relevant to the task, whether the requesting subject is allowed to use them, what data classes may be passed, and whether a proposed action needs approval. Passing a tool definition to a model should grant only the capability necessary for that request.

This is particularly important for MCP-connected agents. A specialist agent may be appropriate for a particular domain, but that does not mean it should receive every connector available to the organization. Resolve access from the active project, group, tenant, environment, and operation, then apply a deny-by-default tool policy.

OpenAI’s RBAC documentation says groups can be synchronized from an identity provider through SCIM, supporting centralized identity and permission management for projects and organizations. Use centrally managed groups as a source for control-plane policy, while still evaluating the immediate operation. Membership in a group may permit an employee to request an action; it does not automatically mean every action should execute without scoped delegation or review.

Make private connectivity part of the policy story

OpenAI’s private-MCP tunnel guidance describes a tunnel client that authenticates to the tunnel control plane, while the product side uses an OpenAI-hosted tunnel endpoint. This allows private APIs to be reached without opening inbound public traffic. That improves the network exposure model, but it does not remove the need to authorize the tool operation itself.

Similarly, OpenAI’s MCP and plugin authentication guidance says ChatGPT presents an OpenAI-managed client certificate when connecting to MCP servers, enabling transport-layer verification with mTLS. MCP servers should verify that transport identity where applicable, then apply their own operation-level authorization rules. A valid client certificate should not turn every server method into an unrestricted action.

Design principle: the model may propose an action, but the control plane grants the capability, and the tool service enforces its own final boundary.

For side-effecting tools, add explicit controls that prompt wording cannot bypass:

• Operation allowlists by agent role and environment.

• Schema validation and server-side input constraints.

• Idempotency keys for create, update, send, or payment-like actions.

• Approval gates for defined risk classes.

• Rate limits and spend or volume limits tied to tenant and workload identity.

• Tool receipts that include the request ID, outcome, and resulting resource reference.

Make traces the reconstruction record for per-request work

Once requests become independently assembled and authorized, observability is not an optional operations feature. It is how the control plane explains what happened after handoffs, retries, tool failures, and delegated agent work.

The Agents API launch material says the platform surfaces traceable execution details including turns, model responses, tool calls, outputs, duration, and status. Those details help a control plane reconstruct per-request work. Use them alongside, rather than instead of, your internal execution ledger, because your control plane is the system that knows the initiating identity, policy decision, routing choice, and business outcome.

Use a shared correlation model across the path:

• Session ID:

  the business conversation or unit of work.

• Turn ID:

  one user- or system-driven progression within that session.

• Execution ID:

  one control-plane attempt, including retries and recovery branches.

• Trace ID:

  the distributed observability context propagated to agents and tools.

• Tool receipt ID:

  the downstream proof associated with a side effect or result.

Do not put raw secrets, full access tokens, or unrestricted customer payloads into trace attributes. Log stable references, selected policy outcomes, tool names, result classifications, durations, and error categories. Give sensitive diagnostic data separate access controls and retention rules.

Account for tool results and retries

OpenAI’s observability documentation notes that each agent call follows the model’s token-pricing and prompt-caching rules, and that tool results are included in input tokens. This affects per-request accounting. A control plane that charges back or budgets only initial prompts will miss a material part of the work represented by iterative tool use.

Attribute usage to the execution ID, tenant, agent, model, and tool path. Distinguish a completed business operation from a completed model call: a model response can be successful while a downstream tool is denied, times out, or produces an invalid result. This distinction prevents misleading reliability dashboards and gives operations teams a usable basis for triage.

Separate retention controls from request-state controls

Retention is a platform property and a governance decision, not a boolean your client can infer from a single request. OpenAI’s data-controls documentation states that when ZDR is enabled, store is always treated as false for eligible endpoints, even if a request attempts to set it to true. The developer guidance also emphasizes that store=false is not the same as ZDR.

The same warning appears in OpenAI’s Bedrock integration documentation: store: false does not guarantee ZDR. The correct engineering response is to model retention at several layers and confirm the applicable service configuration, endpoint eligibility, and feature behavior with the relevant platform documentation and organizational agreements.

A useful control-plane review asks four different questions:

1. Does this request require provider-side response state after completion?

2. What data does our application retain for workflow continuity, operations, and audit?

3. What telemetry or tool-server records are created outside the model API?

4. Which retention promises apply to the endpoints and optional features actually used?

Answering these independently avoids false assurances. It also helps product and operations teams decide when to redact inputs, replace direct values with references, or route a task to a workflow that uses a different data-handling path.

Roll out stateless agent workflows with bounded failure modes

A migration should start with a narrow workflow whose inputs, outputs, and tool side effects are understood. Avoid beginning with a broad autonomous agent that can invoke many connectors; it becomes difficult to distinguish state bugs from policy bugs, tool failures, and routing mistakes.

OpenAI’s Agents API launch post describes creating a production-ready agent in a single API call by specifying the task, model, tools, and environment, with the platform managing the agent loop. It also says the platform can run tools in parallel and chain related operations in code. These capabilities can reduce external state stitching for some workflows, but the control plane still needs to decide which request receives which tools, identities, and limits.

A practical rollout sequence

1. Inventory hidden state.

   Identify where existing agents rely on stored response history, browser-like runtime state, shared API keys, implicit network access, or operator knowledge.

2. Define the execution contract.

   Specify the context envelope, identity assertions, allowed tool set, required receipts, retention class, and terminal states.

3. Build the policy decision point.

   Evaluate user, group, tenant, agent, environment, risk, and requested operation before any tool is exposed.

4. Introduce short-lived credentials.

   Where supported, move from broadly shared secrets toward federated, scoped tokens and certificate-based verification.

5. Implement reconstruction.

   Test a lost WebSocket, expired token, worker restart, duplicate delivery, and partial tool completion.

6. Instrument before scaling.

   Ensure a single execution can be followed from intake through agent turns, tool calls, approvals, and final status.

7. Expand by risk tier.

   Start with read-only or reversible operations, then add controlled side effects with idempotency and approval mechanisms.

Test the negative path as seriously as the happy path. Verify that an unauthorized agent cannot obtain a tool definition, a tool cannot act outside the delegated scope, a retry does not duplicate a side effect, and a missing continuation ID produces a clear reconstruction state rather than silent data loss.

Key design decisions for a durable control plane

Stateless tool APIs work best when the control plane becomes explicit about responsibilities that were previously implicit. The API request should contain only the context and tools required now; durable workflow knowledge should live in a governed ledger; and each action should carry an identity and policy decision that can be evaluated independently.

Keep the distinctions clear: store: false controls a form of API response state but is not a ZDR guarantee; mTLS verifies certificate identity but does not replace tokens or service credentials; WebSocket continuation can be efficient but is not a durable fallback; and managed agent state can be useful when long-running execution genuinely needs it. With these boundaries in place, teams can route work among specialist agents and MCP tools while preserving least privilege, recoverability, and a defensible record of every request.