Home/Blog/From session state to per-call connectors: how recent protocol changes improve security and scale

From session state to per-call connectors: how recent protocol changes improve security and scale

November 7, 2026

From Session State To Per Call Connectors How Recent Protocol Changes Improve Security And Scale

Long-lived session state can make AI tool integrations difficult to secure, audit, and scale: identity, authorization, routing, and execution context risk becoming tightly coupled to a connection that persists longer than a single action. Per-call connectors offer a more explicit operating model, where each tool invocation can be governed, executed, and returned as a discrete event rather than assumed to inherit broad, implicit session access.

That shift is visible in recent OpenAI protocol, product, and infrastructure changes. The practical implication for platform engineers is not that session state disappears; it is that teams should minimize the state that must remain trusted in the data path, centralize connector governance, and make authorization, scope, and execution boundaries clear at the point a tool is called.

Per-call connectors: the direct answer

Per-call connectors improve security and scale by making tool access explicit for each invocation instead of relying on broadly attached, long-lived session context. This design can reduce implicit trust, support narrower policy enforcement and auditing, and lessen the infrastructure burden created when a data-plane component must own and terminate every active session.

The model is straightforward in principle. A model determines that a function call is needed and emits the function-call arguments. Application code executes the requested action, then sends the result back to the model for that call. OpenAI’s Realtime with tools documentation describes this sequence directly.

This separation matters because an AI workflow has several different responsibilities that are often blurred together in early implementations:

  • Decision:

    determining whether a tool is needed and which action to request.

  • Authorization:

    deciding whether the requesting user, agent, or workflow may use that connector and operation.

  • Execution:

    performing the external action through an approved connector or service.

  • Observation:

    recording the request, result, policy decision, and any security-relevant outcome.

A session can still carry useful conversational or interaction context. It may identify the active user experience, preserve an ongoing voice interaction, or correlate related requests. The design question is whether that session is also the place where broad connector permissions, routing ownership, and execution authority live by default. Per-call orchestration gives engineering teams a way to separate these concerns.

For an agent orchestration workspace, this distinction is especially important. A central control plane may route work to specialist agents, transfer only the context needed for a handoff, and invoke MCP-connected tools. If every downstream agent inherits a session’s full connector surface automatically, the effective access boundary can become hard to reason about. If each tool action is instead evaluated through a connector-specific path, the control plane has a clearer place to apply policy and generate evidence.

Why protocol changes are moving away from older connector attachment patterns

OpenAI’s Realtime with tools guide states that the model emits function-call arguments, code executes the action, and the result is returned per call. The same guide says that connector_id is deprecated for models released after September 1, 2026. Taken together, these details are a practical signal that newer integration patterns are moving away from older connector attachment mechanisms and toward more explicit orchestration.

Deprecation alone does not prescribe a complete enterprise architecture. It does, however, force an important review: teams should locate where older connector identifiers are assumed to be durable session attributes, then decide how to represent connector selection, policy, and execution in the newer flow.

What changes in the request lifecycle

In a session-centric design, a connector may be attached early in an interaction and treated as available until the session ends. Later actions can appear simple because the session implicitly supplies the connector relationship. The operational cost of that simplicity is that access decisions may be separated from the precise action that uses the access.

In a per-call design, the tool request becomes a first-class boundary. The model provides arguments, the application performs the action, and the application returns the result. This creates a natural point to validate the tool name, inspect arguments, select the approved connector path, and attach a correlation record before any external work occurs.

  1. An agent or model proposes a specific function call and arguments.

  2. The

    orchestration layer

    identifies the intended tool and applicable connector policy.

  3. The execution service validates the request and performs the external action.

  4. The result is returned to the model as the output of that call.

  5. Operational and security records associate the outcome with the specific invocation.

Not every use case needs the same checks. A read-only search action, a retrieval action that exposes regulated material, and a write action that modifies an external system carry different risks. The advantage of a per-call lifecycle is not that every call must be treated identically; it is that the architecture gives teams a stable place to differentiate them.

What should not be inferred

Per-call execution is not a guarantee of safety. A harmful or manipulated request can still be proposed, an overly broad connector policy can still exist, and an external system can still enforce permissions incorrectly. The protocol boundary is valuable because it makes controls easier to place and inspect, not because it replaces authorization design, input validation, monitoring, or testing.

Reducing long-lived session coupling improves the scaling path

The infrastructure case for reducing session coupling is as important as the security case. In OpenAI’s low-latency voice architecture write-up, stateful sessions at scale are described as difficult because the process that creates a session must continue to receive its packets to validate connectivity, complete handshakes, decrypt traffic, and handle later session changes.

That arrangement ties session ownership to particular processes and network paths. It can create friction when systems must scale horizontally, rebalance workloads, recover from failures, or operate cleanly with Kubernetes and load balancers. A component cannot be treated as easily interchangeable if it must keep receiving traffic for the sessions it originated.

The redesign described in the write-up uses cached state and avoids protocol termination in the relay path. OpenAI reports that this improves scalability and operational isolation. The lesson for tool and connector systems is architectural rather than a claim that voice media and tool calls are identical: minimize the state that a data-plane component must hold and avoid giving the relay layer more session responsibility than it needs.

Apply the same reasoning to agent tool execution

An orchestration platform often needs to route requests across specialist agents, tool runners, approval services, and connectors. If a particular worker must own all future actions for a user’s session because it retains the connector relationship, scaling becomes more rigid. Retries, failover, deployment changes, and workload balancing must account for hidden ownership.

A more composable pattern retains durable policy and configuration in controlled systems of record or caches while allowing the execution path to resolve the required context for the individual call. The relay or routing layer can then focus on delivering the invocation to an appropriate executor instead of terminating and maintaining every protocol relationship itself.

  • Less sticky routing:

    requests need not remain bound to the initial session process solely to preserve connector availability.

  • Cleaner isolation:

    routing and relay components can have narrower responsibilities than components that authenticate and execute external actions.

  • More predictable recovery:

    the system can reason about an invocation as a bounded unit with a specific policy and result path.

  • Better control-plane alignment:

    workspace policy, connector configuration, and agent routing can be managed separately from transient execution workers.

There is a trade-off. Resolving context per call can add design and operational complexity. Teams need reliable policy lookup, careful cache behavior, clear correlation identifiers, and explicit handling for failures between a proposed tool call and its result. For high-throughput systems, those details must be engineered deliberately rather than assumed away.

Use workspace governance to control connector access at scale

Per-call protocol boundaries are strongest when connector governance is centralized. OpenAI Enterprise and Edu release notes say that administrators can review and publish connectors to the workspace and use role-based access control (RBAC) to control who can develop, test, and use each connector. This is a meaningful operational distinction from allowing connectors to spread as unmanaged, user-by-user configuration.

Workspace administration gives organizations a defined lifecycle for a connector: it can be built, tested, reviewed, published, used by approved roles, and changed under an administrative process. That does not remove the need for application-level policy, but it provides a common governance layer before a connector enters broad use.

Separate connector lifecycle from runtime invocation

A connector’s availability and a specific call’s authorization are related but different decisions. Publishing a connector answers whether it is an approved integration for the workspace. RBAC determines who may develop, test, or use it. A runtime call then needs to be evaluated in the context of the selected operation, request arguments, and the external system’s own permissions.

This separation helps avoid a common failure mode: treating approval of an integration as blanket approval for every possible action through that integration. A connector may be appropriate for a limited population, a particular environment, or a defined workflow, while individual operations still require constrained handling.

For a multi-agent environment, make the operational roles explicit. A builder who develops a connector, an evaluator who tests an agent workflow, and an end user who triggers a production action should not automatically have the same authority. OpenAI’s documented RBAC controls support this kind of differentiated rollout at the workspace level.

A practical governance baseline

  1. Keep an inventory of the connectors approved for a workspace and the teams responsible for them.

  2. Use role controls to distinguish connector development, testing, and production use.

  3. Define which specialist agents or workflows may request each connector.

  4. Associate runtime calls with the approved connector and the actor or workflow that initiated them.

  5. Review policy changes and connector publication as operational changes, not as incidental configuration.

This approach is more deliberate than attaching tools opportunistically to an active session. It may slow unstructured experimentation, but it creates a repeatable path to production for teams that need to operate many agents and connectors across a shared enterprise environment.

Centralized authentication does not require broad connector scope

OpenAI’s Enterprise and Edu release notes provide a concrete example of centralized administration with controlled scope. Administrators can authenticate once and deploy SharePoint connectors at workspace scale. They can choose to sync all sites or specific folders, while ChatGPT enforces existing SharePoint permissions.

This combination is useful because it separates several layers that are often mistakenly collapsed. Administrative authentication can establish the workspace deployment. Scope selection can limit what the connector syncs. Existing SharePoint permissions continue to determine what individual users can access through the connected content.

Design for layered authorization, not a single all-powerful grant

When planning connector integrations, identify the boundaries that should remain independent. A central administrator may be permitted to enable a service for a workspace without being the person who decides every user’s content access. A workspace policy may permit a connector without granting every agent every operation. The source system may remain the authority for resource-level permissions.

  • Deployment authority:

    who can authenticate and make a connector available to a workspace.

  • Connector scope:

    what sites, folders, resources, or operations are included.

  • Workspace authorization:

    which roles may develop, test, or use the connector.

  • Source-system authorization:

    what a user may access under the external system’s existing permission model.

  • Call-level controls:

    what a particular agent or workflow may request at execution time.

The SharePoint example should not be generalized into claims about every connector’s semantics. Its value is as a pattern: centralized setup can coexist with scoped synchronization and enforcement of source permissions. Platform teams can use that pattern when deciding where administrative consent ends and runtime access checks begin.

For MCP-connected agents, this also supports a cleaner handoff model. The control plane can pass the minimum relevant task context to a specialist rather than handing over a vague assertion that the entire session is trusted. The specialist can request an allowed tool action, and the execution layer can apply the connector’s configured scope and applicable policies.

Make connector enablement and session visibility deliberate

Security controls are more effective when users and administrators can see what is active. ChatGPT Business release notes added an Active sessions view with device, app, approximate location, sign-in time, trusted-device status, and an indication of whether the session is the current session. This improves visibility into account and session risk by making session metadata reviewable rather than invisible.

At the connector level, OpenAI’s ChatGPT release notes say that users must go to Settings → Connectors → Connect to enable connectors. For deep research, each connector must still be enabled separately and selected whenever a new request starts. These are product UX choices, but they express an important systems principle: access should require deliberate action at a boundary where users can understand what they are enabling.

Visibility changes operational behavior

A long-lived session often feels harmless until it is difficult to answer basic questions: Which device is still signed in? Which application is using the account? Which connectors are active for the task at hand? Explicit views and enablement steps do not solve every risk, but they make it easier to detect unexpected state and to establish accountability.

For enterprise orchestration systems, carry that principle into the control plane. Operators should be able to distinguish a user’s login session from a connector’s workspace publication state, a connector’s enabled status for a workflow, and a single tool invocation. These are different objects with different lifetimes, owners, and remediation actions.

Do not use “the user has an active session” as a substitute for “this agent is authorized to make this connector call with these arguments.”

A practical implementation can present invocation records alongside their policy context: the requesting workflow or agent, the connector involved, the operation requested, the result category, and the associated session or user identity where appropriate. The exact telemetry depends on the product and privacy requirements, but the goal is consistent: make the boundaries that matter visible enough to review.

Prompt injection remains a connector security problem

Explicit per-call orchestration creates better enforcement points, but it does not neutralize prompt injection. OpenAI’s 2026 system cards continue to evaluate robustness against prompt injection on connectors and search/function-calling. That continuing focus shows that connector security remains a first-class safety problem even as protocol and product controls evolve.

The risk is especially relevant when tools retrieve or act on external content. Indirect prompt injection can place adversarial instructions inside material that an agent reads. If the agent interprets that content as authority to change its behavior or use tools inappropriately, the problem is not merely a bad answer; it may become an unsafe connector action.

OpenAI’s GPT-5.6 safety reporting shows strong gains in indirect prompt-injection defense, including a Connectors robustness line that reaches very high levels in later columns of the evaluation table. This indicates that connector-related safety defenses are improving materially. It should not be interpreted as a reason to remove system controls: evaluation performance and production authorization are complementary safeguards.

Use the call boundary for defenses that the model cannot own alone

Models can be evaluated and improved against adversarial instructions, but the execution environment must still enforce what is allowed. A per-call tool boundary provides a practical place to apply controls that do not depend solely on model behavior:

  • Validate that requested operations are in the connector’s approved action set.

  • Constrain arguments according to the operation’s expected schema and scope.

  • Keep high-impact actions distinct from low-risk retrieval where the product requires it.

  • Record the tool request and outcome for investigation and policy refinement.

  • Test end-to-end flows with adversarial content, not only clean prompts.

OpenAI says GPT-Red is an automated red-teaming model trained with self-play reinforcement learning to discover new attack strategies, particularly prompt-injection attacks that exploit human-like behavior and probe deeper weaknesses in tool-using systems. The broader lesson is that connector hardening requires ongoing adversarial testing. A static review at initial deployment is unlikely to cover the evolving behaviors exposed by real tools, content, and users.

Security at this layer is a systems responsibility. OpenAI’s cybersecurity action plan describes an accelerating threat environment that includes large-scale ransomware, supply-chain compromise, and state-backed cyber activity. In that context, connector design cannot be reduced to model selection. It must include protocol boundaries, permission architecture, operational monitoring, and an incident-ready understanding of what each external action did.

How to migrate from session-bound tools to per-call orchestration

A migration should begin with architecture discovery, not a mechanical replacement of a deprecated identifier. The immediate trigger may be OpenAI’s documented deprecation of connector_id for models released after September 1, 2026, but the enduring goal is to identify which assumptions currently depend on long-lived session coupling.

Start by mapping the actual lifecycle of a tool request. Identify where connector identity is selected, where authentication material is managed, which service performs authorization, how a tool result returns to the agent, and what logs or audit events exist. Teams frequently find that these responsibilities are distributed across session creation, routing middleware, agent prompts, and tool code.

Migration sequence

  1. Inventory session-bound assumptions.

    Document where a session implicitly grants connector availability, preserves routing affinity, or carries authority that is not rechecked at execution.

  2. Define a stable invocation contract.

    Represent a tool request with the tool name, structured arguments, requesting agent or workflow, correlation information, and the connector policy context needed by the executor.

  3. Move connector policy to a governed control plane.

    Align connector publication and role controls with the workspace administration model. Keep runtime workers focused on evaluating and executing a specific request.

  4. Build a dedicated execution boundary.

    The component that executes the action should validate the request, call the external service through the approved path, and produce a bounded result for the model or agent.

  5. Preserve source-system permissions.

    Where the source system provides its own permission model, do not replace it with an overly broad workspace assertion. The SharePoint deployment example demonstrates the value of enforcing existing permissions alongside centrally managed setup.

  6. Add reviewable telemetry.

    Correlate the session, user or workflow, connector, requested action, policy outcome, and execution result without treating them as the same object.

  7. Red-team the full path.

    Test indirect prompt injection and unsafe tool-use scenarios across retrieval, handoff, policy evaluation, and execution.

Choose the right level of state

The objective is not “stateless at all costs.” Some state is essential: conversation continuity, cache entries, workspace configuration, and source authorization context may all be legitimate. The key engineering choice is to avoid placing unnecessary, long-lived state in the high-volume data plane or making a single session owner responsible for more than it needs to own.

Cached state and relay-style handling, as described in OpenAI’s voice architecture work, offer one direction for reducing data-plane burden. For agent systems, the equivalent may be a policy-aware executor that resolves approved context for a call while a routing layer remains lightweight. The exact implementation will vary by workload, but the separation of control-plane governance from data-plane execution is a durable design principle.

What per-call connector architecture does and does not solve

Per-call connectors improve the structure of a system’s security and scaling decisions. They make it easier to attach controls to discrete events, to reduce the operational drag of sticky session ownership, and to align runtime behavior with centralized connector governance. They also fit naturally with multi-agent systems, where context handoffs and specialist tools need explicit boundaries.

They do not automatically determine which data should be available, which agents deserve access, or whether a requested action is appropriate for a business process. Those decisions still require policy ownership, source-system permissions, careful tool design, user experience choices, and testing against adversarial behavior.

  • Use sessions for interaction continuity and identity context where needed.

  • Use workspace controls for connector publication, roles, and administrative rollout.

  • Use per-call execution boundaries for action-specific validation, routing, and evidence.

  • Use source-system permissions as an active layer, not as an assumption.

  • Use ongoing evaluation and red-teaming to test the gap between intended policy and actual agent behavior.

The direction of recent changes is clear enough to guide design now. OpenAI’s per-call Realtime tool flow, the deprecation of older connector identifiers for newer models, workspace RBAC and connector publication, intentional connector enablement, expanded session visibility, and work on prompt-injection resilience all favor explicit, reviewable boundaries over ambient, long-lived trust.

For platform teams, the most useful next step is to trace one production agent workflow from user session to connector call to external result. Mark every place where the system relies on an implicit session assumption, then decide whether that assumption belongs in the control plane, in the source system, or at the per-call execution boundary.

A scalable connector strategy is not defined by eliminating state. It is defined by keeping state and authority in the right places: governed at the workspace level, visible at the session level, constrained by source permissions, and evaluated explicitly whenever an agent asks a tool to act.

Per-Call Connectors: Security and Scale