Streaming tool-call assembly: execute only a finished call

A model to inspect

Safe streamed call

01Buffer by call ID
02Receive terminal event
03Parse and validate
04Check current turn
05Dispatch or reject

Partial arguments never reach an executor.

Original conceptual diagram · not a live trace or measured result.

Fragment streams are not proposals

A streamed tool argument delta is transport data, not an instruction to act. Key each buffer by the response or turn identity, output item identity, and tool-call identity supplied by the protocol. Append only fragments from that exact tuple. Keep text output and tool arguments separate even when they arrive interleaved. A buffer should have a byte limit, syntax state, received-event sequence, and terminal status; it should not be exposed as a live object to an executor.

The Responses streaming reference documents server-sent events and function-call argument streaming. It is provider-specific documentation, so use its event names only in an adapter. The portable rule is original guidance: fragments must remain inert until the adapter knows that one call has ended under its protocol.

Require a terminal transition

Write an explicit state machine: absent, collecting, terminal candidate, validated, dispatched, cancelled, failed. A terminal event permits parsing; it does not itself authorize execution. Parse the assembled arguments once, enforce the declared schema, bind them to the active turn and authenticated actor, and pass them through the existing permission and confirmation boundary. Persist the terminal event ID so reconnect delivery cannot dispatch it twice.

Hypothetical example: fragments for book_slot arrive as {"day":"Tue" and later ,"time":"10:00"}. The client may display a non-actionable “preparing request” cue, but it must not reserve anything after the first fragment. Only the completed, validated proposal can request a confirmation.

Make cancellation inert

Cancellation, supersession, connection loss, malformed data, unknown tool name, and size limit all move the buffer to a non-dispatching terminal state. Remove it from active assembly and retain only bounded diagnostic metadata under the product’s retention policy. A late terminal event for a cancelled turn must be rejected by turn ownership, not revived because its JSON happens to parse.

Counterexample: a user says stop, then a delayed final delta completes a payment call. If the dispatcher checks only arguments_complete, it can make an action after the user withdrew attention. The dispatcher must additionally check the current turn’s cancellation and confirmation state at execution time.

Validate per call, not per stream

One response can contain more than one candidate call. Never concatenate all argument deltas into a single response-wide string. Validate each call against its own tool declaration, sequence, size allowance, and policy. If ordering is not guaranteed by a protocol, reject or reassemble by its documented position rather than guessing from arrival time. A repeated delta is a delivery condition, not evidence that the user requested the action twice.

Record an assembly ledger with the tuple key, count, terminal reason, parse result, validation result, and dispatch decision. It lets an operator distinguish a malformed stream from a permission rejection without retaining sensitive argument values in broad logs.

Exercise awkward event orders

Fixtures should cover two calls interleaved, duplicate deltas, terminal before a missing fragment, malformed JSON, cancellation before terminal, reconnection replay, a tool renamed mid-release, and a new user turn while old fragments arrive. Assert that no fixture causes a mutation before terminal validation and that an old tuple cannot affect a newer turn. Test visible status separately from action safety.

Acceptance checks: every executor input names a completed call ID and originating turn; each buffer has a bounded lifecycle; every cancellation prevents dispatch; and each final call is schema- and business-validated exactly once. Streaming can reduce perceived wait, but it cannot make incomplete intent trustworthy.

Make operators able to stop assembly

Expose a bounded operator control that can quarantine a response, disable a named tool, or force all in-flight buffers for a tenant into inert cancellation. The control must not itself re-run the fragment or reveal arguments to a broad audience. Log the reason and the affected call keys so a reviewer can distinguish a provider interruption from a product safety hold.

Acceptance checks include an emergency stop during collecting and during terminal validation. In both paths no executor receives the call, the user sees a truthful retry or handoff route, and a subsequent new turn starts with a fresh call identity rather than inheriting partial arguments. Inspect the quarantine audit after restart: buffers must stay terminal, never become candidates because an adapter reconnects.

Take it into the review

Call assembly ledger

Call key	State	Terminal event	Dispatch
turn-8/call-a	collecting	none	inert
turn-8/call-a	validated	completed	await confirmation
turn-8/call-b	cancelled	late completed	reject
turn-9/call-a	failed	malformed	reject

A starting artifact to adapt to your service—not a ready-made policy, compliance certificate or test result.

Primary reading

Sources and limits

These links support the architecture, policy, or product behavior discussed above. Vendor documentation describes vendor features; it is not independent proof of performance. Current details should be rechecked before a production decision.

What the sources establish

OpenAI function calling guide

The reference describes server-sent streaming events, including function-call argument streaming.

Limits: It documents one API and does not establish an application execution policy.

Checked 2026-09-19 · OpenAI · source publication date not established.

Open original source ↗

W3C Trace Context

Trace Context provides a way to propagate a trace identity across services.

Limits: Trace correlation does not prove event ordering or user intent.

Checked 2026-09-19 · W3C · source publication date 2021-11-23.

Open original source ↗

Procedures and worked examples are editorial synthesis. Preparation/review dates are not claimed historical publication dates.