laravel/ai: Fix terminal Responses API stream handling

Fix (3) · Upgrade notes (11)

Streaming against the OpenAI and Azure Responses APIs had a quiet failure mode in v1.2.0: when a generation hit its output limit, the provider ended the stream with a response.incomplete event the SDK didn't recognize. Usage vanished, finish reasons went missing, and — most concerning — tool calls taken from a truncated response could still be executed. PR #1113 fixes terminal-event handling in the shared OpenAI stream parser (which also backs the Azure Responses implementation): incomplete responses now keep their usage and map to a real finish reason, tool calls are skipped when the overall response is incomplete, and failed or prematurely closed streams surface as errors instead of ending in silence.

The problem: a terminal event nobody was listening for

The Responses API signals "the model stopped before finishing" with response.incomplete — typically when a reply runs into max_output_tokens or a content filter trips:

event: response.incomplete
data: {"type":"response.incomplete","response":{"status":"incomplete","incomplete_details":{"reason":"max_output_tokens"},"usage":{"input_tokens":42,"output_tokens":128}}}

Before this patch, the streaming loop in HandlesTextGeneration only recognized response.completed as a terminal event. response.incomplete matched no branch and was dropped, so three things went wrong. Token usage attached to that final event never reached your StreamEnd, under-counting every truncated generation. The finish reason was lost — consumers simply saw the text stop. And because the step never formally ended, tool calls already emitted inside the partial response were still collected and executed, even though the response they belonged to was incomplete.

What changed

The patch reworks three behaviors in the gateway:

• Incomplete responses are terminal. The stream loop now treats response.incomplete like response.completed: the response payload is captured, usage flows into StreamEnd, and extractFinishReason() maps incomplete_details.reason to SDK finish reasons — max_output_tokens becomes FinishReason::Length, content_filter becomes FinishReason::ContentFilter, and anything else falls back to FinishReason::Unknown.
• Tools don't run on truncated responses. When the overall response is incomplete, emitted function calls are preserved but not executed. The regression test asserts the agent makes exactly one HTTP request in that scenario — no follow-up tool round-trip against a response the provider cut short.
• Failures are reported, not swallowed. response.failed events now yield an Error stream event carrying the provider's code and message, whether the error is nested under response.error or sent top-level. A stream that closes without any terminal event no longer resolves to an empty success: the parser returns no step and the SDK throws StreamErrorException with the message "The provider ended the stream without completing the step."

What this looks like in your code

The happy path is unchanged — you just start receiving signals that were previously dropped. The event and exception names below mirror those the gateway yields (and that the PR's tests exercise):

try {
    foreach ($ai->stream('Summarize this 40-page contract...') as $event) {
        if ($event instanceof Error) {
            report("Provider error [{$event->type}]: {$event->message}");
            break;
        }

        if ($event instanceof StreamEnd) {
            logger()->info('Generation finished', [
                'finish_reason' => $event->reason,          // 'length', 'content_filter', ...
                'input_tokens'  => $event->usage->inputTokens,
                'output_tokens' => $event->usage->outputTokens,
            ]);
        }
    }
} catch (StreamErrorException $e) {
    // The provider closed the stream without sending a terminal
    // response.completed / response.incomplete event.
    report($e);
}

If you branch on finish reasons, length is now a value you will actually see on truncated output — a natural trigger for "continue the thread" or "raise max_output_tokens" logic in agent loops.

Upgrading from v1.2.0

There is no migration work: no configuration, method signatures, or public API changed. What changes is runtime behavior in edge cases, and three of them are worth checking against your error handling:

1. Streams that previously ended silently without a terminal event now throw StreamErrorException. If you consume streams inside try/catch, that path may start firing.
2. Tool calls inside a response.incomplete response no longer execute. If a workflow relied on tools running despite truncation, it will stop — deliberately. Treat a FinishReason::Length (or ContentFilter) end as the signal to continue the conversation or adjust limits.
3. response.failed now surfaces as an Error event; add a branch for it if you only handled text deltas and StreamEnd before.

If you were already reading usage and finish reasons from StreamEnd, you'll simply notice more complete and accurate data. Regression tests in the PR cover both the OpenAI and Azure providers, including the three incomplete-reason mappings and the unterminated-stream exception.

Takeaways

• response.incomplete from OpenAI/Azure is no longer ignored: usage is preserved and incomplete_details.reason maps to FinishReason::Length, ContentFilter, or Unknown.
• Tool calls are skipped when the overall response is incomplete, preventing execution of actions derived from truncated output.
• response.failed yields an Error event, and streams ending without a terminal event throw StreamErrorException.
• No API changes to migrate; edge-case behavior becomes stricter and more observable.

Sources

• Pull request #1113
• Release v1.2.1