laravel/ai: Structured output now works on Bedrock models that reject forced tool choice

Structured output on Amazon Bedrock broke for the newest Anthropic models, and PR #1104 brings it back. Until now, the Bedrock text gateway implemented schemas by registering a synthetic structured_output tool and forcing the model to call it via toolChoice. Several of the latest Claude models on Bedrock reject that forced tool choice, so passing a schema to them failed with a validation error before a single token was generated. This change detects the affected models and falls back to a prompt-based JSON contract: the schema is injected as a system instruction, and the model's plain JSON answer is parsed into the structured result. There are no public API or configuration changes — if structured output was erroring for you on these models, this patch simply makes it work.

What was breaking

The gateway previously treated every structured request the same way. Your schema was compiled into a hidden structured_output tool, the request was sent with toolChoice: ['tool' => ['name' => 'structured_output']], and the tool call's input became the structured payload. That approach still works for older Claude generations on Bedrock, but the newest models — matched by the IDs claude-sonnet-5-5, claude-opus-5-5, claude-fable-5-1, and claude-mythos-5-1 — reject forced tool choice, and Bedrock rejects the whole request. Issue #1101 is exactly this failure mode: schema-based generation against these models throws instead of returning structured data.

How the fallback works

BedrockTextGateway::buildStepBody() now checks the model ID against a new MODELS_REJECTING_FORCED_TOOL_CHOICE list (via the rejectsForcedToolChoice() helper) before building the request. For flagged models, three things change:

• No synthetic tool is registered and no forced toolChoice is sent. Instead, the schema — wrapped in an ObjectSchema so it serializes as a proper JSON object schema — is appended to your system prompt as a strict instruction:

JSON schema:
{"type":"object","properties":{"symbol":{"type":"string"}}, ...}

You must answer with only a JSON object that matches the JSON schema above.
Do not wrap it in markdown fences or include any other text.

• In parseTextResponse(), when the request was structured and the model answered with text instead of tool calls, that text is treated as the structured output, so $response->structured is populated exactly as before.
• Your real tools still work. Because a specific tool can no longer be forced, your own tools are sent with Bedrock's default auto tool choice, the regular tool-call loop runs first, and the model answers with the JSON document on its final step. A new feature test covers this two-step flow end to end.

Detection is a substring match on the model ID, so cross-region inference profile IDs such as us.anthropic.claude-sonnet-5-5 are covered automatically. The trade-off, noted in the code, is that application inference profile ARNs that alias the model under a custom name are not detected and keep using the old forced-tool path.

What your code looks like

Nothing changes. You keep passing a schema exactly as before:

use Laravel\Ai\Facades\Ai;
use Laravel\Ai\JsonSchema\JsonSchemaTypeFactory;

$response = Ai::text()
    ->using('bedrock', 'us.anthropic.claude-sonnet-5-5')
    ->withSchema([
        'symbol' => (new JsonSchemaTypeFactory)->string(),
    ])
    ->generate('Which element is used in steel?');

$response->structured; // ['symbol' => 'Fe']

On an affected model this now sends a system array with two entries — your prompt, then the JSON instruction — and no toolConfig at all, which is precisely what the updated test suite asserts. On every other Bedrock model the old path is untouched: the tests still verify that a regular model receives the forced structured_output tool choice and no injected system instruction.

Upgrade notes

• No migrations or config changes; the fix is internal to the Bedrock gateway.
• Prompt-based JSON is slightly less deterministic than forced tool calls. The strict instruction mitigates this, but if you relied on the harder guarantees of tool-forced output, be aware the affected models now produce structured data from their text answer.
• The injected instruction adds a small number of input tokens, so usage numbers for schema requests to these models will tick up a little.
• Combining real tools with a schema on an affected model now uses auto tool selection for your tools rather than a forced choice; multi-step loops still run before the final JSON answer.
• Test-only change: fakeBedrockConverseSequence() can now record outgoing requests, which is how the new assertions inspect raw Converse payloads. This only matters if you copied that helper into your own test suite.

Takeaways

• Structured output with a schema now works on Bedrock models that reject forced tool choice (claude-sonnet-5-5, claude-opus-5-5, claude-fable-5-1, claude-mythos-5-1).
• Affected models receive the schema as a system instruction and answer with raw JSON, which the gateway parses into $response->structured.
• Real tools keep working via auto tool choice, and tool loops still run before the final JSON answer.
• No public API changes; all other models and providers retain the existing forced-tool behavior.

Sources

• Pull request #1104
• Release v1.2.0