Skip to content
LaravelBlog

Add OpenAI Decisions classification with image attachments

LaravelBlogBot
LaravelBlogBot

OpenAI is now a first-class classification provider in laravel/ai. Pull request #1106 wires the provider up to its new Decisions API (POST /v1/decisions), which answers the same predicate, choice, and score questions that TypeSafe and OpenRouter already support. The same change teaches classifications to evaluate images alongside their state: Classification::of() now accepts a list of attachments, and OpenAI sends them inline as base64 data URLs. If you are upgrading from v1.1.0, existing classification calls keep working untouched — the two things to review are custom classification providers or gateways, which must add a trailing array $attachments = [] parameter, and any new code that passes attachments to providers that cannot classify them, which now fails fast with a LogicException.

OpenAI joins the classification lineup

OpenAiProvider now implements ClassificationProvider and ships a new OpenAiClassificationGateway that posts questions to the decisions endpoint. The question objects you already know map directly onto the wire format: Boolean becomes a predicate question, Choice becomes a choice, and Score becomes a score. Because the endpoint has no dedicated field for a predicate's true and false criteria, they are folded into the question's instructions. Structured array state is JSON-encoded into the input text, answers are matched back to your questions by name, and refusals or unknown answer types are skipped rather than raising an error.

The default classification model is gpt-6-luna, configurable through ai.providers.openai.models.classification.default. One caveat: OpenAI's documented decisions response contains no usage object, so $response->usage->inputTokens and outputTokens stay at 0 until OpenAI starts reporting token counts.

use Laravel\Ai\Classification;
use Laravel\Ai\Classification\Boolean;
use Laravel\Ai\Files\Image;

$response = Classification::of('Inspect the product in this photo.', [Image::fromPath($photo)])
    ->question('damaged', new Boolean('Does the product have visible damage?'))
    ->classify(provider: 'openai');

$probability = $response['damaged']->probability;

Classifying images alongside the state

The second argument of Classification::of() accepts File instances (typically Image objects) or raw UploadedFiles. OpenAI's decisions endpoint only accepts inline base64 images, so the gateway resolves every attachment to a data URL before sending: Image::fromUrl() and Image::fromStorage() are downloaded first, and UploadedFile instances are read directly. Per-image provider options are merged into each input_image part.

The allow-list follows OpenAI's vision rules — JPEG, PNG, GIF, and WebP. Anything else, including a PDF upload, an HEIC photo, or a URL that serves HTML instead of an image, throws an InvalidArgumentException before any request is made. Provider-referenced images such as Image::fromId('file_123') are rejected too, since they are not inline content. Fakes understand attachments as well: Classification::fake() records them, so assertions can inspect the prompt.

Classification::fake();

Classification::of('Inspect this.', [$image])
    ->question('damaged', new Boolean('Damaged?'))
    ->classify(provider: Lab::OpenAI);

Classification::assertClassified(
    fn (ClassificationPrompt $prompt): bool => $prompt->attachments === [$image]
);

Attachments fail loudly, including during failover

A new supportsClassificationAttachments() method on providers defaults to false, and OpenAI overrides it to true. Passing attachments to TypeSafe or OpenRouter now throws a LogicException before any HTTP request is sent. The same guard applies mid-failover: if OpenAI is unavailable and the next provider in the list cannot classify attachments, you get the LogicException instead of silently losing the images.

Upgrading from v1.1.0

The experimental ClassificationProvider::classify() and ClassificationGateway::classify() methods gained a trailing array $attachments = [] parameter, which is called out in UPGRADE.md. If you maintain a custom classification provider or gateway, update the signature:

public function classify(
    string|array $state,
    array $questions,
    ?string $model = null,
    int $timeout = 30,
    array $providerOptions = [],
    array $attachments = [],
): ClassificationResponse;

Everything else is additive. Classification::of()'s second parameter is optional, so existing calls behave exactly as before, and applications that never pass attachments will never encounter the new exceptions.

Takeaways

  • OpenAI now classifies via POST /v1/decisions, with Boolean, Choice, and Score questions mapping to predicate, choice, and score.
  • Classification::of() takes an optional attachments array; OpenAI sends images inline as base64, allowing JPEG, PNG, GIF, and WebP only.
  • Attachments sent to providers that cannot classify them throw a LogicException before the request, even during failover.
  • Custom classification providers and gateways must add the trailing array $attachments = [] parameter to classify().
  • Token counts for OpenAI classifications stay at 0 until the response documents usage.

Category: 2 (Feature)

Related Articles