Skip to content
LaravelBlog

laravel/framework: Improve Cloud job retries

LaravelBlogBot
LaravelBlogBot , 5 mins read

Contributed by timacdonald

Pull request #61584 improves how queue:retry handles large volumes of failed jobs by allowing a failed job provider's find() method to return a collection of jobs instead of a single job. Until now, retrying many jobs meant passing every individual ID on the command line, and each ID was resolved one at a time. This PR enables a provider — most notably the Laravel Cloud failed job provider — to resolve a single identifier into a whole batch of failed jobs, fetched lazily and page by page.

What changed

The queue:retry command no longer assumes find($id) returns exactly one job. If the provider returns an Enumerable (a Collection or LazyCollection), the command iterates over it and retries each job, using the collection key when dispatching JobRetryRequested and calling forget(). A single job object is still supported — it is wrapped in a one-element collection internally — so existing providers keep working unchanged.

The Cloud provider in Illuminate\Foundation\Cloud now returns a LazyCollection from find(). Responses are expected to use a paginated shape (data plus links.self / links.next), versioned via a Cloud-Payload-Version: 1 header. Responses without the header (the old single-job shape) are still handled for compatibility. Each yielded job is keyed by {url}:{job-id}, and forget() now clears that key from the in-memory cache. The underlying HTTP fetch was also reworked: it runs via Http::withoutGlobalConfiguration, retries up to 10 times with exponential backoff starting at 500 ms, skips global client configuration, and gives up immediately on a 403 instead of retrying.

Why it matters

Retrying at scale was previously impractical on Cloud:

php artisan queue:retry c5963249-ac84-4b3f-808d-f7b095271fa7 c8a97d57-f4d2-4dea-b87e-bf842002cfd8 ...

At ten thousand IDs the command becomes unusable, and because each ID hit find() separately, the provider issued one HTTP request per job. Now a single identifier can represent a group of jobs, and because the result is a LazyCollection, pages are only fetched when the previous page's jobs have actually been pushed back onto the queue. The new tests demonstrate this ordering explicitly: page 1 is fetched, job-1 is pushed, then page 2 is fetched, and so on — memory and request counts stay flat regardless of batch size.

Using it: retrying batches of failed jobs

On Laravel Cloud, retrying a paginated batch is now a single command:

php artisan queue:retry https://cloud.laravel.com/api/failed-jobs?page=1

Third-party providers can opt in by returning an Enumerable from find(). For example, a database provider that supports batch identifiers:

use Illuminate\Support\LazyCollection;

public function find($id)
{
    if (str_starts_with((string) $id, 'batch:')) {
        $batchId = substr((string) $id, 6);

        return new LazyCollection(function () use ($batchId) {
            foreach (
                DB::table('failed_jobs')->where('batch_id', $batchId)->cursor()
                as $row
            ) {
                yield $row->uuid => (object) [
                    'id' => $row->uuid,
                    'connection' => $row->connection,
                    'queue' => $row->queue,
                    'payload' => $row->payload,
                ];
            }
        });
    }

    return parent::find($id);
}

The yielded key is what forget() receives, so it should be the identifier your provider can actually delete. Batches and ordinary IDs mix freely:

php artisan queue:retry "batch:9f2c" 5 7

Upgrade impact

Most applications using the database, redis, sqs, or dynamodb failed job providers are unaffected: find() still returns a single object, the command wraps it, and behavior is identical. The command also gained a clearer error path — an ID that resolves to an empty collection now reports "Unable to find any failed jobs with ID [id]" while remaining jobs continue to be retried.

If you interact with the Cloud failed job provider directly, three changes to be aware of:

  • find() now returns a LazyCollection, not a job object. Iterate it, or use ->first() / ->all() if you need individual jobs.
  • The HTTP request and payload decryption are deferred until iteration begins, so DecryptException and RequestException surface while looping, not at find() call time.
  • Because the fetch runs under Http::withoutGlobalConfiguration, global HTTP client middleware no longer applies to these requests, and 403 responses fail fast instead of being retried.

Unsupported Cloud-Payload-Version header values now throw a RuntimeException rather than being silently accepted.

Takeaways

  • queue:retry now accepts IDs that resolve to a Collection or LazyCollection of failed jobs, alongside single-job IDs and all.
  • The Cloud failed job provider returns a lazy, paginated LazyCollection; each page is fetched only after the previous page's jobs are retried.
  • find() on the Cloud provider no longer performs network work until you iterate — catch decryption and HTTP exceptions around the loop.
  • HTTP retries moved to 10 attempts with exponential backoff (500 ms doubling), and 403s abort immediately.
  • Custom providers can add batch support by returning an Enumerable from find(); the collection key is passed to forget(), so make it the deletable identifier.
  • Existing single-job providers require no changes to keep working.

Sources

Related Articles