Diagnosing and Restarting Background Jobs

Almost nothing in LLM Metrix happens inside the request that asked for it. A scan, a digest email, a Slack post, a Search Console sync, a data export, a billing reconciliation — each is a job placed on a queue and run by a worker later. That is why a scan returns immediately and finishes a minute afterwards, and it is why six other tutorials, when something looks wrong, tell you to go and check Activity.

This is the tutorial for that page. It covers what the statuses mean, which failures fix themselves before you get there, and the one property that makes the Retry button safe to press: restarting re-drives the same job rather than creating a new one, so a scan cannot be charged twice. A restarted notification is a weaker promise, and Step 5 says exactly how weak.

Step 1: Open Activity and know what the list is

Activity sits in the Workspace group of the sidebar, under the heading Recent activity. It shows your fifty most recent jobs, newest first, and refreshes itself every fifteen seconds while the tab is open — so a job you just restarted changes status under you without a reload.

One thing to expect before it confuses you: the list is scoped to the workspace in the switcher, not to the project selected beside it. Switch workspaces and the list changes; switch projects and it does not, because most job types belong to a workspace rather than to one project. If you consult for several clients, you see one client at a time.

Two kinds of row ignore that filter, and should: a data export and the notification emails sent to your address belong to you rather than to any workspace, so they appear whichever workspace is selected. There is nowhere else they could be shown.

Each row gives you four things: the job type, a status pill, how long ago it was created, and — once a job has retried at least once — an attempt count. Failed and given-up rows additionally carry the error, truncated to one line with the full text on hover.

Step 2: Recognise the job types

The internal type names are mapped to plain labels, and the list is short enough to learn:

Label	What produced it
Scan	A visibility scan, whether you pressed Run scan or the monitoring cron did
Email notification · Slack notification · Webhook notification	One delivery each, per channel, per alert
Recommendations	The GEO action plan precomputed after a scan completes
Billing sync	Metered usage reported to Stripe, and seat/domain quantity syncs
Data export	Your GDPR data export being assembled
Account deletion	A confirmed account purge running after its cancellation window

Anything whose type has no friendly label — a newer job kind, say — is shown with its raw name rather than hidden. A page that silently dropped rows it did not recognise would be worse than one that occasionally shows you gsc_sync.

Step 3: Read the six statuses, and act on two of them

Pill	Meaning
Queued	Committed and waiting for a worker
Running	A worker has claimed it
Completed	Done
Failed	An attempt failed; more attempts remain
Gave up	Attempts exhausted — the terminal failure state
Canceled	Withdrawn, typically because its workspace was scheduled for deletion

The Retry button appears on exactly two of these: Failed and Gave up. That is deliberate, and it is the most useful thing on the page to understand.

Failed does not mean stop. A failed job is retryable and something is already going to retry it. It is showing you a bad attempt, not a final outcome. The right response to a Failed row that is minutes old is to wait and refresh.

Gave up does mean stop. Attempts are exhausted, nothing further will pick it up, and it will sit there until you or an operator does something. When a job reaches this state the platform also raises an ops alert, so you are not the only line of defence — but you are the fastest one.

Completed is not always complete. A scan row can carry an amber Partial coverage line under it naming an engine that returned no answer at all, prompts dropped at the per-scan ceiling, or prompts abandoned when the run hit its time budget. This is the one status that hides something: the job succeeded, the score moved, nothing errored — and an engine that was down for the whole run looks exactly like an engine that had nothing to say about your brand. Those mean opposite things, so read that line before you read the score it produced. The two prompt counts are shown separately and never added together, because one is a limit you configured and the other is a symptom of a slow run.

Step 4: Let the automatic recovery do its work first

Three mechanisms run continuously underneath this page, and between them they resolve most of what you would otherwise click Retry on:

• Backoff retries. A failed job is rescheduled with a growing delay before each new attempt, rather than hammering a provider that is already struggling. That pattern is standard practice for exactly the reason Google’s SRE book gives in its chapter on addressing cascading failures: aggressive immediate retries during a partial outage are how a slow dependency becomes a dead one.
• A dispatcher, every minute. Every job is written to the database first and published to the queue second. If the publish fails, the row is still committed, and a sweep picks it up within the minute. A row that seems stuck in Queued for thirty seconds is almost always this, mid-recovery.
• A reaper, every few minutes. It re-drives three kinds of stranded row: a job Running for over ten minutes (its worker timed out or was killed), a Failed job that never got redelivered, and a job that was published but never claimed at all. Anything that has genuinely run out of attempts is moved to Gave up here, which is what makes that state reachable rather than theoretical.

The practical rule: give a Failed job five minutes before intervening. Clicking Retry immediately does not make it recover faster, and on a scan it spends real money to find that out.

Step 5: Restart without fear of double-charging

Pressing Retry does not enqueue a new job. It resets the existing row and re-drives it, keeping the same job id — and every side-effect in the system is keyed on that id or on the job’s deduplication key. Concretely:

• A scan cannot be charged twice. The credit charge is written in the same database transaction as the scan itself, guarded on the job id. If the original attempt got far enough to commit a scan, the restart adopts that scan instead of running the engines again.
• A notification may well be sent twice, and the page says so. This is the honest half. Email deliveries carry an idempotency key, so a prompt restart is collapsed by the provider rather than landing in someone’s inbox twice — but that key has a lifetime, and a restart hours later falls outside it. Slack and webhook deliveries carry no de-duplication at all. Restarting a notification is therefore a decision about whether a duplicate is worse than a miss, not a free action like restarting a scan.

The scan half is idempotence in the ordinary sense HTTP defines it — RFC 9110 §9.2.2 describes a request whose repetition has the same effect as a single one — applied to a queue rather than to a method. The queue promises at-least-once delivery, never exactly-once, so every handler has to survive being run again. Where a handler can guard itself on the job id it does, and the restart costs nothing; where the effect leaves our systems and lands in a third party’s inbox or channel, the guard is only as good as that party’s own de-duplication window. Knowing which of the two you are restarting is the whole skill.

Step 6: Know what a restart still checks

A restart is not a bypass. Scans re-run the same paid pipeline, so they honour the same gates the original request did:

• Viewers cannot restart a scan. Read-only members can look at the page and press nothing that spends credits.
• A payment-suspended workspace is refused, so a failing card cannot keep accruing usage through the back door.
• A workspace pending deletion is refused — its queued jobs were cancelled on purpose.
• Restarts are throttled to five per ten minutes per person. Retry on a job that keeps failing is otherwise an unmetered way to burn AI spend.
• A Running job is refused with a message saying so, unless it has been running long enough to be presumed dead.

If you get one of these, the message names the reason. None of them is a bug report.

Step 7: Know when restarting is the wrong move

Some failures are information, and re-driving them just reproduces the failure:

• A bounced or complained-about email. A hard bounce is a delivery outcome, not a retryable fault — the address is added to a suppression list and later mail to it is dropped deliberately. Restarting cannot make a dead mailbox exist.
• A failure whose error names a missing configuration. An unconfigured provider, a revoked Google grant, a retired model id. The job will fail identically until the configuration changes. A revoked connector in particular is fixed by reconnecting it, not from this page.
• A repeatedly dead billing sync. That one is a revenue-reporting problem rather than a user one, and it already alerts operators. Restarting is harmless; it is just unlikely to be the fix.

Read the error text before clicking. It is the whole reason the page shows it.

Finally, note what this page is not: it is not an audit log and not permanent. Completed and given-up jobs are removed after ninety days. The immutable record of what was done to your account lives in the workspace audit log, which is retained on a different basis entirely.

Step 8: Where to go next

• Real-time alerts — the feature whose deliveries make up most of the notification rows on this page.
• Setting Up Smart Alerts — where those deliveries are configured, including the four cases in which nothing is sent at all.
• Getting Started with LLM Metrix — the first scan whose job is usually the first row anyone sees here.
• Monitoring AI Traffic and Crawlers — log uploads are parsed in a background job, so this is the page you follow them on.