Background Jobs

Offload work to background jobs for email sending, webhook delivery, credit resets, and RAG indexing. The job adapter is selected by JOB_PROVIDER.

Providers

Enqueuing Jobs

import { enqueue, enqueueEmail } from '@/lib/jobs';

// Generic job
await enqueue('webhook-retry', { webhookId: 'wh_123', attempt: 1 });

// Shorthand for emails
await enqueueEmail('user@example.com', 'Welcome!', emailHtml);

Pass a stable jobId when the same logical work may be enqueued more than once:

await enqueue('rag-index', document, { jobId: `rag:${document.sourceId}` });

Adapters treat a repeated jobId as the same queued job. This is an enqueue-level guard; handlers should still be safe to retry.

Built-in Jobs

Cron Jobs

Cron jobs run automatically when the app starts. They use the same job adapter.

Switching to BullMQ

For production, use BullMQ with Redis for durable, concurrent job processing:

JOB_PROVIDER="bullmq"
REDIS_URL="redis://localhost:6379"
JOB_QUEUE_NAME="my-app"   # optional, defaults to `appConfig.appName`

Worker Process

In production, run the worker as a separate process alongside your Next.js server. The worker picks up jobs from Redis and processes them independently.

For Kubernetes deployments, the Helm chart includes a dedicated worker Deployment. For Docker, add a separate service in docker-compose.yml.

Important: The memory adapter processes jobs in-process and loses pending jobs on restart. Always use bullmq in production.

BullMQ keeps exhausted jobs in Redis' failed set (removeOnFail retains the latest failed jobs). Treat that set as the dead-letter queue: alert on growth, inspect the error and payload, then replay only after the handler's side effects are idempotent. The failed count is exposed through getStats() for monitoring.

Environment Variables

* Required when JOB_PROVIDER=bullmq.