Background jobs

ReleaseDesk runs background work through BullMQ on Redis, for both SQLite and Postgres installs. Redis is required unless the process runs with RD_JOBS_MODE=api.

What runs where

Work	Mechanism	Schedule
Outbox relay (mail, Play, App Store Connect)	scheduler outbox-relay	every 5 s
Mail delivery, Play and App Store Connect operations	queued job per outbox row (mail, play, asc queues)	on demand
Delta generation	scheduler delta-scan plans missing deltas, one queued job per (artifact, base release)	every 10 min
Play / App Store Connect status sync	schedulers play-sync, asc-sync	every 10 / 5 min
Membership refresh, stats retention, artifact retention, registry GC	daily schedulers	03:00, 03:30, 04:00, 04:30 UTC
Dunning, usage warnings, storage add-on right-sizing	schedulers, only with RD_CLOUD_MODE	hourly

Schedulers are registered with upsertJobScheduler under fixed ids, so restarts and multiple replicas never create duplicates. Calendar schedules always carry an explicit tz (UTC). Schedulers that no longer apply (for example cloud-only jobs on a self-hosted instance) are removed at startup.

The stats buffer is not queued: every API process buffers its own download/check hits and flushes them to the database itself.

Delivery guarantees

The database is the source of truth; Redis only carries delivery hints.

• Mail and store operations are written to email_outbox, play_operations, asc_operations in the request's database transaction. The relay turns every due row into a job whose id is <rowId>-<attempt>, so re-submitting is harmless.
• If the process dies between the database write and the enqueue, or Redis loses its data, the next relay tick re-submits the row. Nothing depends on the request thread reaching Redis.
• A worker must win a conditional UPDATE (a lease on next_attempt_at) before touching a row, so two workers never process the same operation concurrently. If a worker crashes mid-job, the lease expires (mail 2 min, store operations 15 min) and the row is retried. Each claim counts as an attempt, so a row that keeps crashing workers is failed after 5 attempts.
• Retries: a failed attempt is recorded on the row with exponential backoff (1 min doubling, capped at 1 h, 5 attempts). BullMQ additionally retries unexpected errors 3 times (5 s exponential).
• Delivery is at-least-once, not exactly-once. Mail carries a stable Message-ID (SMTP) / Idempotency-Key (Resend); Postmark has no such mechanism. Google Play pushes reconcile against the track before re-uploading; App Store Connect pushes look up the existing build first. A crash between a successful send and the status update can still send a mail twice.
• Maintenance jobs hold a Redis lock per job name (renewed while running), so a slow run makes later ticks skip rather than overlap, across all workers.
• Delta generation runs in a short-lived child process (releasedesk __delta-job), keeping CPU-heavy work off the event loop that serves HTTP and renews BullMQ locks. One pair is built at a time per process, with a 30-minute timeout.

Rate limits per worker process: mail 20 jobs/s, Google Play and App Store Connect 30 jobs/min each (concurrency 2). Completed jobs are kept for 1 h (max 1000), failed jobs for 7 days (max 5000).

Configuration

Variable	Default	Meaning
RD_REDIS_URL	–	redis:// or rediss:// URL, optionally with password and db. Never logged (only host and db are).
RD_REDIS_PREFIX	rd	BullMQ key prefix.
RD_JOBS_MODE	embedded	embedded: HTTP and workers in one process. api: HTTP only.
RD_JOBS_SHUTDOWN_TIMEOUT_MS	30000	How long SIGTERM waits for active jobs.

Redis requirements

• maxmemory-policy noeviction. Any eviction policy can drop queue keys; ReleaseDesk logs a warning at startup when it detects another policy.
• Persistence: appendonly yes with appendfsync everysec (the shipped Compose files do this). Without it, a Redis restart loses waiting jobs and scheduler state; schedulers are re-registered on the next start and the outbox relay re-submits pending mail and store operations, but delayed BullMQ-level retries are lost.
• Size: queues are bounded by the retention above, so a few tens of MB is plenty.

Deployment

Single process (default). Run Redis next to ReleaseDesk (see deploy/prod/docker-compose.yml, set REDIS_PASSWORD in .env) and keep RD_JOBS_MODE unset.

Dedicated workers. Set RD_JOBS_MODE=api on the HTTP replicas and run releasedesk worker (or bun apps/server/src/worker.ts in development) with the same environment, database and storage configuration. The worker does not run migrations, so start it after the API. It re-reads site settings (mail, storage, retention) every 30 s. With SQLite the worker must share the data directory with the API on the same host.

Shutdown. On SIGTERM/SIGINT the HTTP server stops, the local stats buffer is flushed, workers stop taking jobs, active jobs get RD_JOBS_SHUTDOWN_TIMEOUT_MS to finish, then the queue and Redis connections are closed. Jobs still running at the deadline become stalled and are re-run by another worker. Set the container stop_grace_period above the timeout. A second signal exits immediately.

Upgrading

Earlier versions ran jobs with in-process timers and needed no Redis. After upgrading, add Redis and RD_REDIS_URL; the server refuses to start without it. Pending outbox rows are picked up automatically.