What are queues and background jobs?
A queue lets a request hand off slow work — sending email, processing an upload, generating a report — to a separate worker, so the user gets a fast response. The job runs in the background, with retries if it fails. It is the standard way to keep request handlers quick.
Why it matters
Doing slow work inside a request makes users wait and ties up server capacity. A payment confirmation should not block on a slow email send. Queues decouple the slow part, improve responsiveness, and add resilience through retries. This pattern shows up in nearly every real backend.
What to learn
- The producer, queue, worker pattern
- Why background work improves response time
- At-least-once delivery and designing idempotent jobs
- Retries, backoff, and dead-letter queues
- Choosing a backend: Redis-based, managed, or broker
- Scheduling recurring jobs
- Monitoring queue depth and worker health
Common pitfall
Writing jobs that are not idempotent under at-least-once delivery. Most queues can deliver the same message twice, so a job that charges a card or sends an email must detect "already done" and skip — usually with an idempotency key. Otherwise a routine retry double-charges a customer.
Resources
Primary (free):
- BullMQ — Documentation · docs
- AWS — What is a message queue? · docs
- Microsoft — Queue-based load leveling · docs
Practice
Move a slow operation — say sending a welcome email — out of the request and into a queued background job. Make the job idempotent so a retry does not send twice, and add a retry with backoff. Trigger a deliberate failure and confirm it retries then lands in a dead-letter queue. Done when duplicate delivery is safe.
Outcomes
- Offload slow work to a background worker via a queue.
- Make a job idempotent so retries are safe.
- Configure retries, backoff, and a dead-letter queue.
- Monitor queue depth to catch a backed-up worker.