Skip to Content

Handle failed messages using a Dead Letter Queue in Jitterbit Studio

Introduction

The Jitterbit MQ connector lets you process messages from a queue using a Get activity. When processing fails, a common approach is to use the NACK activity with Requeue Messages After NACK to return the message to the queue for a later attempt. With Quorum queues, however, this approach has a hard limit: after 20 failed acknowledgments, the message is permanently removed from the queue without an error. For integrations that require controlled retry behavior and guaranteed delivery, use the Dead Letter Queue (DLQ) pattern instead.

This guide walks through building a DLQ pattern using multiple queues and a set of Studio operations. Failed messages are rejected from the main queue and routed to a series of retry queues with increasing delays. After a configurable number of attempts, messages are sent to a final DLQ for manual inspection.

flowchart TD A([Message in orders.main]) --> B[Main processor] B -->|Success| C([ACK: message removed]) B -->|Failure| D{retryCount} D -->|1| E([orders.retry.1m]) D -->|2| F([orders.retry.5m]) D -->|3 or 4| G([orders.retry.30m]) D -->|greater than 4| H([orders.dlq]) E -->|Forwarder runs every 1 min| A F -->|Forwarder runs every 5 min| A G -->|Forwarder runs every 30 min| A

For simpler retry needs that do not require multi-tier delay routing, see Retry a failed operation. For background on configuring on-success and on-fail behavior, see Configure error handling in operations.

This guide assumes the following:

  • You have a working Jitterbit MQ connection in Studio. For setup instructions, see the Jitterbit MQ connector documentation.
  • You are familiar with creating and running Studio operations.
  • You are using Quorum queues, which are required for durability but impose the 20-attempt delivery limit that this pattern addresses.

Part 1: Create the queues

Step 1: Create the queues in Management Console

  1. Open the Management Console Message Queues page.

  2. Create the following five queues. Set the Type to Quorum for each.

    Queue name Purpose Message TTL
    orders.main Primary processing queue —
    orders.retry.1m First retry (1-minute delay) 3,600,000 ms
    orders.retry.5m Second retry (5-minute delay) 3,600,000 ms
    orders.retry.30m Third retry (30-minute delay) 3,600,000 ms
    orders.dlq Final queue for manual inspection 864,000,000 ms

    Adapt the queue names and number of retry tiers to fit your integration.

Note

The TTL values on the retry queues are a safeguard that prevents messages from accumulating indefinitely if a forwarder operation fails. The delay between retries is controlled by how often the forwarder operations run, not by the TTL.

Part 2: Configure the main processor operation

Step 2: Add a retryCount field to your message payload

The DLQ pattern uses a retryCount field in the message payload to track how many times processing has been attempted. Add this field to your message payload with an initial value of 0 before the operation is first triggered.

Example payload

{
  "orderId": "ORD-12345",
  "retryCount": 0
}

Use a consistent, top-level field name across all operations in the pattern. The retryCount field is the only mechanism for tracking attempt history across runs.

Step 3: Build the on-fail routing operation

When the main processor fails, the on-fail operation must read the current retryCount, route the message to the appropriate retry queue, and then reject the original message from the main queue.

  1. Read retryCount from the messageBody field in the Get activity response schema.

  2. Increment the value by 1 using a script or transformation.

  3. Route to the appropriate queue based on the updated count:

    retryCount value Destination queue
    1 orders.retry.1m
    2 orders.retry.5m
    3 or 4 orders.retry.30m
    Greater than 4 orders.dlq

  4. Write the updated retryCount back into the messageBody field of the Send activity request schema when routing to the target queue.

  5. After the Send activity completes, use the NACK activity with the Reject Messages After NACK option to remove the message from orders.main.

Important

The Send activity to the retry queue must complete before the NACK activity runs. This order ensures the message is not lost if the Send activity fails. Do not use Requeue Messages After NACK, which bypasses the DLQ routing logic and counts toward the 20-attempt delivery limit.

Step 4: Configure the main processor operation

  1. In Studio, open or create the operation that processes messages from orders.main.

  2. Add a Get activity configured to read from orders.main.

  3. Add the steps that process each message (for example, querying an ERP system for the expected order status).

  4. Open the operation's actions settings and configure the following:

    • On Success: Run an operation that uses the Acknowledge activity to acknowledge the message from orders.main.
    • On Fail: Run the on-fail routing operation built in Step 3.

Part 3: Configure the retry forwarder operations

Create one forwarder operation for each retry queue. Each forwarder reads messages from its retry queue, sends them back to orders.main for reprocessing, and runs on a schedule that matches the intended delay for that tier.

Step 5: Build each forwarder operation

For each retry tier, create an operation with the following steps:

  1. Add a Get activity to retrieve a message from the retry queue.

  2. Add a Send activity to send the message to orders.main, preserving the retryCount value in the payload.

  3. Add an Acknowledge activity to acknowledge the message from the retry queue.

Important

The Send activity to orders.main must complete before the Acknowledge activity runs. If the Send activity fails, the message remains in the retry queue and will be picked up on the next forwarder run. Acknowledging before the Send activity completes would silently remove the message from the retry queue with no guarantee it reached orders.main.

Step 6: Schedule each forwarder operation

Attach an operation schedule to each forwarder operation with an interval matching its retry tier:

Operation Source queue Schedule interval
Retry forwarder 1m orders.retry.1m Every 1 minute
Retry forwarder 5m orders.retry.5m Every 5 minutes
Retry forwarder 30m orders.retry.30m Every 30 minutes

For instructions on setting up a schedule, see Schedule an operation to run automatically.

DLQ inspector operation (optional)

Create an operation that reads from orders.dlq to alert on or log messages that have exhausted all retry attempts. This operation can send an email notification, write to a log, or trigger a manual review workflow. For instructions on sending email from a Studio operation, see Send an email notification from a Studio operation.

Note

Messages in orders.dlq require manual intervention. They remain in the queue until explicitly acknowledged or their TTL expires.

Verify the integration

  1. Deploy the project.

  2. Send a test message to orders.main with retryCount: 0 in the payload.

  3. Trigger a processing failure in the main processor operation (for example, by returning an error from the processing script).

  4. Open the operation logs in Studio and confirm that the on-fail operation ran and that the message was sent to orders.retry.1m.

  5. Wait for the 1-minute forwarder to run. Confirm in the operation logs that the forwarder retrieved the message from orders.retry.1m and sent it back to orders.main.

  6. Continue triggering failures to advance the message through the remaining retry tiers. The orders.retry.5m queue handles one pass (retryCount 2), and orders.retry.30m handles two passes (retryCount 3 and 4). After the fifth failed attempt, the message routes to orders.dlq.

  7. Open the Management Console Message Queues page and confirm that a message has arrived in orders.dlq.