Dead- letter message processing

Dead- letter Queue

Azure Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter queue. The dead-letter queue does not need to be explicitly created and cannot be deleted or otherwise managed independent of the main resource. The purpose of the dead-letter queue is to hold messages that cannot be delivered to any receiver, or messages that could not be processed.

There are several automated tasks in Service Bus that cause messages to get pushed to the dead-letter Queue. In real time, business applications can explicitly move messages to the dead-letter Queue. As the message gets moved by the broker, two properties are added to the message: DeadLetterReason and DeadLetterErrorDescription.

Applications can define their own codes for the DeadLetterReason property when messages are explicitly dead- lettered. There are system set dead- letter reasons too.

The Challenge

Business demands reprocessing these messages in the dead-letter Queue. Consider a scenario where the Logic App listening to the Service Bus Queue was disabled for a while which caused the messages in the queue to move to Dead- Letter Queue with dead- letter reason 'TTLExpiredException'. Requirement here would be to resubmit the messages to the same or another Service Bus Queue or Topic.

Retrieve messages

Serverless360 facilitates retrieving messages in Peek Lock or Defer mode from the dead-letter Queue. Retrieve messages from a sequence number without affecting the message reception by downstream application using 'Peek Lock' mode. Defer the messages from Service Bus Queue or Topic subscription to make it unavailable for the downstream application to receive. In both the modes get visibility on the following parameters of the message.

• Message Id
• Sequence Number
• Size
• Label
• Error Reason
• Enqueued Time
• Expires At
• Message Content
• Message Custom Properties
• Error Details

Message Processing on Service Bus dead-letter Queue

In real time the requirement would be lot more than just retrieving the messages or having a copy of the same.

Below is a tabular column which explains different modes of message retrieval possible on Service Bus dead-letter Queue.

Below is an illustration to process the dead letter messages in the Service Bus Queue associated with Serverless360.

Purge messages

Purge all dead lettered messages to completely clear the contents of the dead-letter Queue. Below is an illustration on the inline Purge Messages automated task to purges either all or specific count of from a Service Bus dead-letter Queue.

Regenerate/Modify or Edit a message ID

When you try to resubmit a message to a 'DuplicateDetection' enabled Queue, this capability will help you to prevent the message from being lost. When you choose automated regeneration, the system will assign a new GUID as Message Ids. Alternately, you can edit to set custom message ids. for eg: invoice/201#/####.

Backup messages

Serverelss360 offers out of box dead letter message processing capabilities on Service Bus Queues and Topics Subscriptions. There could be business scenarios those demand to retain a copy of a Service bus messages being processed. Hence the back up to Storage blob capability was introduced.

In Serverless360, it is possible to back up the Service Bus messages being processed to a Storage blob associated in the same Composite Application, in which the Service Bus resource exists. It is possible to have multiple Storage Blob Containers associated in a Composite Application. The decision on back up target can be made in run time while performing message processing.

Below is an illustration on message back up to associated Storage blob while dead letter processing,

On successful resubmission the success notification will contain the blob URI where the backed-up message can be accessed from. Even in case of failure in resubmission the message will be backed-up. This behaviour holds good for of all the dead letter management scenarios, delete, repair and resubmit and resubmit.

When back up is enabled on a dead letter action, the back-up URI will be captured in the Governance & Audit record corresponding to this action. This detail can also be exported to Excel or PDF format using the export capability in Governance & Audit.