Dead- letter message processing
  • 25 Sep 2020
  • 5 Minutes to read
  • Dark
    Light
  • PDF

Dead- letter message processing

  • Dark
    Light
  • PDF

Article Summary

The features explained in this document meets both Service Bus Queues and Topics

Dead- letter Queue

Azure Service Bus queues and topic subscriptions provide a secondary sub-queue, called a dead-letter queue. The dead-letter queue does not need to be explicitly created and cannot be deleted or otherwise managed independent of the main resource. The purpose of the dead-letter queue is to hold messages that cannot be delivered to any receiver, or messages that could not be processed.

There are several automated tasks in Service Bus that cause messages to get pushed to the dead-letter Queue. In real time, business applications can explicitly move messages to the dead-letter Queue. As the message gets moved by the broker, two properties are added to the message: DeadLetterReason and DeadLetterErrorDescription.

Applications can define their own codes for the DeadLetterReason property when messages are explicitly dead- lettered. There are system set dead- letter reasons too.

The Challenge

Business demands reprocessing these messages in the dead-letter Queue. Consider a scenario where the Logic App listening to the Service Bus Queue was disabled for a while which caused the messages in the queue to move to Dead- Letter Queue with dead- letter reason 'TTLExpiredException'. Requirement here would be to resubmit the messages to the same or another Service Bus Queue or Topic.

There is no straight forward solution in the Azure portal to achieve this. Here is where dead letter message processing capabilities of Serverless360 can help.

Retrieve messages

Serverless360 facilitates retrieving messages in Peek Lock or Defer mode from the dead-letter Queue. Retrieve messages from a sequence number without affecting the message reception by downstream application using 'Peek Lock' mode. Defer the messages from Service Bus Queue or Topic subscription to make it unavailable for the downstream application to receive. In both the modes get visibility on the following parameters of the message.

  • Message Id
  • Sequence Number
  • Size
  • Label
  • Error Reason
  • Enqueued Time
  • Expires At
  • Message Content
  • Message Custom Properties
  • Error Details
To have permanent local copy of the Service Bus messages, download selected messages into a compressed file with each of the messages represented as an individual text file

DLQmsgs_download(1)

Message Processing on Service Bus dead-letter Queue

In real time the requirement would be lot more than just retrieving the messages or having a copy of the same.

Below is a tabular column which explains different modes of message retrieval possible on Service Bus dead-letter Queue.

Message Type / ModePeek LockDeferred
Dead Letter MessagesIn this mode of retrieval the messages are peeked. It does not remove the message from dead letter queue.
  • Resubmit – A copy of a message will be submitted to the target Queue. Message in source dead-letter Queue will not be deleted.
  • Repair and Resubmit - A copy of a message with updated content and properties will be submitted to the target Queue. Message in source dead-letter Queue will not be deleted.
Retrieving messages in this mode will change the state of the message to 'deferred' which can be processed later.
  • Resubmit and Delete – A copy of a message will be submitted to the target Queue. Message in source dead-letter Queue will be deleted.
  • Repair and Resubmit - A copy of a message with updated content and properties will be submitted to the target Queue. Message in source dead-letter Queue will not be deleted.
  • Resubmit and Delete after Repair - A copy of a message with updated content and properties will be submitted to the target Queue. Message in source dead-letter Queue will be deleted.
  • Delete – Message will be deleted from the Queue
In Serverless360, when messages in dead-letter Queue are retrieved in ‘Defer’ mode and not processed now. Those messages can be found in the ‘Deferred Dead- letter’ tab for processing any time later.

Below is an illustration to process the dead letter messages in the Service Bus Queue associated with Serverless360.
dead-letter-message-processing-queues.gif

Purge messages

Purge all dead lettered messages to completely clear the contents of the dead-letter Queue. Below is an illustration on the inline Purge Messages automated task to purges either all or specific count of from a Service Bus dead-letter Queue.

Purging(1)

Regenerate/Modify or Edit a message ID

When you try to resubmit a message to a 'DuplicateDetection' enabled Queue, this capability will help you to prevent the message from being lost. When you choose automated regeneration, the system will assign a new GUID as Message Ids. Alternately, you can edit to set custom message ids. for eg: invoice/201#/####.

edit message ID

Backup messages

Serverelss360 offers out of box dead letter message processing capabilities on Service Bus Queues and Topics Subscriptions. There could be business scenarios those demand to retain a copy of a Service bus messages being processed. Hence the back up to Storage blob capability was introduced.

In Serverless360, it is possible to back up the Service Bus messages being processed to a Storage blob associated in the same Composite Application, in which the Service Bus resource exists. It is possible to have multiple Storage Blob Containers associated in a Composite Application. The decision on back up target can be made in run time while performing message processing.

Below is an illustration on message back up to associated Storage blob while dead letter processing,
Backingup messages.gif

On successful resubmission the success notification will contain the blob URI where the backed-up message can be accessed from. Even in case of failure in resubmission the message will be backed-up. This behaviour holds good for of all the dead letter management scenarios, delete, repair and resubmit and resubmit.

Success.png

When back up is enabled on a dead letter action, the back-up URI will be captured in the Governance & Audit record corresponding to this action. This detail can also be exported to Excel or PDF format using the export capability in Governance & Audit.

These Storage blobs to which the messages had been backed-up can also be monitored from Serverless360 itself. Refer the article on ‘Monitoring Storage Accounts’ for more details
We'd love to hear your thoughts
Please visit our feedback system to suggest new Features or Enhancements. You can also take a look at our Roadmap

Was this article helpful?