Inkey Solution Logo
banner

Blogs

Handling Dead Letter Queue(DLQ) using Azure Functions

, January 31, 2019 31548 Views

Microsoft Azure Service Bus is a secure platform for transferring the messages across various platforms and applications. This is a reliable service for asynchronous data transfer using messages. Queues are one among the data structures used while transmitting the data across the cloud. Messages are stored in the queue until processed at the receiver’s end.

In this blog, I am discussing the concept of reading the messages from the “Dead Letter Queues”. Now, what is a “Dead Letter Queues”?? It is a queue where all the undelivered messages in the Active Queue, are moved to. What could be the reason for such undelivered messages? – This could be due to various reasons like exceeding MaxDeliveryCount, exceeding TimeToLive, errors while processing subscription rules, Dead-lettering in ForwardTo or SendVia scenarios like: Message passes through more than 4 queues or topics that are chained together, The destination queue or topic is disabled or deleted etc.

For more details please refer:
https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dead-letter-queues

These Dead Letter Queue messages can be read and managed either using the Azure Functions or the Logic Apps. Basically, you connect to your Dead Letter Queue in exactly the same way as your normal queue, but you need to concatenate “/$DeadLetterQueue” to the queue name. For example: “myQueue/$DeadLetterQueue”

Below are the ways to read the messages from the DLQ using the Azure Functions:

1) Using Code

In the below example, we are reading the messages from the DLQ at a regular time interval and are resending the messages to the Active Queue.

The comments in the below code will help you to understand the entire process aptly.

[FunctionName("Function2")]

public static async Task Run([TimerTrigger("0 */2 * * * *")]TimerInfo myTimer, TraceWriter log)
{
log.Info($"C# Timer trigger function executed at: {DateTime.Now}");
string queueName = "Name of the Active Queue";
try
{
//Create a MessageFactory object to access the service bus where the messages are transferred
MessagingFactory messageFactory = MessagingFactory.CreateFromConnectionString("Endpoint=Provide the end point url of the Service Bus; SharedAccessKeyName=NAME; SharedAccessKey=KEY");

/*Create a receiver object that will receive the messages from the DLQ with PeekLock as the receiver mode. So that, the message will be locked till we resend the message to the Active Queue and execute the ‘Complete’ action of the BrokeredMessage object.*/
MessageReceiver deadletterReceiver = await         messageFactory.CreateMessageReceiverAsync(QueueClient.FormatDeadLetterPath(queueName), ReceiveMode.PeekLock);

//Create a sender object that will send the messages received from the DLQ to the Active queue
MessageSender sender = await messageFactory.CreateMessageSenderAsync(queueName);
log.Info($"message");

//Read the messages received by the receiver
BrokeredMessage deadLetter = await deadletterReceiver.ReceiveAsync(TimeSpan.Zero);
if (deadLetter != null)
{
log.Info($"reason: " + deadLetter.Properties["DeadLetterReason"]);

/*Create a new message with the same content as read above from the body of the DLQ message*/
BrokeredMessage newMessage = new BrokeredMessage(deadLetter.GetBody<Stream>())
{
ContentType = deadLetter.ContentType,
CorrelationId = deadLetter.CorrelationId
};

//Send the message to the Active Queue
await sender.SendAsync(newMessage);
await deadLetter.CompleteAsync(); //Unlock the message and remove it from the DLQ
}
}
catch (Exception ex)
{
log.Info($"Exception: {ex}");
}
}

2) Using Trigger:

The other way of reading the Dead Letter Queue messages is by creating the Azure Function Trigger in the Azure portal. In the Function, provide the name of the DLQ as “QueueName/$DeadLetterQueue” as shown in the below image.

Note: If you want to access the undelivered message from the Topic then, the syntax of reading the Dead Letter Queue will be “TopicName/Subscriptions/SubscriptionName/$DeadLetterQueue”.

 


Discover the power of our services:

Microsoft Fabric, Power BI, Microsoft Business Intelligence, SQL Server, and Business Central. By the power of these services, from advanced analytics to seamless business integration, we’ve got the expertise you need to optimize operations and drive growth. Harness the potential of your data infrastructure with our comprehensive suite of solutions.


 

Now, define what should be done with the DLQ messages. Here, as shown in the below screenshot, we are sending the DLQ messages of “myqueue” to the Topic named of “queue” using the Azure Service Bus.

Sample code:

In this manner, we can handle the DLQ messages as required very easily using the Azure Functions.

We can also manage the Dead Letter Queue using the Logic App.

For more details, please refer below link:
https://www.inkeysolutions.com/blogs/handling-dead-letter-queuedlq-using-logic-app/

I hope this helps you.


 

Insert data into Many-to-Many relationship in Dynamics CRM very easily & quickly, using the Drag and drop listbox.
http://www.inkeysolutions.com/what-we-do/dynamicscrmaddons/drag-and-drop-listbox

Create a personal / system view in Dynamics CRM with all the fields on the form/s which you select for a particular entity using the View Creator.
http://www.inkeysolutions.com/what-we-do/dynamicscrmaddons/view-creator

mm

Inkey

INKEY is your solution partner.
Our focus is to deliver you in-time intelligent innovative solutions ("key") for the problems in hand. Maintaining a quality standard right from the inception of a project is our top most priority.

Our team of talented professionals will execute your projects with dedication and excellence. We take ownership and accountability for the effort that goes into meeting our client’s needs.

Years of experience and proven success of delivering innovative custom solutions.

More posts by

7 responses to “Handling Dead Letter Queue(DLQ) using Azure Functions”

  1. Arun Verma says:

    Hi Team

    We had a similar need and may provide some inputs to process dead letter messages into Active Q using Azure Functions/Triggers. Could you please provide some more details on the overall process.

  2. ashish says:

    i have a question i have azure function which triggers when i have new message into my service bus topic now if message is not processed after 10 attempts it goes into DLQ now using same function can i process it? by checking something like deadletter q is empty or not? and one more after process message how to complete it? i’m using simple strnig message right now and convert json and process how to complete it?

    • Admin says:

      Hello Ashish,

      There are a few important things to understand here –
      1. Why is the process failing for 10 attempts? If it is because of some logical error or data error, it will fail again even if you process it from DLQ. Hope, you get my point here. Better, handle it in some other way.
      2. If the process is failing due to some external factors, like – some 3rd party API call is failing or Server is not responding or because of any temporary glitch, then instead of re-processing the message from DLQ, just push it back to the Active Queue. And then it will be picked from there in the normal process.
      3. It is not a good practice to process DLQ messages in the same function which process the Active Queue. The reason is the triggering point. Say, if you do not get any new message in the active queue, the logic to process the DLQ messages will NEVER execute. Better, write a separate function to process the DLQ.
      4. Once the function executes successfully, the message will be automatically be marked as completed. You need not explicitly do it.

      Hope, the response is useful. If not, please feel free to write back and we will try our best to help you.

      Thanks!

  3. Sean says:

    Hi,

    I’m having a problem creating an instance of the MessagingFactory in my Azure function. The reason is that there is a dependency on .Net Framework 4.6 where it uses the configuration manager in the CreateFromConnectionString method.

    Was your sample above against a version 1 or a version 2 function in Azure? Version 2 is (as far as I know) .Net Core 2.2 based so this will fail.

    Would you be able to suggest a solution to this issue?

  4. Ravi Kumar says:

    Hello,

    I have a question related to dead letter queue. Can users with admin privileges download the messages from dead letter queue and view it?

    • Admin says:

      Hello Ravi,

      any user with permission to read the DLQ can read the message using any of the methods we have showcased. The user being an admin, can too access DLQ.

      Now, if you want to protect your message from unauthorized read, please make sure that the message is encrypted with the public keys kept secure.

      I hope this helps!

  5. DSG says:

    Hey, thanks for this.
    I have a question about using trigger function for DLQ.

    So, what happens when the trigger fails for a message. (example: when you are trying to resubmit/add to another queue). Will the message go back to DLQ. And if I am not wrong, the delivery count will always be 1, right?

Leave a Reply

Your email address will not be published. Required fields are marked *

The maximum upload file size: 2 MB. You can upload: image, audio, video, document, spreadsheet, interactive, text, archive, code, other. Drop file here

Would you like to digitize your business and put it on the cloud?
Do you need clear, concise reports for your organization?