A Dummies Guide to SQS with McDonalds

This is part of my Dummies Series on AWS .

By the end of this 2-part article you should have a good understanding on

1. Messaging Queues
2. De-coupled System Architecture
3. Type of Queues
4. Publisher-Subscriber Model
5. Dead-Letter Queues

What is SQS ?

SQS [ Simple Queue Service ] is a message queing service provided by AWS.

What kind of messages ?

Any message that can be produced by a system can be stored . Ofcourse you’ll have to follow guidelines on the structure and size of the message

Typically a json structure is used to store the message , its attributes and metadata . AWS caps the maximum size of a message that can be stored to a queue to be 256 KB

An example : Observe the json schema .

What is a Queue ?

Just to be sure that we also cover this , a queue is a system to ensure anybody / anything ( in our case the message ) getting into it is governed by the controls and rules of the queue.

Where do I use a Messaging Queue?

Messaging Queue is a powerful tool in establishing a de-coupled system architecture . A de-coupled system architecture is an efficient method where by systems that request for a task doesnt have to wait idle until the task is completed. Confusing right ? . Let’s drive with an example .

There are five critical aspects to an efficient messaging queue . They are

1. Publisher
2. Message
3. Subscriber
4. Type of the queue ( Standard / FIFO )
5. Dead-Letter-Queue

A real-life example to understand de-coupled architecture and Messaging Queue

Imagine you are at the nearest McDonalds .

1. You go up to the counter , order your burger
2. Pickup the bill that contains a unique number to help you collect your order later
3. Go back and sit or wait somewhere until the number is shown on the screen
4. When the unique number you have is flashed you go and get your burger and walk away.

Relating to the terms in a Messaging queue

1. You are the Publisher
2. The message being the order for your favorite burger
3. And the Billing Assistant is the Subscriber who receives your request , ensures that it is processed

So , a decoupled architecture avoids the publisher and subscriber from having to get on a staring contest until the processing is complete. The publisher can move on with their lives after getting an acknowledgement in form a of unique ID ( for future reference ) while the subscriber can complete the task in hand and notify as per the rules defined.

Here’s where things get interesting . What we have seen here is an example of a standard queue where it is not guaranteed that you’ll be served before the next person who placed their order.It could be possible that another person who ordered after you got their’s before you just because their’s got processed quickly . It happens right ? Why ? Because their order was a simple one and your’s were a bit complex . The other important thing you need to realize is the number of kitchen staff who are working on the requests .

Let’s also understand where a FIFO queue is necessary and valid ?

Imagine you are the same McDonalds and there’s only one assistant maning the outlet’s kitchen ( unlikely , but for the sake of learning ) and has to process orders.

Will it be right here for the person who placed their order after you to get their orders before you ? Even if it is a simple one ?

Similarly there are scenarios where technically a First-In-First-Out scenario needs to be ensured . But still the publisher ( you ) and subscriber ( billing assistant ) can move on with their lives after the order is placed with the understanding that the kitchen assistant processes the orders in the order of their received time . And therefore no staring contest .

What is a Dead-Letter Queue ?

In order to understand a Dead-Letter queue the McDonalds example is not sufficient . However in literal terms a Post-Office example will do .

Same players here as well.

1. Publisher ( You)
2. Message ( Wanted to send a Christmas card to your friend far away )
3. Subscriber ( The Postal Clerk , Delivery man )

Everything was fine except you got the address wrong. Now what will happen to the mail ?. As the “Delivery attempt” which is the intended process failed it will reach a place called “Dead-Letter”.

In a technical scenario , whenever a message could not be processed due to various reasons it can be re-directed to a Dead-Letter Queue.

Why Dead-Letter Queues ?

They are very useful in gracefully trapping any un-handled exceptions in your messaging workflow . If you get any message on a DeadLetter Queue you can investigate them with a separate process and do the necessary changes to your system. The goal should be to have no messages on a Dead-Letter Queue.

The second part containing a tutorial is here.

https://medium.com/cloudfordummies/a-dummies-guide-tutorial-to-aws-sqs-b3431ad7be1d