Amazon SQS FIFO

Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. Starting from the end of 2016 you can create a SQS FIFO.
FIFO queues are designed to ensure that the order in which messages are sent and received is strictly preserved and that each message is processed exactly once.
Here you can read more about the release of SQS FIFO:

These pictures found online clearly explain the differences between normal SQS and FIFO SQS:

sqs_standard
sqs_fifo

In this post we are going to see how to create a SQS FIFO queue, how to send some messages and how to consume them using Python. I am using Python 3.4 and the official AWS SDK: Boto3.

Connect to the sqs resource and create a new FIFO queue:

Please be aware that the name of a FIFO queue must end with the suffix .fifo and that FIFO queues are currently available only in the Oregon (us-west-2) and Ohio (us-east-2) regions. This feature will be available in more regions in the coming months.

The ‘ContentBasedDeduplication’ : ‘true’ attribute can be used when the messages are unique (usually a single producer and consumer). Here you can read more about content duplication in the FIFO queue: Recommendations for FIFO Queues

To send a message, use the send_message method:

In FIFO queues, messages are ordered based on message group ID. FIFO queue logic applies only per message group ID. Each message group ID represents a distinct ordered message group within an Amazon SQS queue.

To consume the messages in queue use the receive_message method:

Use the following lines to test that the messages are processed Firs-in First-Out:

You can run the code against a standard SQS queue and you will see that the messages are not consumed in order.
If your applications require messages to be processed in a strict sequence and exactly once you can use SQS FIFO.
Consider to use a SQS (both standard or FIFO) when:

when_use_sqs

Here you can find the code shown in this post: mz1991/AWS SQS FIFO

Here here some useful resources:

Leave a Reply

Your email address will not be published. Required fields are marked *