AWS SQS (Simple Queue Service)
SQS provides a reliable and durable method to send and receive messages.
Once a message is sent / queued, it guaranteed to be delivered at least once to those polling the queue.
Once your process has read a message it should be deleted to prevent processing it a 2nd time. To delete a message from the queue a receipt handle is required – in addition to authority to delete it – which is usually the case if you can read it.
While there are lots of specific use cases for SQS, the most important general case in my mind, is the fact that by using SQS in multi-step applications, you decouple the application as a whole. Removing points of failure, allowing restarts and distributed processing. And in general making the application more robust – less prone to failure.
Message Retention Max and Defaults
Message retention defaults to 4 days and a maximum of 14 days – although it is defined in seconds.
There is also a delay queue by setting a delay value greater than zero / 0. The default is zero, there is no delay.
AWS SQS (Simple Queue Service) Message Visibility Timeout
To help prevent messages from being processed multiple times by distributed applications SQS implements Visibility Timeout.
“Immediately after a message is received, it remains in the queue. To prevent other consumers from processing the message again, Amazon SQS sets a visibility timeout, a period of time during which Amazon SQS prevents other consumers from receiving and processing the message. The default visibility timeout for a message is 30 seconds. The maximum is 12 hours.”
Message Queue Polling
On message queue polling – long polling is strongly preferred as short polling (spinning on the queue) can consume large amounts of CPU / resource. The max and default long poll is 20 seconds.
SQS Queue Types
There are multiple types of queues:
- Standard Queues
- FIFO Queues
- Dead Letter Queues
Two of the best features of SQS Service is it’s extremely high availability and scalability. AWS achieves this especially for standard queues by distributing the service across many many nodes. While taking the distributed approach to the system certainly massively improves high availability and scalability, for pure scalabilities sake standard queues do not guarantee first in first out reading of messages from Standard Queues.
With a standard queue you could add a timestamp to the message data itself and order on that if you choose. Or you can choose the less scalable FIFO queue type to guarantee ordered reads.
Other SQS Facts
- Each Queue Can be individually configured
- Messages can contain up to 256 KB of text data, including XML, JSON, and unformatted text.
- Standard and FIFO queues support server-side encryption (new feature 2017).