AWS offers many great messaging services. Two of their most famous services are Amazon Simple Notification Service (SNS) and Amazon Simple Queue Service (SQS). While both can be used in a pretty similar way they are completely different services.

This blog post will explain to you the similarities, differences, and how to choose which service.

Finally, I will show you some example use cases and a common event-driven pattern called the fanout pattern that combines SNS and SQS.

Let's go 🚀

Amazon Simple Notification Service (SNS)

Amazon's SNS is a fully managed publish and subscribe service. A publisher sends a message to a topic and many consumers/subscribers are subscribed to this topic. The relationship is many to many. You can have many publishers and many subscribers to a topic.

SNS differentiates in sending their sending methods. It is either Application to Application (A2A) or Application to Person (A2P).

Application to Application (A2A) destinations are:

• AWS Lambda
• Amazon SQS
• Amazon Kinesis Data Firehose
• AWS Event Fork Pipelines
• HTTP Endpoints

Application to Person (A2P) destinations are:

• SMS
• Email
• In-App notification
• AWS Chatbot
• PagerDuty

SNS is super performant. Messages will be published within milliseconds.

A common pattern is a fan-out pattern that allows to fan out one event to various subscribers within AWS. I will go more into detail about that pattern later on in the use case section.

SNS allows standard topics or FIFO topics. FIFO topics have a message ordering and standard topics don't. For FIFO topics there are much harder limits!

Amazon Simple Queue Service (SQS)

Amazon SQS is a fully-managed distributed queueing service. SQS is poll-based and not push-based. Even, if it often seems like a push-based system, it is not. Amazon SQS is often used to decouple systems from each other and to enable asynchronous workloads.

The main pattern of SQS is to have producers send messages to a queue. The message retains in the queue for a defined time (by default 4 days, maximum 14 days). See our article on retention period. Consumers can pick up the messages on their own schedule by checking the queue if any new messages are available.

If a consumer works on a message the message will be deleted if it is successful. Otherwise, it can also be picked up by other consumers.

SQS offers many capabilities for retrying messages with its redrive policy. You can define a number of retries and a dead letter queue in case messages are failing. Dead letter queues (DLQ) are used to handle messages with errors. If messages can't be worked on they will be sent to a DLQ to inform the application developer about the issues and optionally save the message to replay it in the original queue.

SQS has a many-to-one relationship. You can send messages to a queue from many different producers but only one consumer can be defined. A consumer is another application, most often some compute instances such as Lambda, EC2, or Fargate.

There are two different types of queues: Standard queues and First-in, first-out (FIFO) queues. The latter one will keep the messages in order.

Fun fact: SQS is AWS's oldest service.

Differences

This article focuses on the differences between both services.

We understand that both services handle messages in some way. Both services enable better decoupling. The backend API and the execution of background logic are loosely coupled and not connected anymore.

But there are notable differences. Let's have a look at all of them.

Push and poll-based

The main difference lies in the foundation of the services. SQS is poll-based and SNS is push-based service.

That means SNS is simply forwarding all messages to your subscribed consumers, whereas SQS really saves the messages in a queue and waits till they get picked up. This is a notable difference in various aspects. The latency in SQS architectures for example is a bit higher since the polling still needs to be accounted for. The persistence and reliability on the other hand are with SQS much better since the message is properly saved for a short period.

Many-to-many vs. many-to-one (Number of Consumers)

A second difference is the type of relationship. Both services can receive messages from different producers. That means both services have a many-to-x relationship.

The main difference is that SNS can have a lot of subscribers whereas SQS can only have one consumer.

The current limit of SNS subscribers is 12,500,000 subscribers per topic. That is a lot! This means you can have many many consumers who will work on your message.

SQS on the other hand can only have one consumer. This consumer is working on the message and deletes the message afterward.

Types of Consumers

SNS sends their messages either to an application or directly to a person, or both. That means it supports a variety of different consumer types.

SNS can send to many different destinations. These are:

Application to Application (A2A) destinations are:

• AWS Lambda
• Amazon SQS
• Amazon Kinesis Data Firehose
• AWS Event Fork Pipelines
• HTTP Endpoints

Application to Person (A2P) destinations are:

• SMS
• Email
• In-App notification
• AWS Chatbot
• PagerDuty

SQS messages on the other hand will typically be picked up using the SQS API. So every client that supports the AWS SDK can use it. Typically, messages in queues will be picked up from AWS Lambda because there is native integration with SQS and Lambda. But it is also possible to simply pick up and remove a message with the SQS API. You can also do it from your local PC 😉

Persistence

Messages in SQS will be saved for some duration. This is called the retention period. The retention period can be between 1 minute and 14 days and its default value are 4 days. If the message wasn't picked up within that timeframe the message will be removed automatically. Here is a more detailed article.

In SNS however, no persistence exists. There is no guarantee that the message will be delivered. If a consumer is not available the message won't be delivered.

This can make quite a difference in the reliability. If a consumer for example is not available in SNS the message will simply not be delivered. Or if a consumer doesn't end successfully the message is simply gone. SQS adds a lot of reliability to that. The fanout pattern can be used to combine both, but later more on that topic.

Reliability - Retries

SQS has the ability to add a Redrive Policy. This policy defines how many times a failed message should be retried before it will be moved to a Dead Letter Queue (DLQ). The DLQ handles failed messages. For example, you could save failed messages in a bucket and inform the developer about them.

SNS doesn't offer retries when the client fails. In case a consumer is not available or the consumer fails in working on the message (e.g. push notification won't come through) the message can't be repeated. This is due to the asynchronous nature of SNS.

Batching

SQS allows you to batch multiple messages together into one. You can define the parameter batch_size. The batch size can be a maximum of 10,000 records for standard queues and a maximum of 10 for FIFO queues.

SNS only works on one message at a time, so no batching is possible.

When to use what

When should I use SNS and when should I use SQS now?

Some general advice:

Use SNS if:

• You have many subscribers
• You need to send to consumers of the type SMS, Email, or app notifications
• You want to use a fanout pattern to send a message to many subscribers at the same time (later more)

Use SQS if:

• you only need one subscriber
• persistence and error handling are really important (each message needs to be delivered)
• you need to batch your requests
• you want to simply decouple your application and enable async background processing

Use Cases

Let's have some example use cases.

SNS

CloudWatch Alarms:

An alarm will be triggered and you want to send messages to 10 different email addresses and SMS to some mobile phones. Persistence, batching, and retries are not important.

In-App Notification

A new user followed you on your new social media application. You want to inform the user about that.

SQS

Background Processing

Your application is doing all tasks in a synchronous way and your user needs to wait till your API returns. By adding an SQS queue you can run background tasks and decouple the whole application.

Batch Processing

You need to work on a large number of messages at the same time. SQS can handle that by handling all messages in batch.

One example:

You host a startup show and you get a huge number of votes at the same time. You need to process these votes. It is not feasible that your API handles all of that and you don't want your users to wait the time it takes to process them.

You can use SQS for that. When a user votes your API sends a message to SQS. Your API returns 200 OK to the end-user and the end-user gets a super-fast response.

The actual business logic is decoupled. SQS and Lambda will take care of the rest. In such scenarios, many messages will be batched together and many lambda functions will be spawned. This is what scalability looks like.

SNS and SQS in Combination - Fanout pattern

In the article so far we've looked at SNS and SQS as two distinguished services. There is a common pattern out there called which combines both services together.

It is called the fanout pattern.

The fanout pattern describes a scenario where messages published to SNS will be sent to several endpoints at the same time. With a pattern like that it is possible that one message triggers several executions. This allows for asynchronous processing.

For example:

A user uploads a picture to your application. You want to have several background processes kicking off when this happens. The following things should happen:

1. Start image recognition and create a headline
2. Process the image and create a thumbnail
3. Notify the users

All of these processes can run at the same time in the background.

Another example could be a social media network.

For each post published several actions should take. place:

• SQS Queue 1: Translate post to different languages
• SQS Queue 2: Transform post into audio
• SQS Queue 3: Update user statistics (number of posts)
• Email: Notify followers about new post
• In-App: Notify followers about new post

Questions

There are a lot of questions I hear a lot when people start learning SNS and SQS. Let's have a look at some of the most common ones.

When should I use SNS vs. SQS?

SNS:

• Multiple receivers
• Receiver type is Email, SMS, or an In-App Notification
• The message needs to be forwarded instantly

SQS:

• Reliability is important
• You have only one consumer
• Complicated retry and error handling
• Messages are picked up after some time and not instantly

What other alternatives are out there?

One alternative to mention is definitely EventBridge. There are use cases where the choice for SNS or EventBridge overlap. EventBridge allows you to send events to a special event bus and EventBridge forwards your event to services such as SNS, SQS, Lambda, etc.

One of the main differences is that EventBridge allows to route and filter based on the whole event. There are also many different limits such as the 5 targets per event rule.

Another common message service is Kinesis. Kinesis is meant for processing and analyzing streaming data.

When to use Kinesis vs. SNS vs. SQS?

Kinesis:

• Ongoing stream of data
• Data needs to be processed or analyzed
• Complex real-time requirements

SNS:

• Multiple receivers
• Receiver type is Email, SMS, or an In-App Notification
• The message needs to be forwarded instantly

SQS:

• Reliability is important
• You have only one consumer
• Complicated retry and error handling
• Messages are picked up after some time and not instantly

Is SQS push or poll?

SQS is a poll service. Consumers poll for messages. Even if you use an EventSourceMapping with AWS Lambda. The lambda is still polling multiple times and receives a lot of empty responses.

This is where SQS can get quite expensive if you have a large number of queues that get polled a lot.

Which targets are available for SQS?

SQS can be used with a lot of different targets. All systems that can call the AWS API can receive and remove messages from a queue.

SQS will often be used in combination with:

• AWS Lambda
• Amazon ECS
• Amazon EC2
• Amazon SNS

But you can use it with almost all systems that are out there.

Is SNS push or poll?

SNS is a push system. It won't persist the message at all but push it directly to its consumers.

Can I use SNS for Push Notifications?

Yes! You can use SNS for push notifications on iOS and Android for example.

Final Words

This article gave you an introduction to both AWS Services Amazon Simple Queue Service (SQS) and Amazon Simple Notification Service (SNS). These services build the basics of many distributed and decoupled applications out there. The service exists for many many years and was one of the main pillars of AWS. Using these services to build reliant and performant applications is crucial.

If you want to integrate a message queue into your application but don't want to learn the ins and outs of SQS and SNS I'll recommend giving serverlessq.com a try. It is a hosted message queue solution that builds on top of these AWS services and makes it easier to use them.

References

• SNS Event Destinations
• SNS Limits
• SNS Retries
• Fanout Pattern