SQS
Most recent version: v0.0.1
Last updated
Was this helpful?
Most recent version: v0.0.1
Last updated
Was this helpful?
See the changelog of this Listener type .
Onum supports integration with Amazon Simple Queue Service.
Amazon Simple Queue Service (AWS SQS) is a fully managed message queuing service. Among its many features, the following ones are of special interest to our use case:
It supports both standard queues (with at-least-once, occasionally unordered delivery semantics) and fifo queues (exactly-once and fully ordered delivery semantics).
It supports scaling through the concept of visibility timeout (a period of time after a consumer reads one message during which this becomes invisible to other consumers). That allows a consumer group to read from the same queue and distribute messages without duplication.
So, what we want is a listener that we can configure to read from an existing SQS queue, and inject queue messages as events into our platform. Please note that because of the nature of the api offered to access sqs messages (http based, max 10 messages each time), this is not a high throughput listener.
This is a Pull listener, therefore should not be used in environments with more than one cluster.
Select SQS from the list of Listener types and click Configuration to start.
Now you need to specify how and where to collect the data, and how to establish a connection with Amazon SQS.
Enter the basic information for the new Listener.
Name*
Enter a name for the new Listener.
Description
Optionally, enter a description for the Listener.
Tags
Add tags to easily identify your Listener. Hit the Enter key after you define each tag.
Now add the configuration to establish the connection.
Region
The region of your AWS data center. Your region is displayed in the top right-hand corner
of your AWS console.
Queue url*
The URL of your existing Amazon SQS queue, acting as the endpoint to interact with the desired queue. Use the GetQueueUrl command or
Go to the AWS Management Console.
In the Search Bar, type "SQS" and click on Simple Queue Service (SQS).
Click on Queues in the left panel.
Locate your queue from the list and click it.
The Queue URL will be displayed in the table under URL.
The correct format: sqs.region.localhost/awsaccountnumber/storedinenvvar
Message system attributes
Optionally, specify which system attributes are wanted in the response. The set of system attributes chosen by the user correspond to attributes inlined in the message/event.
In the Queues area, click on More or scroll down and go to the Monitoring tab.
You will see some system attributes (like deduplication and group ID). However, detailed system attributes are typically accessed via the CLI or SDKs.
Authentication is not specific to SQS but rather AWS IAM (Identity and Access Management). If you are connecting from an IAM console, enter the authentication credentials here.
Access key ID*
In the left panel, click on Users.
Select your IAM user.
Under the Security Credentials tab, scroll to Access Keys and you will find existing Access Key IDs (but not the secret access key).
Secret access key*
Under Access keys, you can see your Access Key IDs, but AWS will not show the Secret Access Key. You must have it saved somewhere. If you don't have the secret key saved, you need to create a new one
These fields already have default values assigned and can be left as is. Here you can assign personalized values if needed.
Maximum number of messages*
Set a limit for the maximum amount of messages to let through (between 1-10).
Visibility timeout*
Set how many seconds to leave a message as hidden in the queue between being sent to the consumer and confirmed as delivered.
Wait time*
When the queue is empty, set how long to wait for messages before deeming the request as timed out (between 1 - 20 seconds).
Minimum retry time*
Set the minimum amount of seconds to retry.
Maximum retry time*
Set the maximum amount of seconds to retry.
Click Create labels to move on to the next step and define the required Labels if needed.
Add the access key from your or create one. The Access Key ID is found in the IAM Dashboard of the AWS Management Console.
Add the secret access key from your or create one.
