Amazon SQS

Most recent version: v1.0.0

See the changelog of this Data sink type here.

Overview

Onum supports integration with Amazon SQS.

Amazon SQS (Simple Queue Service) is a fully managed message queuing service that enables asynchronous communication between different parts of a distributed system. It allows developers to send, store, and retrieve messages without needing to manage the underlying queue infrastructure.

Select Amazon SQS from the list of Data sink types and click Configuration to start.

Data sink configuration

Now you need to specify how and where to send the data, and how to establish a connection with Amazon SQS.

Metadata

Enter the basic information for the new Data sink.

Parameters
Description

Name*

Enter a name for the new Data sink.

Description

Optionally, enter a description for the Data sink.

Tags

Add tags to identify your Data sink easily. Hit the Enter key after you define each tag.


Metrics display

Decide whether or not to include this Data sink info in the metrics and graphs of the Home area.


Configuration

Now, add the configuration to establish the connection.

AWS Authentication

Parameter
Description

Region*

Choose your default region from the list provided. Your region is displayed in the top right-hand corner of your AWS console, as you can see in the image below.

Access Key ID*

Add the access key from your Secrets or create one. The Access Key ID is found in the IAM Dashboard of the AWS Management Console:

  1. In the left panel, click on Users.

  2. Select your IAM user.

  3. Under the Security Credentials tab, scroll to Access Keys, and you will find existing Access Key IDs (but not the secret access key).

Secret Access Key*

Add the secret access key from your Secrets or create one.

Under Access keys, you can see your Access Key IDs, but AWS will not show the Secret Access Key. You must have it saved somewhere. If you don't have the secret key saved, you need to create a new one.

Bulk Configuration

Parameter
Description

Bulk configuration

Set the Bulk configuration parameter to true if you want to allow bulk sending. Configure the following parameters:

  • Event time limit* - Time in seconds to wait before sending the batch. The default value is 1, and the minimum is 0.

  • Trigger conditions

    • Number of events* - Maximum number of events to send in a single batch. The maximum and default value is 10.

    • Event size* - Maximum limit of the batch size in bytes. The maximum and default value is 262144. The minimum value is 1.

Other

Parameter
Description

Custom Endpoint

Specify the full URL of a custom endpoint if you want to use one instead of the default SQS endpoint.

Delay seconds

The length of time, in seconds, for which to delay a specific message. The minimum value is 1 and the maximum value is 900.

Click Finish when complete. Your new Data sink will appear in the Data sinks area list.

Pipeline configuration

When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.

Output configuration

Parameter
Description

Message*

Select the field that contains the message. The data type must be string.

Queue configuration

Parameter
Description

Standard

Queue name*: The URL of your existing Amazon SQS queue to send the data to. To get it:

  1. Go to the AWS Management Console.

  2. In the Search Bar, type SQS and click on Simple Queue Service (SQS).

  3. Click on Queues in the left panel.

  4. Locate your queue from the list and click it.

  5. The Queue URL will be displayed in the table under URL.

This is the correct format: sqs.region.localhost/awsaccountnumber/storedinenvvar

FIFO

  1. Queue name*: The URL of your existing Amazon SQS queue to send the data to. To get it:

    1. Go to the AWS Management Console.

    2. In the Search Bar, type SQS and click on Simple Queue Service (SQS).

    3. Click on Queues in the left panel.

    4. Locate your queue from the list and click it.

    5. The Queue URL will be displayed in the table under URL.

      This is the correct format: sqs.region.localhost/awsaccountnumber/storedinenvvar

  2. Message group ID*: Select an incoming field to serve as the message group ID.

  3. Send message deduplication: Toggle ON to enable message deduplication and configure the following fields

    1. InEventField: choose the incoming field for the message deduplication ID.

    2. Message content hash: when content-based deduplication is enabled, the message deduplication ID is optional.

Click Save to save your configuration.

Last updated

Was this helpful?