> For the complete documentation index, see [llms.txt](https://docs.onum.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.onum.com/listeners/listener-integrations/collect-data-from-amazon-sqs.md).

# Collect data from Amazon SQS

{% hint style="info" %}
See the changelog of the **Amazon SQS** Listener [here](/listeners-changelog/amazon-sqs-listener.md).
{% endhint %}

{% hint style="warning" %}
The **Amazon SQS** Listener is a **Pull** Listener and therefore should not be used in environments with more than one cluster.
{% endhint %}

## Overview

Onum supports integration with [Amazon SQS](https://aws.amazon.com/sqs/?nc1=h_ls).

Amazon Simple Queue Service (AWS SQS) is a fully managed message queuing service. Among its many features, the following ones are of special interest to our use case:

* It supports both standard queues (with at-least-once, occasionally unordered delivery semantics) and FIFO queues (exactly-once and fully ordered delivery semantics).
* It supports scaling through the concept of visibility timeout (a period after a consumer reads one message during which this becomes invisible to other consumers). That allows a consumer group to read from the same queue and distribute messages without duplication.

So, what we have is a Listener that we can configure to **read from an existing SQS queue and inject queue messages as events into our platform**. Please note that because of the nature of the API offered to access SQS messages (HTTP-based, max 10 messages each time), this is not a high-throughput Listener.

## Prerequisites

You will need an **IAM** User, role or group with the correct permissions to access and manage SQS. 

{% hint style="warning" %}
When S3 events flow through SNS before reaching SQS, enabling [**Raw Message Delivery**](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html) on the SNS subscription is essential. Without this setting, S3 notifications become wrapped in an SNS JSON envelope, creating nested JSON that's difficult to parse. See more about Raw Message Delivery [here](https://docs.aws.amazon.com/sns/latest/dg/sns-large-payload-raw-message-delivery.html).
{% endhint %}

## Amazon SQS Setup

**Go to IAM (Identity and Access Management)** to manage users, groups, roles and permissions. 

Under **Permissions Policies**, make sure you have assigned the policy `AmazonSQSFullAccess` to give full access to SQS resources. Alternatively, if you have custom permissions, go to **Policies > Create Policy** and in the **JSON** tab, paste your custom JSON. For example:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "sqs:CreateQueue",
        "sqs:GetQueueAttributes",
        "sqs:SendMessage"
      ],
      "Resource": "*"
    }
  ]
}
```

## Onum Setup

{% stepper %}
{% step %}
Log in to your Onum tenant and click **Listeners > New listener**.
{% endstep %}

{% step %}
Double-click the **Amazon SQS** Listener.
{% endstep %}

{% step %}
Enter a **Name** for the new Listener. Optionally, add a **Description** and some **Tags** to identify the Listener.
{% endstep %}

{% step %}
Enter the **Region** displayed in the top right-hand corner of your AWS console.
{% endstep %}

{% step %}
Enter the **Queue URL** of your existing Amazon SQS queue, acting as the endpoint to interact with the desired queue. Use the `GetQueueUrl` command or:

1. **Go to the** AWS Management Console.
2. In the **Search Bar**, type `SQS` and click on **Simple Queue Service (SQS)**.
3. Click on **Queues** in the left panel.
4. Locate your queue from the list and click it.
5. The **Queue URL** will be displayed in the table under **URL**.

This is the correct URL format: `sqs.region.localhost/awsaccountnumber/storedinenvv`
{% endstep %}

{% step %}
Now, choose your **Authentication type** between **manual** and **auto**:

* **Manual** - Choose this option to manually enter your AWS credentials:
  * **Access key ID** - Add the access key from your [Secrets](https://docs.onum.com/administration/global-settings/organization-settings/secrets-management) or create one. The **Access Key ID** is found in the **IAM Dashboard** of the **AWS Management Console**. 
    * In the left panel, click on **Users**.Select your **IAM user**.
    * Under the **Security Credentials** tab, scroll to **Access Keys** and you will find existing **Access Key IDs** (but not the secret access key).
  * **Secret access key** - Add the secret access key from your [Secrets](https://docs.onum.com/administration/global-settings/organization-settings/secrets-management) or create one. 

    Under **Access keys**, you can see your **Access Key IDs**, but AWS **will not show the Secret Access Key**. You must have it saved somewhere. If you don't have the secret key saved, you need to create a new one.
* **Auto** - Choose this option if your IAM role is attached to the host.
  {% endstep %}

{% step %}
You can enable the **Assume Role** option, which provides a more secure and scalable alternative to traditional credential-based authentication.

Instead of using static credentials, this method leverages an Amazon Resource Name (ARN) to define and assume an IAM role with the necessary permissions to access your SQS queue. By introducing the ARN into the Listener configuration, Falcon Onum can securely assume the designated role and interact with your AWS resources in a controlled and auditable way. To enable access, we need cross-account access from our AWS infrastructure to yours. This is a one-time setup. 

First, you must create an IAM role in your AWS account (or use an existing one) with the following trust policy:

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::721751517261:role/<CustomersRole>"
      },
      "Action": ["sts:AssumeRole", "sts:TagSession"]
    }
  ]
}
```

{% hint style="warning" %}
This trust policy allows our AWS account to assume the role. You can attach whatever permissions you want to the role — the trust policy only controls who can assume it, not what it can do.
{% endhint %}

Once the IAM role is ready, please [contact us](broken://spaces/cSjT21I4EUhzghjc1rER/pages/nW8oZycpfVthtI5KYdQs) to share your full role ARN (e.g. `arn:aws:iam::012345678901:role/YourRoleName`) so we can complete the configuration on our end. Then, configure the **Assume Role** options as follows:

* **Role ARN** -  **Amazon Resource Name** used to access S3 and SQS resources. This is the unique identifier for the specific IAM Role that you want to assume and use.
* **External ID** - Shared secret used to authenticate the usage of this role.
* **Role Session** - Name of the session, used to audit usage of this role ( `s3-listener` by default)
* **STS Region** - If not set, it will use the bucket or the queue region.
* **STS Session Duration** - How much the Assume Role session will last before reauthentication. Uses Golang duration strings, like `1s`, `1m`, `1h`. If not set, it uses the maximum session duration configured for that role. The maximum and default duration is 1h.
  {% endstep %}

{% step %}
Optionally, specify which **Message system attributes** are wanted in the response. The set of system attributes chosen by the user correspond to attributes inlined in the message/event.

1. In the **Queues** area, click on **More** or scroll down and go to the **Monitoring** tab.
2. You will see some system attributes (like deduplication and group ID). However, detailed system attributes are typically accessed via the CLI or SDKs.
   {% endstep %}

{% step %}
Optionally, configure the settings in the **Advanced** section:

{% hint style="warning" %}
Proceed with caution when modifying the **Advanced** options. Default values should be enough in most cases.
{% endhint %}

* **Service endpoint** - If you have a custom endpoint, enter it here. The default SQS regional service endpoint will be used by default.
* **Maximum number of messages** - Set a limit for the maximum number of messages to receive in the notifications queue for each request. The minimum value is `1`, and the maximum and default value is `10`.
* **Visibility timeout** - The time during which messages delivered to a consumer, but not yet acknowledged, are hidden from other consumers. Valid values go from `30s` to `12h`, and the default value is `2m`.
* **Wait time** - Set a limit for the maximum number of messages to receive in the notifications queue for each request. The minimum value is `5`, and the maximum and default value is `10`.
* **Minimum retry time** - Set the minimum amount of time to wait before retrying. The default and minimum value is `1s`, and the maximum value is `10m`.
* **Maximum retry time** - Set the minimum amount of time to wait before retrying. The default and minimum value is `1s`, and the maximum value is `10m`.
  {% endstep %}

{% step %}
Finally, click **Create labels**. Optionally, you can set labels to be used for internal Onum routing of data. By default, data will be set as **Unlabelled**.  &#x20;

{% hint style="info" %}
Learn more about labels in [this article](/listeners/labels.md).
{% endhint %}
{% endstep %}

{% step %}
Click **Create listener** when you're done.
{% endstep %}
{% endstepper %}

## Output Ports <a href="#ports" id="ports"></a>

The **Amazon SQS** Listener has only a single output port:

* **Default port** - Events are sent through this port if no error occurs while processing them.