Amazon S3 Listener

Most recent version: v2.1.0

Learn more about this Listener type in this article.

v2.1.0

Released on 12/11/2025

• Possibility to use external S3 buckets and SQS queues using AssumeRole.

v2.0.1

Released on 11/24/2025

• New JSON type where you can specify the path you want to retrieve and if you want to generate one event for each array element or the whole element.

• Element selection will discard data that is not pointed by the path so the whole document does not need to be stored in memory at once.

v1.2.0

Released on 10/30/2025

• Added support for configurable authentication type: auto (e.g., EC2 IAM role) or manual (static credentials).

• Improved JSON Lines parsing: trailing newline at the end of the file is now optional.

• Added CSV format support: Full support for processing CSV files from S3 buckets.

  • Streaming CSV processing using Go's builtin encoding/csv package with memory usage bounded independently of file size

  • Dual output modes: CSV (preserves original formatting) or JSON (converts to structured objects)

  • Comprehensive CSV configuration options:

    • Header row handling (with/without headers)

    • Custom delimiters (comma, semicolon, tab, or custom)

    • Text encoding support (UTF-8, UTF-16, ISO-8859-1, Windows-1252)

    • Advanced parsing options (trimLeadingSpace, lazyQuotes, fieldsPerRecord, comment character)

  • Auto-detection: CSV files with .csv extension are automatically detected when format is set to "auto"

  • Robust error handling: Malformed CSV records are logged and skipped without stopping processing

v1.0.0

Released on 06/04/2025

• Added support for configurable authentication type: auto (e.g., EC2 IAM role) or manual (static credentials).

• Improved JSON Lines parsing: trailing newline at the end of the file is now optional.

v0.0.1

Released on 03/25/2025

Initial version