Apache Kafka
Most recent version: v2.0.0
This is a Pull Listener and therefore should not be used in environments with more than one cluster.
Overview
Onum supports integration with Apache Kafka.
Apache Kafka is a distributed, fault-tolerant, high-throughput, and scalable streaming platform. It's used for building real-time data pipelines and streaming applications.
Select Apache Kafka from the list of Listener types and click Configuration to start.
Listener configuration
Now you need to specify how and where to collect the data and how to establish a connection with Apache Kafka.
Metadata
Enter the basic information for the new Listener.
Name*
Enter a name for the new Listener.
Description
Optionally, enter a description for the Listener.
Tags
Add tags to easily identify your Listener. Hit the Enter key after you define each tag.
Configuration
Now, add the configuration to establish the connection.
Bootstrap servers
The initial host-port pair that acts as the starting point to access the full set of alive servers in the cluster. This is a comma-separated list of host and port pairs using : as the separator, e.g. localhost:9092,another.host:9092
Enter your value and click Add element to add the required elements.
Group ID
The group ID is a string that uniquely identifies the group of consumer processes. Find this in your Kafka Cluster at Home > Configuration > Consumer Properties.
Topics
The topic(s) to connect to. Use kafka-topics --bootstrap-server :9092 --describe and write the result here. Click Add element to add as many topics as required.
Auto offset reset policy*
This policy defines the behavior when there are no committed positions available or when an offset is out of range. Choose between Earliest, Latest, or None.
Authentication configuration
Select the required authentication type in the Authentication type* parameter to enable these settings:
None
Choose this option if no authentication is required.
Plain
Enter your Username* and select your Password* from your Secrets or create a new one.
Scram
Enter the required information:
Username* - Enter your username.
Password* - Select your password from your Secrets or create a new one.
SCRAM mechanism* - Choose either SHA-256 or SHA-512.
mTLS
Enter the required information:
CA Certificate* - Select your CA certificate from your Secrets or create a new one.
Client certificate* - Select your client certificate from your Secrets or create a new one.
Client key* - Select your client key from your Secrets or create a new one.
Skip verify - Select true to skip or false to require verification.
Server name - Enter the name of the server to connect to.
Minimum TLS version - Select the required minimum version from the menu.
Click Create labels to move on to the next step and define the required Labels if needed.
Last updated
Was this helpful?