Send data to Azure Event Hubs

Most recent version: v0.0.1

See the changelog of the Azure Event Hubs Data sink type here.

Overview

Onum supports integration with Azure Event Hubs.

Azure Event Hubs is a real-time data ingestion that allows you to stream millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges.

Prerequisites

• A valid Microsoft Azure subscription

• An Azure Resource Group

• An Event Hubs namespace container for one or more Event Hubs

• An Event Hub to send events to

Azure Event Hubs Setup

Once you have your Hub to send to, you’ll need to authenticate to receive events.

• Shared Access Signature (SAS)

• Azure Active Directory (Azure AD)

• Connection String (with SAS key)

• Shared Access Policies — Namespace- or Event Hub–level access rules defining specific permissions (Send, Listen, Manage).

Onum setup

1

Log in to your Onum tenant and click Data Sinks> New Data sink.

2

Double-click the Azure Event Hubs Sink.

3

Enter a Name for the new Data Sink. Optionally, add a Description and some Tags to identify the Sink.

4

Decide whether or not to include this Data sink info in the metrics and graphs of the Home area.

5

The Event Hub Namespace is the container for your various event hubs. Click it to see the individual hubs it contains.

6

To see the individual Event Hubs in your Namespace, click it in the list to open its contents. Scroll down to the bottom to see the list of Event Hubs.

Find the event hub to connect to and enter the Event Hub Name in this field.

7

To communicate with an event hub or a namespace, you need a Connection String.

Note that this is required if you select None in the Authentication Method option.

1. Click the Event Hub name in the list to open it.

2. In the left menu, go to Shared Access Policies.

3. If no policy has been created for an event hub, create one with Manage, Send, or Listen access.

4. Select the policy from the list.

5. Select the copy button next to the Connection string-primary key field.

Depending on the version of Azure you are using, the corresponding field may have a different name, so to help you find it, look for the string with the same format:

Endpoint=sb://.servicebus.windows.net/; SharedAccessKeyName=RootManageSharedAccessKey; SharedAccessKey=

8

If an Authentication Method* is required, select the method and enter the credentials of the selected method below. If not, select None.

9

Enter your SASL authentication parameters for Azure Event Hubs.

Parameters	Description
SASL Username	Username for SASL authentication. Note that this is only required if you select SASL in the Authentication Method option.
SASL Password	Add your SASL password from your Secrets or create one. Note that this is only required if you select SASL in the Authentication Method option.
Authentication Token	Add your token from your Secrets or create one. Note that this is only required if you select SAS Token or AAD Token in the Authentication Method option.
Tenant ID	The unique Tenant ID associated with your Azure account. Note that this is only required if you select AAD Token in the Authentication Method option.
Client ID	The Client ID used for authenticating with Azure Active Directory. Note that this is only required if you select AAD Token in the Authentication Method option.

10

Click New secret to create a new one:

• Give the secret a Name.

• Turn off the Expiration date option.

• Click Add new value and paste the secret corresponding to the JWT token you generated before. Remember that the token will be added in the Zscaler configuration.

• Click Save.

Learn more about secrets in Onum in this article.

11

Enter your TLS authentication parameters for Azure Event Hubs.

Note that these parameters are only required if you select TLS in the Authentication Method option.

Add your CA certificate from your Secrets or create one.

Add your Client Certificate from your Secrets or create one.

12

Enter the batch configuration settings, or leave blank if you do not wish to enable batch sending.

• Batch Size - The number of messages to batch before sending to Event Hubs.

• Batch Timeout - The maximum amount of time in milliseconds to wait before sending a batch.

• Retry Attempts - Number of retry attempts before failing.

• Retry Interval - Interval in milliseconds between retry attempts.

Click Create data sink when complete.

Your new Data sink will appear in the Data sinks area list.

Pipeline configuration

When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.

Output configuration

Choose the Message* field that contains the data to be sent.

Click Save to save your configuration.