Google Pub/Sub
Most recent version: v1.0.0
Overview
Onum supports integration with Google Pub/Sub.
Google Pub/Sub is an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages. Pub/Sub allows services to communicate asynchronously.
Select Google Pub/Sub from the list of Data sink types and click Configuration to start.
Data sink configuration
Now you need to specify how and where to send the data, and how to establish a connection with Google Cloud Publisher.
Metadata
Enter the basic information for the new Data sink.
Name*
Enter a name for the new Data sink.
Description
Optionally, enter a description for the Data sinks.
Tags
Add tags to easily identify your Data sink. Hit the Enter key after you define each tag.
Metrics display
Decide whether or not to include this Data sink info in the metrics and graphs of the Home area.
Configuration
Now, add the configuration to establish the connection.
Project ID*
This is a unique string with the following format my-project-123456. To get it:
Go to the Google Cloud Console.
In the top left corner, click on the project drop-down next to the Google Cloud logo (where your current project name is shown).
Each project will have a Project Name and a Project ID.
You can also find it in the Settings tab on the left-hand side.
Topic ID*
Publish a message to a specific topic using the topic ID found on the Pub/Sub Topics area.
Locational endpoint
Enter either a global or regional endpoint to connect to, if you need to override the default regional endpoint. Regional endpoints are in the format:
https://<REGION_NAME>-pubsub.googleapis.com
Note that only gRPC is supported. HTTP is not supported. For the list of endpoints, check this article.
Authentication
Credentials file
The Google Pub/Sub connector uses OAuth 2.0 credentials for authentication and authorization. Create a secret containing these credentials or select one already created.
To find the credentials file, go to Google Cloud Settings > Interoperability.
Scroll down to the Service Account area.
You need to generate and download a service account key from the Google Cloud Console. You will not be able to view this key, so you must have it copied somewhere already. Otherwise, create one here and save it to paste here.
To see existing Service Accounts, go to the menu in the top left and select APIs & Services > Credentials.
Advanced configuration
If you want to use bulk sending, toggle on the Bulk configuration button and configure the following settings:
Event amount
Enter the maximum number of events to accumulate into a batch. The default value is 10.
Event time limit
The number of seconds to wait before considering the bulk as full and sending it on. The default value is 10.
Batch publish timeout
The number of milliseconds to wait before considering a request as timed out. The default value is 100.
Event max size
The maximum bytes to allow through a batch. The default value is 1024.
Max queue size
The maximum number of messages that can be pending acknowledgment. The default value is 100.
Max queue size bytes
Maximum number of bytes that can be pending acknowledgment. The default value is 1024.
Max workers
Number of concurrent workers used to send messages in parallel. The default value is 10.
Click Finish when complete. Your new Data sink will appear in the Data sinks area list.
Pipeline configuration
When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.
Output configuration
Message*
Select the incoming field containing the message to send on.
Click Save to complete the process.
Last updated
Was this helpful?