Google Pub/Sub
Most recent version: v0.0.2
Last updated
Was this helpful?
Most recent version: v0.0.2
Last updated
Was this helpful?
The Google Publisher Sink lets you send data to your Google Pub messaging service.
Select Google Pub/Sub from the list of Data sink types and click Configuration to start.
Now you need to specify how and where to send the data, and how to establish a connection with Google Cloud Publisher.
Enter the basic information for the new Data sink.
Name*
Enter a name for the new Data sink.
Description
Optionally, enter a description for the Data sinks.
Tags
Add tags to easily identify your Data sink. Hit the Enter key after you define each tag.
Now add the configuration to establish the connection.
Project ID*
This is a unique string with the following format my-project-123456
Go to the Google Cloud Console.
In the top left corner, click on the project drop-down next to the Google Cloud logo (where your current project name is shown).
Each project will have a Project Name and a Project ID.
You can also find it in the Settings tab on the left-hand side.
Topic ID*
Publish a message to a specific topic using the topic ID found on the Pub/sub Topics area.
Bulk allow*
Select true to set your bulk settings, or false to ignore.
Event amount
Enter the amount of events to accumulate into a bulk.
Event time limit
The number of seconds to wait before considering the bulk as full and sending it on.
Event max size
The maximum size of the events to allow through.
Max outstanding messages
Maximum number of messages that can be pending acknowledgment.
Max outstanding bytes
Maximum number of bytes that can be pending acknowledgment.
Batch size
Number of messages to group together in a single batch for sending.
Max workers
Number of concurrent workers used to send messages in parallel.
Max retries
Maximum number of retry attempts for failed message deliveries.
Authentication credentials*
To find the credentials file, go to Google Cloud Settings>Interoperability.
Scroll down to the Service Account area.
You need to generate and download a service account key from the Google Cloud Console. You will not be able to view this key, so you must have it copied somewhere already. Otherwise, create one here and save it to paste here.
To see existing Service Accounts, go to the menu in the top left and select APIs & Services>Credentials.
Click Finish when complete.
When you use this sink in a Pipeline, you can configure the output parameters. This is where you provide the message to be processed in Google Publisher.
Message*
Select the incoming field containing the message to send on.
Click Save to complete the process.
For information on where to find these values, see .
Decide whether or not to include this Data sink info in the metrics and graphs of the area.
The Google Pub/Sub connector uses OAuth 2.0 credentials for authentication and authorization. Create a containing these credentials or select one already created.
