# Send data to Google BigQuery

{% hint style="info" %}
See the changelog of the **Google Big Query** Data sink type [here](/data-sinks/google-bigquery-data-sink.md).
{% endhint %}

## Overview

Onum supports integration with [Google BigQuery](https://cloud.google.com/bigquery?hl=en).

Google BigQuery is an autonomous data-to-AI platform, automating the entire data life cycle, from ingestion to AI-driven insights, so you can go from data to AI to action faster.

## Prerequisites

You will need:

* a [Google Cloud account](mailto:undefined)
* an existing project in the [Google Cloud Console](https://console.cloud.google.com/projectselector2/home/dashboard)

## Google BigQuery Setup 

Once you have your project to connect to, make sure your user or service account has the proper roles:

* **BigQuery Admin** (`roles/bigquery.admin`) – full access.
* **BigQuery Data Editor/Viewer** – if you only need to write or read data.
* **BigQuery Job User** (`roles/bigquery.jobUser`) – required to run queries.

\
Go to **IAM & Admin > IAM > Add Principal** and choose a role like *BigQuery Admin.*

Enable these in your project:

* **BigQuery API**
* **Cloud Storage API** 

Make a note of the **Project ID** for later use.

## Onum setup

{% stepper %}
{% step %}
Log in to your Onum tenant and click **Data Sinks> New Data sink**.
{% endstep %}

{% step %}
Double-click the **Google BigQuery** Sink.
{% endstep %}

{% step %}
Enter a **Name** for the new Data Sink. Optionally, add a **Description** and some **Tags** to identify the Sink.
{% endstep %}

{% step %}
Decide whether or not to include this Data sink info in the metrics and graphs of the [**Home**](/the-workspace/home.md) area.
{% endstep %}

{% step %}
Enter the **Project ID**<mark style="color:red;">**\***</mark> unique string with the following format `my-project-123456`.&#x20;

To get it:

1. Go to the Google Cloud Console.
2. In the top left corner, click on the **project drop-down** next to the Google Cloud logo (where your current project name is shown).
3. Each project will have a **Project Name** and a **Project ID**.
4. You can also find it in the **Settings** tab on the left-hand side.

<figure><img src="/files/vXJj20SWeXeF75m0H3O6" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
The Google BigQuery connector uses OAuth 2.0 credentials for authentication and authorization.&#x20;

Enter the **Credentials File**<mark style="color:red;">**\***</mark> by creating a [secret](/administration/tenant-menu.md) containing these credentials or select one already created. To find the credentials file:

1. Go to Google Cloud **Settings > Interoperability**.
2. Scroll down to the **Service Account** area.
3. You need to generate and download a **service account key** from the Google Cloud Console. You will not be able to view this key, so you must have it copied somewhere already. Otherwise, create one here and save it to paste here.
4. To see existing Service Accounts, go to the menu in the top left and select **APIs & Services > Credentials**.

<figure><img src="/files/t15BPftuN6tUgtso7Vwj" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}
Click **New secret** to create a new one:

* Give the secret a **Name**.
* Turn off the **Expiration date** option.
* Click **Add new value** and paste the secret corresponding to the JWT token you generated before. Remember that the token will be added in the Zscaler configuration.
* Click **Save**.

<figure><picture><source srcset="/files/NeeWsSQzoChVxRIY76Nt" media="(prefers-color-scheme: dark)"><img src="/files/1oTccyPmgZJ1laY7IhZH" alt=""></picture><figcaption></figcaption></figure>

{% hint style="info" %}
Learn more about secrets in Onum in [this article](/administration/global-settings/organization-settings/secrets-management.md).
{% endhint %}
{% endstep %}
{% endstepper %}

Click **Create data sink** when complete.&#x20;

Your new Data sink will appear in the **Data sinks** area list.

## Pipeline configuration

When it comes to using this Data sink in a [Pipeline](https://docs.onum.com/the-workspace/pipelines), you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select **Configuration**.

### Output configuration

{% stepper %}
{% step %}

#### Data to insert

Here is where you decide how your data will appear in your BigQuery project.

* **Dataset**<mark style="color:red;">**\***</mark> Give a name to the dataset that will appear in your BigQuery storage.
* **Table**<mark style="color:red;">**\***</mark> Enter a name for the table to insert the values into.
  {% endstep %}

{% step %}

#### Column / Value pairs

Click **Add Element** to add as many pairs as needed.

* **Column**<mark style="color:red;">**\***</mark> Enter a name for the column that will appear in Big Query.
* **Value**<mark style="color:red;">**\***</mark> Select the incoming event that contains the value to send.
  {% endstep %}

{% step %}

#### Bulk options

If you wish to enable bulk sending, set **Bulk allow**<mark style="color:red;">**\***</mark> to ***True.*** Otherwise, set it to **false**.

Enter the **Event max amount**<mark style="color:red;">**\***</mark> number of events to accumulate into a bulk. The minimum value is `0` and the maximum value is `5000`.

The **Event max time in seconds**<mark style="color:red;">**\***</mark> is the number of seconds to wait before considering the bulk as full and sending it on. The minimum value is `0`.
{% endstep %}
{% endstepper %}

Click **Save** to complete the process.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.onum.com/the-workspace/data-sinks/data-sink-integrations/send-data-to-google-cloud-products/send-data-to-google-bigquery.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.