> For the complete documentation index, see [llms.txt](https://docs.onum.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.onum.com/the-workspace/data-sinks/data-sink-integrations/send-data-using-http/send-data-to-elasticsearch.md).

# Send data to Elasticsearch

## Overview <a href="#overview" id="overview"></a>

You can send logs to [Elasticsearch](https://www.elastic.co/) via HTTP using our [HTTP Data sink](/the-workspace/data-sinks/data-sink-integrations/send-data-using-http.md).

## Prerequisites  <a href="#data-sink-configuration" id="data-sink-configuration"></a>

Connecting to an Elasticsearch cluster requires different configurations depending on whether you are using Elastic Cloud or a self-managed cluster.

To connect to Elastic Cloud, you need the **Cloud ID** and an **API Key**. Both of these credentials can be obtained from the Elastic Cloud console.

You will typically pass these values into a configuration object, like this Go example:

```
cfg := elasticsearch.Config{
    CloudID: "YOUR_CLOUD_ID", // Found in the cloud console
    APIKey:  "YOUR_API_KEY"   // Generated via the cloud console
}
es, err := elasticsearch.NewClient(cfg)
```

> Ensure you securely store your generated API Key immediately after creation, as Elastic Cloud does not allow you to view the key again for security reasons.

#### Connecting to a Self-Managed Cluster

Security features,specifically Authentication and TLS, are enabled by default. Trust the cluster's certificates to establish a secure HTTPS connection.

When a self-managed Elasticsearch cluster is started for the first time, the terminal output provides the necessary security credentials. You will see a block containing the following critical information:

* `elastic` user password: The initial password for the built-in superuser.
* HTTP CA certificate SHA-256 fingerprint: A unique hash used to identify and verify the cluster's self-signed security certificate.

You should record both the `elastic` user password and the CA fingerprint for use in your client connection configuration. These are commonly stored as environment variables, such as `ELASTIC_PASSWORD` and `CERT_FINGERPRINT`.

The root CA Certificate used by the cluster (named `http_ca.crt`) is located within your Elasticsearch configuration directory (e.g., `$ES_CONF_PATH/certs/`). If you are running Elasticsearch in a container environment like Docker, consult the relevant documentation for how to retrieve this file.

Once you have access to the `http_ca.crt` file, you pass its contents via the client configuration's `CACert` field:

```
cert, _ := os.ReadFile("/path/to/http_ca.crt") // Load the certificate file content

cfg := elasticsearch.Config{
    Addresses: []string{
        "https://localhost:9200", // Cluster address
    },
    Username: "elastic",
    Password: ELASTIC_PASSWORD, // Use the recorded password
    CACert:   cert               // Pass the certificate content
}
es, err := elasticsearch.NewClient(cfg)
```

{% hint style="info" %}
For more detailed information, [see this article.](https://www.elastic.co/docs/reference/elasticsearch/clients/go/connecting)
{% endhint %}

## Data sink configuration <a href="#data-sink-configuration" id="data-sink-configuration"></a>

To start sending data to Elasticsearch, follow these steps:

{% stepper %}
{% step %}
Create a new [HTTP Data sink](/the-workspace/data-sinks/data-sink-integrations/send-data-using-http.md). To do it, go to **Data sinks > New Data sink** and double-click **HTTP**.
{% endstep %}

{% step %}
Give your Data sink a **Name** and, optionally, add a **Description** and some **Tags**. Click **Finish** when you're done.
{% endstep %}

{% step %}
Now, drag your Data sink to the required [Pipeline](/the-workspace/pipelines.md) canvas. Link it to the required [Listener](/the-workspace/listeners.md)/[Action](/the-workspace/pipelines/actions.md) and double-click it to configure it.
{% endstep %}

{% step %}
Fill the following parameters as follows:

* **HTTP method**<mark style="color:red;">**\***</mark> - Choose `POST`
* **URL**<mark style="color:red;">**\***</mark> - Enter `<elastic_endpoint>`
* Choose the field containing the **Message** you wish to send on.
* **Content-Type** - Choose your required data type.
* **Support special characters** - Set to `true`
* **Use gzip**, **HTTP headers** Set as required
  {% endstep %}

{% step %}
In the **Bulk configuration** section, fill in the parameters as follows:

* **Bulk allow**<mark style="color:red;">**\***</mark> - Set to `true`
* Choose **Manual delimiter**<mark style="color:red;">**\***</mark> and leave it as new line (`\n`).
* **Maximum number of buffers per server URL**<mark style="color:red;">**\***</mark> - Enter `50`.
* **Event amount**<mark style="color:red;">**\***</mark>, **Event time limit**<mark style="color:red;">**\***</mark> - Set both to `1`.
  {% endstep %}

{% step %}
Set the **Authentication type**<mark style="color:red;">**\***</mark> to **API key**. Enter your **API key name**<mark style="color:red;">**\***</mark> and choose the required **API key value**<mark style="color:red;">**\***</mark>. Create a [Secret](https://docs.onum.com/administration/tenant-menu) containing your API key or select one already created.
{% endstep %}

{% step %}
In the [Secrets](/administration/global-settings/organization-settings/secrets-management.md) area, **New secret** to create a new one:

* Give the secret a **Name**.
* Turn off the **Expiration date** option.
* Click **Add new value** and paste the corresponding value.
* Click **Save**.

<figure><picture><source srcset="/files/NeeWsSQzoChVxRIY76Nt" media="(prefers-color-scheme: dark)"><img src="/files/1oTccyPmgZJ1laY7IhZH" alt=""></picture><figcaption></figcaption></figure>

{% hint style="info" %}
Learn more about secrets in Onum in [this article](/administration/global-settings/organization-settings/secrets-management.md).
{% endhint %}

You can now select the created secrets in the configuration.
{% endstep %}

{% step %}
Fill in the rest of the parameters and required, and click **Save**.
{% endstep %}
{% endstepper %}

<figure><picture><source srcset="/files/okH1pjlK0WMHaUCkOtaQ" media="(prefers-color-scheme: dark)"><img src="/files/FYW2qrE6g9eQvp8KueWN" alt=""></picture><figcaption></figcaption></figure>