# Send data to Elasticsearch

## Overview <a href="#overview" id="overview"></a>

You can send logs to [Elasticsearch](https://www.elastic.co/) via HTTP using our [HTTP Data sink](/the-workspace/data-sinks/data-sink-integrations/send-data-using-http.md).

## Prerequisites  <a href="#data-sink-configuration" id="data-sink-configuration"></a>

Connecting to an Elasticsearch cluster requires different configurations depending on whether you are using Elastic Cloud or a self-managed cluster.

To connect to Elastic Cloud, you need the **Cloud ID** and an **API Key**. Both of these credentials can be obtained from the Elastic Cloud console.

You will typically pass these values into a configuration object, like this Go example:

```
cfg := elasticsearch.Config{
    CloudID: "YOUR_CLOUD_ID", // Found in the cloud console
    APIKey:  "YOUR_API_KEY"   // Generated via the cloud console
}
es, err := elasticsearch.NewClient(cfg)
```

> Ensure you securely store your generated API Key immediately after creation, as Elastic Cloud does not allow you to view the key again for security reasons.

#### Connecting to a Self-Managed Cluster

Security features,specifically Authentication and TLS, are enabled by default. Trust the cluster's certificates to establish a secure HTTPS connection.

When a self-managed Elasticsearch cluster is started for the first time, the terminal output provides the necessary security credentials. You will see a block containing the following critical information:

* `elastic` user password: The initial password for the built-in superuser.
* HTTP CA certificate SHA-256 fingerprint: A unique hash used to identify and verify the cluster's self-signed security certificate.

You should record both the `elastic` user password and the CA fingerprint for use in your client connection configuration. These are commonly stored as environment variables, such as `ELASTIC_PASSWORD` and `CERT_FINGERPRINT`.

The root CA Certificate used by the cluster (named `http_ca.crt`) is located within your Elasticsearch configuration directory (e.g., `$ES_CONF_PATH/certs/`). If you are running Elasticsearch in a container environment like Docker, consult the relevant documentation for how to retrieve this file.

Once you have access to the `http_ca.crt` file, you pass its contents via the client configuration's `CACert` field:

```
cert, _ := os.ReadFile("/path/to/http_ca.crt") // Load the certificate file content

cfg := elasticsearch.Config{
    Addresses: []string{
        "https://localhost:9200", // Cluster address
    },
    Username: "elastic",
    Password: ELASTIC_PASSWORD, // Use the recorded password
    CACert:   cert               // Pass the certificate content
}
es, err := elasticsearch.NewClient(cfg)
```

{% hint style="info" %}
For more detailed information, [see this article.](https://www.elastic.co/docs/reference/elasticsearch/clients/go/connecting)
{% endhint %}

## Data sink configuration <a href="#data-sink-configuration" id="data-sink-configuration"></a>

To start sending data to Elasticsearch, follow these steps:

{% stepper %}
{% step %}
Create a new [HTTP Data sink](/the-workspace/data-sinks/data-sink-integrations/send-data-using-http.md). To do it, go to **Data sinks > New Data sink** and double-click **HTTP**.
{% endstep %}

{% step %}
Give your Data sink a **Name** and, optionally, add a **Description** and some **Tags**. Click **Finish** when you're done.
{% endstep %}

{% step %}
Now, drag your Data sink to the required [Pipeline](/the-workspace/pipelines.md) canvas. Link it to the required [Listener](/the-workspace/listeners.md)/[Action](/the-workspace/pipelines/actions.md) and double-click it to configure it.
{% endstep %}

{% step %}
Fill the following parameters as follows:

* **HTTP method**<mark style="color:red;">**\***</mark> - Choose `POST`
* **URL**<mark style="color:red;">**\***</mark> - Enter `<elastic_endpoint>`
* Choose the field containing the **Message** you wish to send on.
* **Content-Type** - Choose your required data type.
* **Support special characters** - Set to `true`
* **Use gzip**, **HTTP headers** Set as required
  {% endstep %}

{% step %}
In the **Bulk configuration** section, fill in the parameters as follows:

* **Bulk allow**<mark style="color:red;">**\***</mark> - Set to `true`
* Choose **Manual delimiter**<mark style="color:red;">**\***</mark> and leave it as new line (`\n`).
* **Maximum number of buffers per server URL**<mark style="color:red;">**\***</mark> - Enter `50`.
* **Event amount**<mark style="color:red;">**\***</mark>, **Event time limit**<mark style="color:red;">**\***</mark> - Set both to `1`.
  {% endstep %}

{% step %}
Set the **Authentication type**<mark style="color:red;">**\***</mark> to **API key**. Enter your **API key name**<mark style="color:red;">**\***</mark> and choose the required **API key value**<mark style="color:red;">**\***</mark>. Create a [Secret](https://docs.onum.com/administration/tenant-menu) containing your API key or select one already created.
{% endstep %}

{% step %}
In the [Secrets](/administration/global-settings/organization-settings/secrets-management.md) area, **New secret** to create a new one:

* Give the secret a **Name**.
* Turn off the **Expiration date** option.
* Click **Add new value** and paste the corresponding value.
* Click **Save**.

<figure><picture><source srcset="/files/NeeWsSQzoChVxRIY76Nt" media="(prefers-color-scheme: dark)"><img src="/files/1oTccyPmgZJ1laY7IhZH" alt=""></picture><figcaption></figcaption></figure>

{% hint style="info" %}
Learn more about secrets in Onum in [this article](/administration/global-settings/organization-settings/secrets-management.md).
{% endhint %}

You can now select the created secrets in the configuration.
{% endstep %}

{% step %}
Fill in the rest of the parameters and required, and click **Save**.
{% endstep %}
{% endstepper %}

<figure><picture><source srcset="/files/okH1pjlK0WMHaUCkOtaQ" media="(prefers-color-scheme: dark)"><img src="/files/FYW2qrE6g9eQvp8KueWN" alt=""></picture><figcaption></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.onum.com/the-workspace/data-sinks/data-sink-integrations/send-data-using-http/send-data-to-elasticsearch.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.