Send data using OpenTelemetry

Most recent version: v0.0.1

See the changelog of this Data sink type here.

Overview

Onum supports integration with OpenTelemetry.

OpenTelemetry (OTel) is an open-source observability framework that provides a standardized way to collect and manage telemetry data (logs, metrics, and traces) from applications and infrastructure.

Open Telemetry Setup

The prerequisites for sending data to OpenTelemetry generally fall into three categories: Instrumentation, Collector Setup, and Configuration. You need these pieces in place to capture, process, and export observability data (traces, metrics, and logs).

• Use the OTel API to manually or automatically instrument the code.

  • Traces: Define spans around operations you want to measure (e.g., database calls, function execution).

  • Metrics: Record counter, gauge, or histogram values for system health (e.g., CPU usage, request latency).

  • Logs: Configure your logging frameworks (like Log4j or Logrus) to use the OTel format and include trace context (known as contextual logging).

While you can send data directly from Onum to a backend, the recommended and best practice method is to use the OpenTelemetry Collector.

• You must have the OTel Collector deployed in a location accessible to Onum (e.g., as a sidecar, a daemon set on Kubernetes, or a standalone service).

• The collector needs a YAML configuration file that defines three things:

  • Receivers: How it receives data from your application (usually using the OTLP protocol).

  • Processors: How it manipulates the data (e.g., filtering, batching, adding resource attributes).

  • Exporters: How it sends the final data to the backend (e.g., to Jaeger, Prometheus, or a cloud provider).

Both Onum and the collector must be configured to use the OpenTelemetry Protocol (OTLP).

• Set the environment variable OTEL_EXPORTER_OTLP_ENDPOINT in Onum to point to the collector's address and port (e.g., http://otel-collector:4318).

• Establish a network connection to the OTLP receiver endpoint (often using gRPC on port 4317 or HTTP on port 4318).

Onum Setup

1

Log in to your Onum tenant and click Data Sinks > New Sink.

2

Double-click the Open Telemetry Sink.

3

Enter a Name for the new Sink. Optionally, add a Description and some Tags to identify the Sink.

4

Choose the desired protocol to send your data (gRPC or HTTP).

5

Configure your OTLP/gRPC or OTLP/HTTP endpoint (e.g. otel-collector.mycompany.com:4317-

6

Set the HTTP Port* to establish the connection with the protocol.

7

Set the Use gzip Compression option to true if you want to compress your data in gzip format.

8

Set the milliseconds to wait for data to be successfully transmitted before terminating the attempt. The minimum value is 1.

9

Choose how to export the three distinct types of telemetry data:

Parameter	Description
Send Traces*	Set this option to true if you want to send Traces to your OpenTelemetry account. If you select the HTTP sending protocol, you'll be prompted to optionally include your HTTP Traces Path e.g. http://collector:port/v1/traces
Send Metrics*	Set this option to true if you want to send Metrics to your OpenTelemetry account. If you select the HTTP sending protocol, you'll be prompted to optionally include your HTTP Metrics Path e.g. http://collector:port/v1/metrics
Send Logs*	Set this option to true if you want to send Logs to your OpenTelemetry account. If you select the HTTP sending protocol, you'll be prompted to optionally include your HTTP Logs Path e.g. http://collector:port/v1/logs

Depending on the type(s) of data you're sending (Traces, Metrics, and/or Logs), your events must have a specific set of information in OpenTelemetry format. Check an example for every data type below:

Traces

{
  "resourceSpans": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my.service"
            }
          }
        ]
      },
      "scopeSpans": [
        {
          "scope": {
            "name": "my.library",
            "version": "1.0.0",
            "attributes": [
              {
                "key": "my.scope.attribute",
                "value": {
                  "stringValue": "some scope attribute"
                }
              }
            ]
          },
          "spans": [
            {
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "EEE19B7EC3C1B174",
              "parentSpanId": "EEE19B7EC3C1B173",
              "name": "I'm a server span",
              "startTimeUnixNano": "1544712660000000000",
              "endTimeUnixNano": "1544712661000000000",
              "kind": 2,
              "attributes": [
                {
                  "key": "my.span.attr",
                  "value": {
                    "stringValue": "some value"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Metrics

{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my.service"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {
            "name": "my.library",
            "version": "1.0.0",
            "attributes": [
              {
                "key": "my.scope.attribute",
                "value": {
                  "stringValue": "some scope attribute"
                }
              }
            ]
          },
          "metrics": [
            {
              "name": "my.counter",
              "unit": "1",
              "description": "I am a Counter",
              "sum": {
                "aggregationTemporality": 1,
                "isMonotonic": true,
                "dataPoints": [
                  {
                    "asDouble": 5,
                    "startTimeUnixNano": "1544712660300000000",
                    "timeUnixNano": "1544712660300000000",
                    "attributes": [
                      {
                        "key": "my.counter.attr",
                        "value": {
                          "stringValue": "some value"
                        }
                      }
                    ]
                  }
                ]
              }
            },
            {
              "name": "my.gauge",
              "unit": "1",
              "description": "I am a Gauge",
              "gauge": {
                "dataPoints": [
                  {
                    "asDouble": 10,
                    "timeUnixNano": "1544712660300000000",
                    "attributes": [
                      {
                        "key": "my.gauge.attr",
                        "value": {
                          "stringValue": "some value"
                        }
                      }
                    ]
                  }
                ]
              }
            },
            {
              "name": "my.histogram",
              "unit": "1",
              "description": "I am a Histogram",
              "histogram": {
                "aggregationTemporality": 1,
                "dataPoints": [
                  {
                    "startTimeUnixNano": "1544712660300000000",
                    "timeUnixNano": "1544712660300000000",
                    "count": 2,
                    "sum": 2,
                    "bucketCounts": [1,1],
                    "explicitBounds": [1],
                    "min": 0,
                    "max": 2,
                    "attributes": [
                      {
                        "key": "my.histogram.attr",
                        "value": {
                          "stringValue": "some value"
                        }
                      }
                    ]
                  }
                ]
              }
            },
            {
              "name": "my.exponential.histogram",
              "unit": "1",
              "description": "I am an Exponential Histogram",
              "exponentialHistogram": {
                "aggregationTemporality": 1,
                "dataPoints": [
                  {
                    "startTimeUnixNano": "1544712660300000000",
                    "timeUnixNano": "1544712660300000000",
                    "count": 3,
                    "sum": 10,
                    "scale": 0,
                    "zeroCount": 1,
                    "positive": {
                      "offset": 1,
                      "bucketCounts": [0,2]
                    },
                    "min": 0,
                    "max": 5,
                    "zeroThreshold": 0,
                    "attributes": [
                      {
                        "key": "my.exponential.histogram.attr",
                        "value": {
                          "stringValue": "some value"
                        }
                      }
                    ]
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}

Logs

{
  "resourceLogs": [
    {
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {
              "stringValue": "my.service"
            }
          }
        ]
      },
      "scopeLogs": [
        {
          "scope": {
            "name": "my.library",
            "version": "1.0.0",
            "attributes": [
              {
                "key": "my.scope.attribute",
                "value": {
                  "stringValue": "some scope attribute"
                }
              }
            ]
          },
          "logRecords": [
            {
              "timeUnixNano": "1544712660300000000",
              "observedTimeUnixNano": "1544712660300000000",
              "severityNumber": 10,
              "severityText": "Information",
              "traceId": "5B8EFFF798038103D269B633813FC60C",
              "spanId": "EEE19B7EC3C1B174",
              "body": {
                "stringValue": "Example log record"
              },
              "attributes": [
                {
                  "key": "string.attribute",
                  "value": {
                    "stringValue": "some string"
                  }
                },
                {
                  "key": "boolean.attribute",
                  "value": {
                    "boolValue": true
                  }
                },
                {
                  "key": "int.attribute",
                  "value": {
                    "intValue": "10"
                  }
                },
                {
                  "key": "double.attribute",
                  "value": {
                    "doubleValue": 637.704
                  }
                },
                {
                  "key": "array.attribute",
                  "value": {
                    "arrayValue": {
                      "values": [
                        {
                          "stringValue": "many"
                        },
                        {
                          "stringValue": "values"
                        }
                      ]
                    }
                  }
                },
                {
                  "key": "map.attribute",
                  "value": {
                    "kvlistValue": {
                      "values": [
                        {
                          "key": "some.map.key",
                          "value": {
                            "stringValue": "some value"
                          }
                        }
                      ]
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

10

Set the Allow batch configuration option to true and fill in the following options to establish batch configuration:

• Sizer* - Decide whether to set the batches in bytes or items.

• Minimum size of the batch - Set the minimum capacity of the batch. The default value is 8192 and the minimum value is 0.

• Max size of the batch - Set the maximum capacity of the batch. It must be higher than the minimum set above. The default value is 8192 and the minimum value is 0.

• Time in milliseconds (ms) to send the batch - Enter the timeout to send the batch in milliseconds. The default value is 200 and the minimum value is 0.

11

Set Allow TLS configuration as true if you decide to include TLS authentication in this Listener:

• Add your TLS Certificate* from your Secrets or create one.

• Add your Private Key* from your Secrets or create one.

• Add your CA Chain* from your Secrets or create one.

• Select the Minimum TLS Version* from the menu.

12

The credentials are saved in Onum as Secrets. In the authentication form, click New secret to create a new one:

• Give the token a Name.

• Turn off the Expiration date option.

• Click Add new value.

• Click Save.

Learn more about secrets in Onum in this article.

You can now select the secret you just created in the Token/Password fields.

13

Authentication Configuration

This option will only appear if you choose HTTP in the Protocol option, or you choose gRPC as Protocol and set TLS Configuration.

Choose the authentication configuration for your request:

Authentication Type*

Choose your required authentication type:

• None - Choose this if you don't need any authentication.

• Basic - Enter your Username*, then choose your Password* from your Secrets or create one.

• Bearer token - Enter your Token name*, then choose your Token* from your Secrets or create one.

14

Test Configuration

Select Wait for result as true to test the connection to the destination by waiting for a response from the server.

Click Finish when complete. Your new Data sink will appear in the Data sinks area list.

Pipeline configuration

When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.

Output configuration

Parameters	Description
Event field to be sent*	Choose the field that contains the data to be sent.

Click Save to save your configuration.