WebsiteBlogLogin

Send data using HTTP

Most recent version: v3.0.0

See the changelog of the HTTP Data sink type here.

Overview

Onum supports integration with HTTP.

HTTP, which stands for Hypertext Transfer Protocol, is a foundational protocol for communication on the World Wide Web. It defines how messages are formatted and transmitted between web servers and browsers, enabling the retrieval and display of webpages and other web content.

Onum Setup

1

Log in to your Onum tenant and click Data Sinks> New Data sink.

2

Double-click the HTTP Sink.

3

Enter a Name for the new Data Sink. Optionally, add a Description and some Tags to identify the Sink.

4

Decide whether or not to include this Data sink info in the metrics and graphs of the Home area.

Click Finish when complete. Your new Data sink will appear in the Data sinks area list.

Pipeline configuration

When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.

Output configuration

1

Add the HTTP Sink to your Pipeline and connect it to a Listener or Action to provide it with data.

2

Double-click the HTTP Sink.

3

Enter the HTTP method* used to connect to the server and the URL* to send HTTP requests. This feature supports dynamic URLs allowing you to select an input field to generate it in real-time.

4

Choose the field containing the Message you wish to send on.

5

Set the request Content type. Choose between:

  • text/plain - The message body contains plain text.

  • application/json - The message body is formatted as JSON. This is the default option.

  • application/xml - The message body is formatted as XML.

  • text/html - The message body contains HTML.

  • application/x-ndjson - The message body is formatted as NDJSON.

6

Decide whether to Support Special Characters and whether to use Gzip compression in the request.

7

Map the header key-value pairs to be included in the request.

If Authentication or Content-Type settings are set, they will override any headers entered here.

You can configure separate header-value pairs for Field headers and Secret Headers.

Secret Headers are intended for carrying sensitive, confidential, or security-critical information. Their values must be protected from unauthorized viewing.

8

HTTP bulk configuration refers to setting up an API or system to handle multiple requests or large amounts of data efficiently in a single operation. Instead of sending multiple individual HTTP requests, use the bulk settings to send a single request containing numerous actions.

For example, this sends three user records in one request instead of three separate requests.

POST /api/bulk/users HTTP/1.1
Content-Type: application/json

[
  {"id": 1, "name": "Alice"},
  {"id": 2, "name": "Bob"},
  {"id": 3, "name": "Charlie"}
]
Parameter
Description

Bulk allow*

Set true to set a bulk amount or false to ignore.

Delimiter*

Optionally, choose a delimiter to separate the individual records in the bulk data file. Choose between:

  • Manual delimiter - Each record will appear on its own line. This is the default option. Enter the required delimiter in the Manual delimiter field that appears. The default value is \n

  • Json list - Use a comma (,) to separate individual JSON objects.

Maximum number of buffers per server URL*

Multiple buffers can be assigned to a single URL to prevent excessive memory usage. The default and maximim value is 50.

Event amount*

Enter the bulk amount to allow through.

Event time limit*

If the bulk amount is not reached, enter the seconds to wait before sending through the accumulated events.

9

Choose the required Authentication type*:

  • None - Choose this if no authentication is required.

  • Basic - Enter the required Username and Password. Create a secret containing your password or select one already created.

  • Bearer - If you're using a bearer Token, create a secret containing it or select one already created.

  • API key - If you're using API credentials, enter your API key name and choose the required API key value. Create a secret containing your API key or select one already created.

  • OAuth2 - Choose if you want to set OAuth2 authentication. Set the following:

    • OAuth URL* - OAuth server URL.

    • OAuth method* - OAuth method used when sending requests to the server.

    • Send body as - Select how to send the request body to the server From URL encoded or JSON).

    • OAuth request body - Body that will be sent to auth server.

    • OAuth token path* - Path to token in response body from auth server.

    • Query key / Value pairs - Set of query keys and values. Use this also in case you are sending the request with From URL encoded content type.

    • Query key / Secret value pairs - Set of header keys and values that have been saved as Secrets.

10

Click New secret to create a new one:

  • Give the secret a Name.

  • Turn off the Expiration date option.

  • Click Add new value and paste the corresponding value.

  • Click Save.

Learn more about secrets in Onum in this article.

You can now select the created secrets in the configuration.

11

Activate the TLS Configuration toggle to enable TLS and configure these parameters:

  • Choose the Minimum TLS version* to use

  • Create a secret containing your TLS Certificate or select one already created

  • Create a secret containing your TLS Private key or select one already created

  • Activate or deactivate Skip TLS validations

  • Create a secret containing your CA Chain or select one already created.

12

If your organization uses proxy servers, establish the connection here:

  • Choose the field to set in the Proxy scheme (HTTP or HTTPS)

  • Username used in the proxy

  • Create a secret containing your proxy Password or select one already created

  • Proxy server URL and port.

13

Set a rate limit on the Maximum requests per second to launch on the server. The maximum value is 100.

14

Retry configuration

  • Max attempts - Maximum number of attempts before returning an error. The minimum value is 1, and the maximum value is 5.

  • Wait between attempts - The milliseconds to wait between retry attempts. The minimum value is 100, and the maximum value is 1000.

  • Backoff interval - A multiplication factor you may apply to the time established in the Wait between attempts parameter to increase the time to wait between attempts. For example, if the Wait between attempts is 100, and the Backoff interval is 2, the retry time will increase as follows each time: 100, 200, 400, 800... The minimum value is 1, and the maximum value is 5.

Click Save to save your configuration.

PreviousSend data via emailNextSend data to Elasticsearch

Last updated

Was this helpful?