WebsiteBlogLogin

Send data to MongoDB

Most recent version: v0.0.1

See the changelog of the Mongo DB sink type here.

Overview

Onum supports integration with MongoDB.

MongoDB is a document database used to build highly available and scalable internet applications. Instead of using tables, as in a traditional relational database, MongoDB stores data in JSON documents organized into collections.

Prerequisites

The MongoDB server or cluster must have its firewall configured to explicitly allow inbound traffic.

Make sure to have your connection string (URI) at the ready to establish the connection. This string typically includes:

  • The MongoDB protocol (mongodb+srv:// for Atlas or mongodb:// for self-managed).

  • The username and password.

  • The cluster/server address and port.

  • Any necessary query parameters (e.g., authentication method).

Onum Setup

1

Log in to your Onum tenant and click Data Sinks> New Data sink.

2

Double-click the MongoDB Sink.

3

Enter a Name for the new Data Sink. Optionally, add a Description and some Tags to identify the Sink.

4

Decide whether or not to include this Data sink info in the metrics and graphs of the Home area.

5

Authentication Method

Select one of the following options and complete the fields that appear to authenticate your connection:

SRV Connection String

Enter the Host* URL to connect to a MongoDB cluster using DNS-based seedlists. To get it:

  1. Log in to MongoDB.

  2. Choose the project that contains your cluster.

  3. Click on the Clusters tab in the left-hand sidebar.

  4. For the cluster you want to connect to, click the Connect button.

  5. Select Connect your application for the connection method.

  6. Copy the SRV Connection String, which should have the following format: 

mongodb+srv://:@cluster0.abcde.mongodb.net/myDatabase?retryWrites=true&w=majority

  • Replace username and password with your database credentials.

  • Replace myDatabase with your target database name.

Standard Connection String

Enter the required Host* and Port* for a standard MongoDB connection string (non-SRV). To get the host:

  1. Log in to MongoDB Atlas.

  2. Select your project and cluster.

  3. Click the Connect button for your cluster.

  4. Choose Connect your application.

  5. Click I don’t have a driver or scroll down to find the standard connection string option. You’ll see something like:

plaintextCopyEditmongodb://<username>:<password>@cluster0-shard-00-00.abcde.mongodb.net:27017,cluster0-shard-00-01.abcde.mongodb.net:27017,cluster0-shard-00-02.abcde.mongodb.net:27017/myDatabase?ssl=true&replicaSet=atlas-abcde-shard-0&authSource=admin&retryWrites=true&w=majority

  • Replace username and password with your database credentials.

  • Replace myDatabase with your target database name.

6

Authentication configuration

Choose your required authentication Type* from the dropdown menu. Leave as None if no authentication is needed, or select from the following:

Parameter
Description

SCRAM-SHA-256

  • Username* - Enter your required username.

  • Password* - Add your password from your Secrets or create one.

  • Auth source - This is the database where your credentials are stored. For example: mongodb://username:password@host:port/?authSource=admin

SCRAM-SHA-1

  • Username* - Enter your required username.

  • Password* - Add your password from your Secrets or create one.

  • Auth source - This is the database where your credentials are stored. For example: mongodb://username:password@host:port/?authSource=admin

x.509

  • Client certificate* - Add your client certificate from your Secrets or create one.

  • Client private key* - Add your private key from your Secrets or create one.

  • CA certificate* - Add your CA certificate from your Secrets or create one.

  • Auth source - This is the database where your credentials are stored. For example: mongodb://username:password@host:port/?authSource=admin

AWS IAM

  • Access key ID* - Add the access key ID from your Secrets or create one. The Access Key ID is found in the IAM Dashboard of the AWS Management Console. To get it:

    1. In the left panel, click on Users.

    2. Select your IAM user.

    3. Under the Security Credentials tab, scroll to Access Keys, and you will find existing Access Key IDs (but not the secret access key).

  • Access key* - Add the access key from your Secrets or create one. Under Access keys, you can see your Access Key IDs, but AWS will not show the Secret Access Key.

  • AWS session token - Add the AWS token from your Secrets or create one. To get it:

    1. Log in to the AWS Management Console via IAM Identity Center.

    2. Click your username in the top-right corner and choose Command line or programmatic access.

  • Auth source - This is the database where your credentials are stored. For example: mongodb://username:password@host:port/?authSource=admin

7

Click New secret to create a new one:

  • Give the secret a Name.

  • Turn off the Expiration date option.

  • Click Add new value and paste the corresponding value.

  • Click Save.

Learn more about secrets in Onum in this article.

You can now select the created secrets in the configuration.

8

Set the Connection pool settings

  • Max pool size - Set a limit on the number of events per pool. The minimum value is 0 and the maximum value is 10000. The default value is 100.

  • Min pool size - Set a minimum number of events per pool. The minimum value is 1 and the maximum value is 10000. The default value is 10.

  • Max idle time - Set the maximum amount of milliseconds a request can be idle before being considered timed out. The minimum value is 0 and the maximum value is 300000. The default value is 30000.

  • Max simultaneous connections - Enter a maximum allowance for the number of simultaneous connections. The minimum value is 1 and the maximum value is 1000. The default value is 100.

9

Toggle on the Bulk configuration switch button to enable bulk settings. The values will be filled in by default; however, you can change them if needed.

  • Number of events* Enter a maximum for the number of events in a bulk. The minimum value is 1 and the maximum value is 1000. The default value is 10.

  • Event time limit* Enter the number of seconds to wait before a bulk is triggered. The minimum value is 0 and the maximum value is 30000. The default value is 1.

  • Max number of buffers* Enter the maximum buffers that a system, application, or service is allowed to allocate or use for storing temporary data while processing it. The minimum value is 1 and the maximum value is 50. The default value is 50.

  • Ordered You can toggle on this switch button to send the buffers in order.

10

Toggle on the Bypass Document Validation switch button to allow MongoDB to skip any schema validation rules or constraints that would normally be enforced when inserting or updating a document.

Click Finish when complete. Your new Data sink will appear in the Data sinks area list.

Pipeline configuration

When it comes to using this Data sink in a Pipeline, you must configure the following output parameters. To do it, simply click the Data sink on the canvas and select Configuration.

Output configuration

Parameter
Description

Database*

Choose the input field that contains the name of the database you want to write to.

Collection*

Choose the input field that contains the name of the collection you want to write to.

Data*

Choose the input field that contains the data to be sent to MongoDB.

Click Save to save your configuration.

PreviousSend data to JiraNextSend data to PagerDuty

Last updated

Was this helpful?