JSON Transformation

Most recent version: v0.0.1

See the changelog of this Action type here.

Overview

The JSON Transformation Action modifies your input JSON values using one of the given operations.

In order to configure this Action, you must first link it to a Listener. Go to Building a Pipeline to learn how to link.

Ports

These are the input and output ports of this Action:

Input ports

• Default port - All the events to be processed by this Action enter through this port.

Output ports

• Default port - Events are sent through this port if no error occurs while processing them.

• Error port - Events are sent through this port if an error occurs while processing them.

Configuration

1

Find JSON Transformation in the Actions tab (under the Transformation group) and drag it onto the canvas.

2

To open the configuration, click the Action in the canvas and select Configuration.

3

Enter the required parameters:

Parameter	Description
Select field to transform*	Select the field containing the JSON values from the incoming data. A sample input JSON is in the code box below.
Select operation*	Select the operation to perform on your JSON field: JQ - Performs a jq query on your input JSON values. Enter the following: Query Expression* - Enter the required jq expression. Note that the jq syntax has some limitations when you use it in this operation. Learn more in the note below. Flat JSON - Converts a nested JSON into a new, flattened JSON field with only one layer of key/value pairs. Enter the following: Flatten level* - Set the maximum number of levels to flatten to. Choose an option - Choose to separate values with _ or . (_ by default.
Output field*	Give your output field a name. You will ssee the transformation of the sample input JSON after applying the selected operation in the box below.

4

Click Save to complete the process.

JQ syntax limitations

Note that the JQ language has some limitations in Onum:

• The following jq operations are not supported: get_jq_origin, get_prog_origin, get_search_list , input_line_number and $__loc__

• The following jq flags are not supported: --ascii-output, -a, --seq, --sort-keys, -S and --unbuffered

• JSON extensions cannot be parsed (NaN, Infinity and [000]).

• Some regular expression metacharacters, back references, and look-around assertions are not supported.

• Byte-order mark (BOM) is not supported.

• Keywords cannot be used as function names.

• Module name prefixes cannot be used in function declarations.

Examples

Check below an example with the different JSON operations available:

JQ

Suppose you want to apply a specific jq query to the JSON values in your input events. In this case, we need to get only the names from the values in our JSONs.

Raw JSON

Your input data contains the following raw JSON file in the msg field:

[
   {
      "name":"Alice",
      "age":30
   },
   {
      "name":"Bob",
      "age":25
   },
   {
      "name":"Carol",
      "age":35
   }
]

Flat JSON

Add the JSON Transformation Action to the canvas and enter the following:

• Field to transform - msg

• Select operation - JQ

  • Query expression - jq '.[].name'

• Out field - JQvalue

Results

The output field JQvalue will return the results after applying the specified JQ query:

[
   "Alice",
   "Bob",
   "Carol"
]

Flat JSON

Suppose you wish to flatten a JSON value in your input events.

Raw JSON

Your input data contains the following raw JSON file in the msg field:

{
   "name":John,
   "age":30,
   "address":{
      "street":"123 Main St",
      "city":"Anytown",
      "country":"USA"
   },
   "contacts":[
      {
         "type":"email",
         "value":"[e-mail]"
      },
      {
         "type":"phone",
         "value":"555-1234"
      }
   ]
}

Flat JSON

Add the JSON Transformation Action to the canvas and enter the following:

• Field to transform - msg

• Select operation - FlatJSON

  • Flatten level - 5

  • Separator - .

• Out field - Flattened

Results

The output field Flattened will return the JSON value flattened to a maximum level-5 hierarchy:

{
   "name":John,
   "age":30,
   "address.street":"123 Main St",
   "address.city":"Anytown",
   "address.country":"USA",
   "contacts[0].type":"email",
   "contacts[0].value":"[e-mail]",
   "contacts[1].type":"phone",
   "contacts[1].value":"555-1234"
}