WebsiteBlogLogin

Field Generator

Most recent version: v1.0.0

See the changelog of this Action type here.

Overview

The Field Generator action allows you to add new fields to your events using a given operation. You can select one or more operations to execute, and their resulting values will be set in user-defined event fields.

In order to configure this action, you must first link it to a Listener or another Action. Go to Building a Pipeline to learn how this works.

Ports

These are the input and output ports of this Action:

Input ports

  • Default port - All the events to be processed by this Action enter through this port.

Output ports

  • Default port - Events are sent through this port if no error occurs while processing them.

  • Error port - Events are sent through this port if an error occurs while processing them.

Configuration

1

Find Field Generator in the Actions tab (under the Advanced group) and drag it onto the canvas. Link it to the required Listener and Data sink.

2

To open the configuration, click the Action in the canvas and select Configuration.

3

Choose which operations you want to use to define the new fields in your events:

Operation
Parameters

Now

  • Now - Select true to create a new field with the current Epoch time in the selected time unit.

  • Now output field* - Give a name to the new field.

  • Now timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the Now time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

Today

  • Today - Select true to create a new field with the Epoch time corresponding to the current day at 00:00:00h in the selected time unit.

  • Today output field* - Give a name to the new field.

  • Today timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the Today time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

Yesterday

  • Yesterday - Select true to create a new field with the Epoch time corresponding to the previous day at 00:00:00h in the selected time unit.

  • Yesterday output field* - Give a name to the new field.

  • Yesterday timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the Yesterday time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

This Year

  • This Year - Select true to create a new field with the Epoch time corresponding to the previous day at 00:00:00h in the selected time unit.

  • This Year output field* - Give a name to the new field.

  • This Year timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the This Year time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

This Month

  • This Month - Select true to create a new field with the Epoch time corresponding to the previous day at 00:00:00h in the selected time unit.

  • This Month output field* - Give a name to the new field.

  • This Month timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the This Month time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

This Week

  • This Week - Select true to create a new field with the Epoch time corresponding to the previous day at 00:00:00h in the selected time unit.

  • This Week output field* - Give a name to the new field.

  • This Week timezone* - Enter the required timezone (for example: UTC, America/New_York, Europe/London...). The default value is UTC.

  • Output type* - Choose the type of your output dates:

    • Unix timestamp - This is the default value. Choose the required time unit in the This Week time unit* parameter that appears. The available time units are nanoseconds, microseconds, milliseconds & seconds.

    • Custom format - Enter a specific time format for your output dates in the Custom format* field that appears. For the complete valid format reference, see the gotime Nites Format Documentation.

Random number

  • Random number - Select true to create a new field with a random value.

  • Random output field* - Give a name to the new field.

Custom field

  • Allow custom field - Select true to create a new field with a custom value.

  • New custom field name* - Give a name to the new field.

  • Custom field value* - Set the value you want to add in the new field.

  • Custom field data type* - Choose the data type of the new field between integer, boolean, float or string.

4

Click Save to complete the process.

Example

Imagine we want to add a couple of new fields to our events. We want a new field that indicates the current Epoch time and another that adds the string Test in each event. To do it:

1

Add the Field Generator Action to your Pipeline and link it to your required Data sink.

2

Now, double-click the Field Generator Action to configure it. You need to set the following config:

Operation
Parameters

Now

  • Now - Set it to true.

  • Now output field - We're naming the new field Now.

  • Now time unit - Choose seconds.

Custom field

  • Allow custom field - Set it to true.

  • New custom field name - We're naming the new field Custom.

  • Custom field value - Enter Test.

  • Custom field data type - Choose string.

3

Left the rest of the parameters as default and click Save to apply the configuration.

4

Now link the Default output port of the Action to the input port of your Data sink.

5

Finally, click Publish and choose in which clusters you want to publish the Pipeline.

6

Click Test pipeline at the top of the area and choose a specific number of events to test if your data is transformed properly. Click Debug to proceed.

This is how your data will be transformed with the new fields:

PreviousBring Your Own CodeNextFor Each

Last updated

Was this helpful?