Parser
Most recent version: v1.2.0
See the changelog of this Action type here.
Overview
The Parser Action can be used to turn raw messages into a map of fields as required. The parser is configured with an expression based on a formal grammar, known as PCL (Parser Configuration Language). Learn more about this language in this article.
If your raw message is a CSV, a JSON, a KVL or an XML, the parser will detect it and will parse the output values accordingly. You can still make changes to get the required output parsing.
In order to configure this Action, you must first link it to a Listener. Go to Building a Pipeline to learn how to link.
Ports
These are the input and output ports of this Action:
Output ports
Default port - Events are sent through this port if no error occurs while processing them.
Error port - Events are sent through this port if an error occurs while processing them.
Configuration
Find Parser in the Actions tab (under the Transformation group) and drag it onto the canvas.
To open the configuration, click the Action in the canvas and select Configuration.
Enter the required parameters:
Select field to parse
First, choose the field to parse from your input data by typing it in the search bar or selecting it from the list.
Input
This is where you specify how to read the incoming data. Choose between:
Real data
This is the data taken directly from the linked Listener.
In the Events field at the right part, you can enter how many events to show in this window. You can pause/resume the log loading process or reset the logs shown to show the latest ones by using the icons next to this field.
Paste code
There may be times when you will receive a file with updated data for the Listener. If this is the case, you can paste it here.
Parser
Now that you have specified where to source the data from, you need to determine how to process the events.
By default, the Parser section is displayed in the List view, where you'll see all the resulting fields in a vertical list. You can access the code view of the Parser by clicking the </> icon at the top right corner, where you'll see the results written in the PCL grammar. Learn more about this language and how to modify your messages using it in this article.
Now, choose between Auto and Manual mode. See what happens in each view:
Check what you'll see in the Auto and Manual modes of the parser:
Auto
Automatically parses all fields from your input message. The parser will detect the structure of the message and will offer you a map of output fields with their corresponding names and types.
You can still modify the delimiters, field names/types, etc. if you need to do some changes. You'll be automatically switched to Manual mode when you do it.
Manual
Manually split fields from your input message and edit them as required. You can perform the following actions by clicking a field name tag:
Split fields - Select this option to split a field into several ones. Choose By all delimiters and select/enter the required delimiter to split the whole message as needed. You can also select By delimiter if you only need to split the message once (only the first delimiter in the message will be considered).
Extract fields manually - Use this option if you want to extract fields manually. Choose the type of the new field, its name, and the delimiter to be considered. Add as many fields as required and click Save.
Extract certain fields - By default, all the parsed fields will be visible in your output message. If you only need to display some of them, you can choose them using this action. Choose the fields you want to display and click Extract fields.
Hide fields - You can also hide fields from the output message by clicking the eye icon that appears when you hover over a field. Note that this will only appear if the parser auto detects one of the formats mentioned above (CSV, JSON, KVL or XML).
Extract all fields - This option will appear if you've hidden some fields. Use it if you want to display all fields again.
Change field types - You can change the type of a field by clicking the field name or checking the box that appears next to it. Click Change type and choose the required data type from the list of available ones. You can change types in bulk selecting all the required field boxes and then choosing the new type for all of them. the option will not be available if there's no type you can convert the field into. Learn more about the different data types in this article.
Edit field names - You can edit the name of a field by clicking the pencil icon in its name tag.
Ignore delimiters - Choose one or several delimiters and select by checking the boxes that appear next to them and click Ignore delimiters to delete them and join the fields that were separated by them. Remember to rename the resulting group(s) and choose the required data types.
In this mode, you'll see your input events written using the PCL (Parser Configuration Language) grammar, in both the Auto and Manual modes. Make any required modification and click Apply changes when you're done.
We have provided an extensive run-down of each PCL command in this article.
Output
Here you can see the output fields message that will be generated after parsing. Below, each individual field is color-coded according to the legend and separated into its type and name.
You can see the number of new output fields at the top of this box.
Click Save to complete the process.
Examples
Select Paste code in the Input area and paste the following logs to see various parsing examples:
Choose Paste code in the Input area and enter the following CSV:
Then, in the Parser area, select Auto to automatically parser this data of the CSV into separate fields. The parser will automatically parse the log, having recognized the comma separators. The default values will be fieldName1.1, fieldName1.2...
Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next Action. Change the names of each field:
TIMESTAMPHOSTNAMEEVENT_TYPEPROCESS_NAMEPROCESS_IDUSERNAMEIP_ADDRESSHASHTHREAT_SCORE
Check if the resulting parsed fields are correct and click Save.
Choose Paste code in the Input area and enter the following JSON:
Then, in the Parser area, select Auto to automatically parser this data of the JSON into separate fields with their corresponding names and values.
As you can see in the Output area, field names are preceded by the JSON file name (fieldName1. by default). Edit the JSON name as you need to modify the output field names.
Check if the resulting parsed fields are correct and click Save.
Choose Paste code in the Input area and enter the following KVL:
Then, in the Parser area, select Auto to automatically parser this data of the KVL into separate fields with their corresponding names and values.
As you can see in the Output area, field names are preceded by the KVL file name (fieldName1. by default). Edit the KVL name as you need to modify the output field names.
Check if the resulting parsed fields are correct and click Save.
Choose Paste code in the Input area and enter the following XML:
Then, in the Parser area, select Auto to automatically parser this data of the XML into separate fields with their corresponding names and values.
As you can see in the Output area, field names are preceded by the XML file name (fieldName1. by default). Edit the XML name as you need to modify the output field names.
Check if the resulting parsed fields are correct and click Save.
Last updated
Was this helpful?