# Parser {% hint style="info" %} See the changelog of this Action type [here](/actions/parser.md). {% endhint %} ## Overview The **Parser** Action can be used to turn raw messages into a map of fields as required. The parser is configured with an expression based on a formal grammar, known as **PCL (Parser Configuration Language)**. Learn more about this language in [this article](/the-workspace/pipelines/actions/transformation/parser/pcl-parser-configuration-language.md). If your raw message is a CSV, a JSON, a KVL or an XML, the parser will detect it and will parse the output values accordingly. You can still make changes to get the required output parsing. {% hint style="warning" %} In order to configure this Action, you must first link it to a [Listener](/the-workspace/listeners.md). Go to [Building a Pipeline ](/the-workspace/pipelines/building-a-pipeline.md)to learn how to link. {% endhint %} ## Ports These are the input and output ports of this Action:
Input ports * **Default port** - All the events to be processed by this Action enter through this port.
Output ports * **Default port** - Events are sent through this port if no error occurs while processing them. * **Error port** - Events are sent through this port if an error occurs while processing them.
## Configuration {% stepper %} {% step %} Find **Parser** in the **Actions** tab (under the **Transformation** group) and drag it onto the canvas. {% endstep %} {% step %} To open the configuration, click the Action in the canvas and select **Configuration**. {% endstep %} {% step %} Enter the required parameters: #### Select field to parse First, choose the field to parse from your input data by typing it in the search bar or selecting it from the list. #### Input This is where you specify how to read the incoming data. Choose between:
ParameterDescription
Real data

This is the data taken directly from the linked Listener.

In the Events field at the right part, you can enter how many events to show in this window. You can pause/resume the log loading process or reset the logs shown to show the latest ones by using the icons next to this field.

Paste codeThere may be times when you will receive a file with updated data for the Listener. If this is the case, you can paste it here.
#### Parser Now that you have specified *where* to source the data from, you need to determine *how* to process the events. By default, the **Parser** section is displayed in the **List** view, where you'll see all the resulting fields in a vertical list. You can access the code view of the **Parser** by clicking the **\** icon at the top right corner, where you'll see the results written in the **PCL grammar**. Learn more about this language and how to modify your messages using it in [this article](/the-workspace/pipelines/actions/transformation/parser/pcl-parser-configuration-language.md). Now, choose between **Auto** and **Manual** mode. See what happens in each view: {% tabs %} {% tab title=" List view" %} Check what you'll see in the **Auto** and **Manual** modes of the parser:
ModeDescription
Auto

Automatically parses all fields from your input message. The parser will detect the structure of the message and will offer you a map of output fields with their corresponding names and types.

You can still modify the delimiters, field names/types, etc. if you need to do some changes. You'll be automatically switched to Manual mode when you do it.

Manual

Manually split fields from your input message and edit them as required. You can perform the following actions by clicking a field name tag:

  • Split fields - Select this option to split a field into several ones. Choose By all delimiters and select/enter the required delimiter to split the whole message as needed. You can also select By delimiter if you only need to split the message once (only the first delimiter in the message will be considered).
  • Extract fields manually - Use this option if you want to extract fields manually. Choose the type of the new field, its name, and the delimiter to be considered. Add as many fields as required and click Save.
  • Extract certain fields - By default, all the parsed fields will be visible in your output message. If you only need to display some of them, you can choose them using this action. Choose the fields you want to display and click Extract fields.
  • Hide fields - You can also hide fields from the output message by clicking the eye icon that appears when you hover over a field. Note that this will only appear if the parser auto detects one of the formats mentioned above (CSV, JSON, KVL or XML).
  • Extract all fields - This option will appear if you've hidden some fields. Use it if you want to display all fields again.
  • Change field types - You can change the type of a field by clicking the field name or checking the box that appears next to it. Click Change type and choose the required data type from the list of available ones. You can change types in bulk selecting all the required field boxes and then choosing the new type for all of them. the option will not be available if there's no type you can convert the field into. Learn more about the different data types in this article.
  • Edit field names - You can edit the name of a field by clicking the pencil icon in its name tag.
  • Ignore delimiters - Choose one or several delimiters and select by checking the boxes that appear next to them and click Ignore delimiters to delete them and join the fields that were separated by them. Remember to rename the resulting group(s) and choose the required data types.
  • Mark fields as optional - Optional groups allow you to define parts of a pattern that may or may not be present in your data, without causing a matching error when absent. To mark fields as optional, simply select them and click Mark as optional. You'll see they will be highlighted with a yellow frame and the prefix fieldGroup will be added to their names.
{% endtab %} {% tab title="Code view" %} In this mode, you'll see your input events written using the **PCL (Parser Configuration Language)** grammar, in both the **Auto** and **Manual** modes. Make any required modification and click **Apply changes** when you're done. We have provided an extensive run-down of each PCL command [in this article](/the-workspace/pipelines/actions/transformation/parser/pcl-parser-configuration-language.md). {% endtab %} {% endtabs %} #### Output Here you can see the output fields message that will be generated after parsing. Below, each individual field is [color-coded](/getting-started/understanding-the-essentials/data-types.md) according to the legend and separated into its type and name. You can see the number of new output fields at the top of this box. {% endstep %} {% step %} Click **Save** to complete the process. {% endstep %} {% endstepper %} ## Examples Select **Paste code** in the **Input** area and paste the following logs to see various parsing examples: {% tabs %} {% tab title="CSV example" %} Choose **Paste code** in the **Input** area and enter the following CSV: {% code title="CSV file" %} ```csv 2024-10-02T14:22:03Z,DESKTOP-1234,FileExecution,notepad.exe,4321,JohnDoe,192.168.1.2,a3b5c2d4e6f7,35 ``` {% endcode %} Then, in the **Parser** area, select **Auto** to automatically parser this data of the CSV into separate fields. The parser will automatically parse the log, having recognized the comma separators. The default values will be *fieldName1.1, fieldName1.2...* Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next Action. Change the names of each field: * `TIMESTAMP` * `HOSTNAME` * `EVENT_TYPE` * `PROCESS_NAME` * `PROCESS_ID` * `USERNAME` * `IP_ADDRESS` * `HASH` * `THREAT_SCORE`
Check if the resulting parsed fields are correct and click **Save**. {% endtab %} {% tab title="JSON example" %} Choose **Paste code** in the **Input** area and enter the following JSON: {% code title="JSON file" %} ```json {"timestamp":"2024-10-02T14:45:15Z","client_ip":"203.0.113.45","http_method":"GET","uri":"/login","response_code":403,"action":"BLOCK","rule_id":"981176","rule_description":"SQL Injection Attempt","user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36","headers":{"host":"example.com","referer":"","x_forwarded_for":"203.0.113.45"},"request_body":"","threat_details":{"attack_type":"SQL Injection","payload":"' OR 1=1 --"}} ``` {% endcode %} Then, in the **Parser** area, select **Auto** to automatically parser this data of the JSON into separate fields with their corresponding names and values. {% hint style="warning" %} As you can see in the **Output** area, field names are preceded by the JSON file name (`fieldName1.` by default). Edit the JSON name as you need to modify the output field names. {% endhint %}
Check if the resulting parsed fields are correct and click **Save**. {% endtab %} {% tab title="KVL example" %} Choose **Paste code** in the **Input** area and enter the following KVL: {% code title="KVL file" %} ``` timestamp=2024-10-02T15:00:45Z src_ip=192.168.1.100 dst_ip=203.0.113.45 action=ALLOW protocol=TCP src_port=443 dst_port=54321 bytes_sent=1024 bytes_received=2048 rule_id=1002 threat_level=low ``` {% endcode %} Then, in the **Parser** area, select **Auto** to automatically parser this data of the KVL into separate fields with their corresponding names and values. {% hint style="warning" %} As you can see in the **Output** area, field names are preceded by the KVL file name (`fieldName1.` by default). Edit the KVL name as you need to modify the output field names. {% endhint %}
Check if the resulting parsed fields are correct and click **Save**. {% endtab %} {% tab title="XML example" %} Choose **Paste code** in the **Input** area and enter the following XML: {% code title="XML file" %} ```xml 4658001280100x802000000000000030447483136SecurityS1128QRP038.ad.bbva.comS-1-5-18S1128QRP038$ADBBVA0x3e7Security0x28f40x566cD:\\APPSYS\\VERITAS\\NetBackup\\bin\\bpbkar32.exeThe handle to an object was closed. Subject : \tSecurity ID:\t\tS-1-5-18 \tAccount Name:\t\tS1128QRP038$ \tAccount Domain:\t\tADBBVA \tLogon ID:\t\t0x3E7 Object: \tObject Server:\t\tSecurity \tHandle ID:\t\t0x28f4 Process Information: \tProcess ID:\t\t0x566c \tProcess Name:\t\tD:\\APPSYS\\VERITAS\\NetBackup\\bin\\bpbkar32.exeInformationRegistryInfoSecurityMicrosoft Windows security auditing.Audit Success ``` {% endcode %} Then, in the **Parser** area, select **Auto** to automatically parser this data of the XML into separate fields with their corresponding names and values. {% hint style="warning" %} As you can see in the **Output** area, field names are preceded by the XML file name (`fieldName1.` by default). Edit the XML name as you need to modify the output field names. {% endhint %}
Check if the resulting parsed fields are correct and click **Save**. {% endtab %} {% endtabs %} --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.onum.com/the-workspace/pipelines/actions/transformation/parser.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.