# PCL (Parser Configuration Language)

## Introduction

PCL is a language designed to extract data from a line of text by describing its structure. The language aims to use a concise and intuitive syntax to help visualize the structure of a line of text. 

PCL expressions are used to configure the [**Parser** Action](https://docs.onum.com/the-workspace/pipelines/actions/transformation/parser).

## Syntax basics

A valid PCL expression must be composed of one or more fields, as long as there are separators between them. That is, it must follow this rule:

```
delimiter? fixedLength* field(delimiter fixedLength* field)* delimiter?
```

Where a `delimiter` could be a [literal](#syntax-literals) or an [operator](#syntax-operators). This last one could optionally have surrounding literals. 

When using groups, the PCL behaviour can change, as groups are a special type of field that can be written next to other fields without a delimiter. Check the **Group** section below to learn more about this.

{% hint style="warning" %}
At the moment, the only possible fixed-length field is a string.
{% endhint %}

**Valid example**

```
{myFieldOne:string} {myFieldTwo:int}<while(value=" ")>{myCsv:csv(fields=[0,2],separator=",")} 
```

**Invalid example (no delimiters)**

```
{myFieldOne:string}{myFieldTwo:int}
```

The grammar supports any kind of name that is written with the set of characters `A-Z`, `a-z`, `0-9` and the symbol underscore (`_`). It supports field aliases with any name written with the set of characters `A-Z`, `a-z`, `0-9`, `_`, `-`, `#` and `.` (given that the first character is not `_`).

## Syntax fields

In PCL, we can write any sequence of fields. The type of fields can be the following:

{% hint style="info" %}
Learn more about each field option in the [Field options](#field-options) section below.
{% endhint %}

<details>

<summary>CSV</summary>

`csv` is a configurable field. The available parameters are:&#x20;

<table><thead><tr><th width="169.51953125">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>indices</code> (optional)</td><td>Select which columns you want to extract from the CSV.</td></tr><tr><td><code>separator</code> (optional)</td><td>Define the separator of the columns.</td></tr><tr><td><code>totalColumns</code> (optional)</td><td>Indicate the number of columns of your CSV.</td></tr></tbody></table>

{% hint style="warning" %}
Note that the `totalColumns` parameter is mandatory when there is a delimiter after the field that is equal to the CSV separator. For example, a CSV with 3 columns and a JSON separated by a comma:&#x20;

`1,2,3,{"hello":"world"}`
{% endhint %}

**Examples**

{% code lineNumbers="true" %}

```
{myFieldName:csv(indices=[0,1,3],totalColumns=4,separator=",")}
{myFieldName:csv(indices=[0,1,3],totalColumns=4,separator=",", alias="newCsvName")}
{myFieldName:csv(indices=[0:string(alias="csvFieldName1"), 1:string(alias="csvFieldName2")], alias="newCsvNames")}
```

{% endcode %}

</details>

<details>

<summary>Float</summary>

`float` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.37890625">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>decimalSeparator</code> (optional)</td><td>Define the separator to the decimal.</td></tr><tr><td><code>thousandSeparator</code> (optional)</td><td>Define the separator to the thousands.</td></tr></tbody></table>

{% hint style="warning" %}
Note that the `decimalSeparator` and `thousandSeparator` parameters cannot contain the same value.
{% endhint %}

**Example**

{% code lineNumbers="true" %}

```
{myField:float(decimalSeparator=".")}
```

{% endcode %}

</details>

<details>

<summary>Group</summary>

A `group` is a special type that might contain two or more of the following simple types:

* `Float`
* `Integer`
* `Separator`
* `String`

Groups cannot be used inside other groups. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>optional</code> (optional)</td><td>Everything inside the group marked with this option could or not be in the log to parse.</td></tr></tbody></table>

Optionally, groups can have their type defined.

**Examples**

{% code lineNumbers="true" %}

```
{myGroupName:{{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}}}
{myGroupName:{_{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_}}
{myGroupName:group{_{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_}}
{myGroupName:group(optional=true){_{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_}}
```

{% endcode %}

The concatenation of simple types inside a group behaves in the same way as a normal PCL, always taking into account the restrictions of which types can be used.&#x20;

For a PCL containing a group field to be considered valid, the concatenation of fields/delimiters surrounding the group and the inner content of the group must form a valid PCL. Therefore, when unwrapping the inner PCL and joining it with the outside PCL, it must be valid. This means groups don't need to be separated from other fields by delimiters, as long as the resulting PCL is valid. This is because delimiters can be found at the start or end of a group.

For example, given the following valid PCL:

{% code lineNumbers="true" %}

```
{stringField:string}{myGroupName:group{_{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_}}
```

{% endcode %}

It could be a valid PCL as the concatenation will result in the following:

{% code lineNumbers="true" %}

```
{stringField:string}_{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_
```

{% endcode %}

However, the following PCL would be considered invalid:

{% code lineNumbers="true" %}

```
{stringField:string}{myGroupName:group{{myFieldTwo:int} {myFieldThree:int}_}}
```

{% endcode %}

As the result of the concatenation will result in two fields being together:

{% code lineNumbers="true" %}

```
{stringField:string}{myFieldOne:string} {myFieldTwo:int} {myFieldThree:int}_
```

{% endcode %}

This also applies when using the `optional` operator. The different PCLs generated containing or not the optional group must be valid. A valid example would be:

```
{stringField:string}{myGroupName:group(optional=true){_{myFieldOne:string}}}{myGroupName:group(optional=true){_{myFieldOne:string}}}
```

All the possible PCLs have their fields correctly separated by delimiters. On the other hand, if we had a PCL like the following, it would be considered invalid:

```
{stringField:string}{myGroupName:group(optional=true){_{myFieldOne:string}}}{myGroupName:group(optional=true){{myFieldOne:string}}}
```

As one of the possible PCLs would be:

```
{stringField:string}_{myFieldOne:string}{myFieldOne:string}
```

There are two fields together, which is not considered valid.

</details>

<details>

<summary>Integer</summary>

`int` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>thousandSeparator</code> (optional)</td><td>Define the separator to the thousands.</td></tr></tbody></table>

**Example**

{% code lineNumbers="true" %}

```
{myFieldName:int(thousandSeparator=",")}
```

{% endcode %}

</details>

<details>

<summary>JSON</summary>

`json` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>fields</code> (optional)</td><td>Select which items you want to extract from the JSON.</td></tr></tbody></table>

**Examples**

{% code lineNumbers="true" %}

```
{myFieldName:json(fields=["itemOne","itemTwo"])}
{myfield:json(fields=["hello ":string(alias="hello_"), "bye ":string(alias="bye_")])}
```

{% endcode %}

</details>

<details>

<summary>Key-value list</summary>

`keyValueList` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>kvSeparator</code> (optional)</td><td>Define the separator between keys and values. </td></tr><tr><td><code>listSeparator</code> (optional)</td><td>Define the separator between each key-value item.</td></tr><tr><td><code>indices</code> (optional)</td><td>Select which columns you want to extract from the list by their position in the list.</td></tr><tr><td><code>fields</code> (optional)</td><td>Select which items you want to extract from the list by their key names.</td></tr></tbody></table>

{% hint style="warning" %}
The `indices` and `fields` operators cannot be used simultaneously.
{% endhint %}

**Examples**

{% code lineNumbers="true" %}

```
{myFieldName:keyValueList(kvSeparator=":",listSeparator=",")}
{myFieldName:keyValueList(fields=["hello ":string(alias="hello_")
```

{% endcode %}

{% hint style="info" %}
if you need to parse a list of key-values where there are some keys duplicated, check the example in [this section](#key-value-list-with-duplicated-keys).
{% endhint %}

</details>

<details>

<summary>String</summary>

`string` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>length</code> (optional)</td><td>Define the length of the string.</td></tr><tr><td><code>escapableChar</code> (optional)</td><td>Escape delimiter characters in the string.</td></tr></tbody></table>

**Examples**

{% code lineNumbers="true" %}

```
{myField:string(length=2)}
{myField:string(length=2, alias="newFieldName")}
```

{% endcode %}

</details>

<details>

<summary>XML</summary>

`XML` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>xpaths</code> (optional)</td><td>Select which items you want to extract from the XML using a subset of the <a href="https://www.w3schools.com/Xml/xpath_intro.asp">XPath</a> query language.</td></tr></tbody></table>

**Examples**

{% code lineNumbers="true" %}

```
{myFieldName:xml(xpaths=["/data/event","/data/event@id"])}
{myfield:xml(xpaths=["/data/name":string(alias="name"), "/data/event":listString])}
```

{% endcode %}

</details>

There's also some subtypes:

<details>

<summary>Boolean</summary>

`bool` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr></tbody></table>

**Example**

```
{myField:bool(alias="newFieldName")}
```

</details>

<details>

<summary>Boolean list</summary>

`listBool` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr></tbody></table>

**Example**

```
{myField:listBool(alias="newFieldName")}
```

</details>

<details>

<summary>Integer list</summary>

`listInt` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr></tbody></table>

**Example**

```
{myField:listInt(alias="newFieldName")}
```

</details>

<details>

<summary>Float list</summary>

`listFloat` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr></tbody></table>

**Example**

```
{myField:listFloat(alias="newFieldName")}
```

</details>

<details>

<summary>String list</summary>

`listString` is a configurable field. The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr></tbody></table>

**Example**

```
{myField:listString(alias="newFieldName")}
```

**Special case: fixed-length strings**

There is a special case with `String` fields: if the field is using the option `length`, you may add another field very next to it with no separators:

```
{oneField:string(length=2)}{anotherField:string(length=3)}{lastField:string}
```

</details>

<details>

<summary>Map</summary>

{% hint style="warning" %}
Note that this field can only be as a subtype in Key-value lists.
{% endhint %}

The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>alias</code> (optional)</td><td>Rename the field name.</td></tr><tr><td><code>default</code> (optional)</td><td>Default value to be used when the element is not found.</td></tr><tr><td><code>key</code> (mandatory)</td><td>XPath from where to extract the <strong>key</strong> for the map. This must be an XPath relative to the parent element selected by the XPath you're configuring.</td></tr><tr><td><code>outputFormat</code> (mandatory)</td><td>Format of the map to be stored in the event.</td></tr><tr><td><code>value</code> (optional)</td><td>XPath from where to extract the <strong>value</strong> for the map. This must be an XPath relative to the parent element selected by the configured you're configuring. By default, the value is the text inside the element selected in your XPath.</td></tr></tbody></table>

</details>

## Syntax literals

A literal is a special type of element. This is a string that must exist between two fields. Unlike other fields, the literals are just a string that may contain one or more characters except `<`, `>`, `{,` or `}`, unless they are escaped with `\`

This is an example of a literal (whitespace  ):

{% code lineNumbers="true" %}

```
 {myFieldOne:string} {myFieldTwo:string}
```

{% endcode %}

## Syntax operators

There are two types of operators:

<details>

<summary>Skip</summary>

The `Skip` operator acts like a dynamic separator. It can be used when we want to skip any content until we find a coincidence.&#x20;

This is a configurable operator that is equivalent to the regular expression `(?:from)*(?=to)` where `from` and `to` are the strings to match.&#x20;

The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>from</code> (mandatory)</td><td>Define the string to find one or more times.</td></tr><tr><td><code>to</code> (mandatory)</td><td>Define the string to insert one or more times.</td></tr></tbody></table>

**Example**

{% code lineNumbers="true" %}

```
<skip(from=" ",to="-")>
```

{% endcode %}

A use case could be to skip all characters until a JSON is found. For example, for this log:

{% code lineNumbers="true" %}

```
hello thisisrubbish{"my": "json"}
```

{% endcode %}

We could use this PCL:

{% code lineNumbers="true" %}

```
{f1:string}<skip(from=" ", to="{")> {f2:json}
```

{% endcode %}

</details>

<details>

<summary>While</summary>

The `While` operator acts like a dynamic separator. It is useful if a separator has an unknown number of repetitions on each log.

This is a configurable operator that is equivalent to the regular expression `(?:value)*` where `value` is the string to match. However, if the options `min` and/or `max` are defined, then the equivalent regular expression is `(?:value)*{- min,max}`

The available parameters are:

<table><thead><tr><th width="170.1875">Parameter</th><th>Description</th></tr></thead><tbody><tr><td><code>value</code>  (mandatory)</td><td>Define the string to find one or more times.</td></tr><tr><td><code>max</code>  (optional)</td><td>Set the maximum number of repetitions of <code>value</code> (must be greater than 0).</td></tr><tr><td><code>min</code>  (optional)</td><td>Set the minimum number of repetitions of <code>value</code> (must be greater than 0).</td></tr></tbody></table>

It is not necessary to define both `max` and `min`. However, if both are defined, then it must assert that `min` is strictly lower than `max`, that is, `min < max`.&#x20;

```
<while(value=" ",max=2)>
```

In another example, let  `-` as a separator that appears at least 3 times in all logs:

* `hello - - -world`
* `goodbye - - - - -world`
* `hello - - - -moon`

Then, the PCL could be:

```
 {f1:string}<while(value=" -", min=3)>{f2:string}
```

</details>

## Field options

<details>

<summary>alias</summary>

The value must follow the naming requirements:

* The allowed set of characters is: `A-Z`, `a-z`, `0-9`, `.`, `-`, `_` or `#`.
* An alias cannot start with `_`.

These are valid examples:

* `alias="myNewName"`
* `alias="my-new-name"`&#x20;

These are invalid examples:

* `alias="_myNewName"`
* `alias="my new name"`

</details>

<details>

<summary>decimalSeparator</summary>

The value could be:

* `,`
* `.`  (default value)

These are valid examples:

* `decimalSeparator=","`
* `decimalSeparator="."`&#x20;

These are invalid examples:&#x20;

* `decimalSeparator=""`
* `decimalSeparator="-"`
* `decimalSeparator="_"`

</details>

<details>

<summary>default</summary>

The value must be of the same type as the parent. For example, if it is the *default value* of an integer, then the *default value* must be an integer too.&#x20;

These are valid examples:

* `{myfield:json(fields=["hello":string(default="{}")])}`
* `{myfield:csv(indices=[0:int(default=-1)])}`
* `{myfield:xml(xpaths=["/data/event":listString(default=["one","two"],alias="myField")])}`
* `{myfield:xml(xpaths=["/data/event":listInt(default=[1,2],alias="myField")])}`

These are invalid examples:

* `{myfield:json(fields=["hello":string(default=-1)])}`
* `{myfield:float(default=1.5)}`

</details>

<details>

<summary>escapableChar</summary>

The value must be an ASCII char.

These are valid examples:

* `escapableChar="|"`
* `escapableChar="\"`
* `escapableChar=">"`

These are invalid examples:

* `escapableChar=""`
* `escapableChar="hello"`
* `escapableChar=;`

</details>

<details>

<summary>fields</summary>

The value must be a list of strings. Note that the list **cannot** contain other values (e.g. numbers).&#x20;

Additionally, we may specify the type of each field by writing a colon (`:`) followed by the type:&#x20;

* `bool`
* `float`
* `int`&#x20;
* `string`
* `listBool` (only for Key-value lists)
* `listFloat` (only for Key-value lists)
* `listInt` (only for Key-value lists)
* `listString` (only for Key-value lists)

For example: `fields=["oneField":bool, "middleField", "anotherField":int]`. If the type is omitted, it should be assumed that the type is `string`. In the previous example, it assumes that `middleField` is a string.

Each sub-type may have these parameters:

* `alias` (optional) to rename the field name.
* `default` (optional) to set a fixed value if the field does not exist in the log.&#x20;

These are valid examples:

* `fields=["oneField","anotherField.with.subField"]`
* `fields=["oneField":string(alias="anotherName")]`
* `fields=[]`&#x20;

These are invalid examples:

* `fields=[oneField,anotherField]`
* `fields=["oneField,anotherField"]`
* `fields=[0,1]`

</details>

<details>

<summary>indices</summary>

The value must be a list with numbers. Note that the list **cannot** contain other values apart from positive integers (including zero).&#x20;

Additionally, we may specify the type of each index by writting a colon (`:`) followed by the type: `bool`, `float`, `int` or `string`. For example: `indices=[0:bool, 1, 3:int]`. If the type is omitted, it assumes that the type is `string`. In the previous example, it assumes that `1` is a string.

Each sub-type may have these parameters:

* `alias` (optional) to rename the field name.
* `default` (optional) to set a fixed value if the field does not exist in the log.&#x20;

These are valid examples:

* `indices=[0,1,3]`
* `indices=[1:string(default="not exists")]`
* `indices=[]`&#x20;

These are invalid examples:

* `indices=["0","1"]`
* `indices=[-3,1]`

</details>

<details>

<summary>key</summary>

XPath to extract the keys for a map. This XPath is relative to the parent node and supports the same subset of XPath described in the [XPaths section](#xpaths). Note that the value extracted for the keys is always a string.

These are valid examples:

* `key="/node"`
* `key="/@attribute"`

These are invalid examples:

* `key="//"`
* `key=""`
* `key="/node":float`

</details>

<details>

<summary>kvSeparator</summary>

The value can be as long as needed, there is no character limit. By default, it is `=`.

These are valid examples:

* `kvSeparator=":"`
* `kvSeparator="\t"`
* `kvSeparator="hello"`

These are invalid examples:

* `kvSeparator=""`
* `kvSeparator=:`

Note that `"` must be escaped. For example: `kvSeparator="\""`

</details>

<details>

<summary>length</summary>

The value must be a strictly positive integer.&#x20;

These are valid examples:

* `length=1`
* `length=25`&#x20;

These are invalid examples:

* `length="1"`
* `length=0`
* `length=-3`

</details>

<details>

<summary>listSeparator</summary>

The value must be a non-empty text. By default, it is `,`

These are valid examples:

* `listSeparator=";"`
* `listSeparator="|"`&#x20;
* `listSeparator="hello"`

These are invalid examples:&#x20;

* `listSeparator=""`
* `listSeparator=;`

Note that `"` must be escaped. For example: `listSeparator="\""`.

</details>

<details>

<summary>optionalField</summary>

The value must be a boolean. Therefore, it could be:

* `true`
* `false`

These are the valid options:

* `optional=true`
* `optional=false`

These are invalid options:

* `optional=john`
* `optional=doe`

Currently, this option is only available for `group` fields.

</details>

<details>

<summary>outputFormat</summary>

Sets the output format with which a map is stored in the event. At the moment, the only supported value for this configuration parameter is `json`.

{% hint style="warning" %}
This configuration parameter is mandatory until maps are supported natively by the ActionSDK.
{% endhint %}

These are valid examples:

* `outputFormat="json"`

These are invalid examples:

* `outputFormat=""`
* `outputFormat="xml"`
* `outputFormat="something"`

</details>

<details>

<summary>separator</summary>

The value must be a character from the set: `|`, `;`, `,`, `\t`. By default, it is `,`.

These are valid examples:

* `separator=";"`
* `separator="\t"`

These are invalid examples:

* `separator="-"`
* `separator=;`

</details>

<details>

<summary>thousandSeparator</summary>

The value could be:

* empty string (default value).
* `,`
* `.`

These are valid examples:

* `thousandSeparator=""`
* `thousandSeparator="."`

These are invalid examples:

* `thousandSeparator="-"`
* `thousandSeparator="_"`

</details>

<details>

<summary>totalColumns</summary>

The value must be a strictly positive integer.

These are valid examples:

* `totalColumns=1`
* `totalColumns=5`

A valid value for this option must equal the number of columns in the CSV.

These are invalid examples:

* `totalColumns="1"`
* `totalColumns=0`
* `totalColumns=-3`

</details>

<details>

<summary>value</summary>

XPath to extract the values for a map. This XPath is relative to the parent node and supports the same subset of XPath described in the [XPaths section](#xpaths).

These are valid examples:

* `value="/node"`
* `value="/@attribute"`

These are invalid examples:

* `value="//"`
* `value=""`

</details>

<details>

<summary>xpaths</summary>

This value must be a list of valid XPath strings with some limitations. PCL supports a subset of XPath expressions: only direct paths are supported. Things like predicates, wildcards and `//` selectors aren't supported.

Supported ✅

```
/data
data
/data/event
/data/event/@id
/data/event/site
```

Unsupported ❌

```
/data//site
//@id
/data/event[last()]
/data/event[@id]
/data/*
```

Additionally, we may specify the type of each field by writing a colon (`:`) followed by a type. Currently, supported types for XPath are:

* `bool`
* `float`
* `int.`
* `string`
* `listBool`
* `listFloat`
* `listInt`
* `listString`
* `map`

By default, XPath fields are of type `string`.

</details>

## Examples

<details>

<summary>CSV with 3 columns</summary>

We have the following log:

```csv
foo|bar|\"foo|bar\"|{"hello": "world"}
```

A valid expression to parse the message is:

```
{fieldName:csv(separator="|",totalColumns=3)}|{fieldName2:json()}
```

or

```
{csvField:csv(separator="|",indices=[0,1,2],totalColumns=3)}|{stringField:string}
```

</details>

<details>

<summary>Key-value list with duplicated keys</summary>

We have the following log:

```
key1=value1 key2=value2 key3=3 key1=anotherValue1
```

A valid expression to parse the message is:

```
{field:keyValueList(kvSeparator="=", listSeparator=" ", fields=["key1":listString(alias="key1AsList"), "key2":string(), "key3":int()])}
```

And it would extract these values:

```
field: "key1=value1 key2=value2 key3=3 key1=anotherValue1"
key1AsList:
- value1
- anotherValue1
field.key2: "value2"
key3: 3
```

</details>

<details>

<summary>Unknown repetitions</summary>

A valid PCL expression could be:

```
{lastName:string}, {firstName:string}<while(value=" ")>{age:int}: {info:json(fields=["country","occupation"])}
```

Here, there are 4 fields (`lastName`, `firstName`, `age` and `info`) and 3 delimiters (`,` , `:` and the operator `while`).

This PCL expression can be used to extract fields from different lines of text that have the same structure. For example, given the text `Doe, John 37: {"country": "UK", "occupation": "father"}`, the PCL expression can be used to extract the following fields:

```
lastName: "Doe"
firstName: "John"
age: 37
info: "{\"country\": \"UK\", \"occupation\": \"father\"}"
info.country: "UK"
info.occupation: "father"
```

In another example, given the text `Smith, Jane 19: {"country": "USA", "occupation": "student"}`, the same PCL expression would extract:

```
lastName: "Smith"
firstName: "Jane"
age: 19
info: "{\"country\": \"USA\", \"occupation\": \"student\"}"
info.country: "USA"
info.occupation: "student"
```

</details>

<details>

<summary>String after a CSV</summary>

We have the following log:

```
foo|bar|"foo|bar"|another field after the CSV
```

A valid expression to parse the message is:

```
{fieldName1:csv(separator="|",totalColumns=3)}|{fieldName2:string}
```

or

```
{csvField:csv(separator="|",indices=[0,1,2],totalColumns=3)}|{stringField:string}
```

</details>

<details>

<summary>Syslog message</summary>

We have the following log:

```
<165>1 2003-10-11T22:14:15.003Z myhostname myapp 1234 ID47 - An application event log entry...
```

A valid PCL expression to parse this log would be

```
<while(value=" ",max=2)>\<{priority:string}\>{version:int} {eventtimestamp:string} {hostname:string} {appName:string} {procId:string} {msgId:string} - {msg:string}
```

</details>

<details>

<summary>XML that contains several metadata values</summary>

We have the following log:

```xml
<event>
  <date timezone="UTC">2025-02-27</date>
  <metadata name="ProcId">1</metadata>
  <metadata name="UserId">123</metadata>
  <log>user logged in</log>
  <log>user update registry</log>
  <log>user logged out</log>
</event>
```

A valid PCL expression to parse this log would be:

```
{field:xml(xpaths=["/event/date","/event/date@timezone","/event/metadata":map(key="/@name",outputFormat="json"),"/event/log":listString])}
```

This would extract the following data:

```
field: "..." # Full XML here
field.event.date: "2025-02-27"
field.event.date#timezone: "UTC"
field.event.metadata: "{\"ProcId\":\"1\",\"UserId\":\"123\"}"
field.event.log:
  - "user logged in"
  - "user updated a registry"
  - "user logged out"
```

</details>

<details>

<summary>XML that contains a list</summary>

We have the following log:

```xml
<data>
  <log>user logged in</log>
  <log>user updated a registry</log>
  <log>user logged out</log>
</data>
```

A list can be extracted with the following PCL expression:

```
{field:xml(xpaths=["/data/log":listString)])}
```

And it would extract the following information:

```
field: "..." # Full XML here
field.data.log:
  - "user logged in"
  - "user updated a registry"
  - "user logged out"
```

</details>

<details>

<summary>XML that contains a list of objects</summary>

We have the following log:

```xml
<event>
  <metadata>
    <field>procId</field>
    <value>1234</value>
  </metadata>
  <metadata>
    <field>userId</field>
    <value>4321</value>
  </metadata>
  <metadata>
    <field>message</field>
    <value>hello!</value>
  </metadata>
</event>
```

A valid PCL expression to parse this log would be:

```
{field:xml(xpaths=["/event/metadata":map(key="/field",value="/value",outputFormat="json")])}
```

This would extract the following data:

```
field: "..." # Full XML here
field.event.metadata: "{\"procId\":\"1234\",\"userId\":\"4321\",\"message\":\"hello!\"}"
```

</details>