Filter

Description

This operation allows you to filter lines of text based on specific conditions or patterns. This operation is useful when you want to keep only lines that match a certain pattern or exclude lines that don't meet specific criteria.

---

Data types

These are the input/output expected data types for this operation:

Input data

- The input strings to be split and filtered.

Output data

- The filtered results after applying the regular expression and delimiter.

---

Parameters

These are the parameters you need to configure to use this operation (mandatory parameters are marked with a *):

Delimiter*

Choose the delimiter that separates each segment in your input data:

• Line feed - Select this to have each line of text as a separate value.

• Space - Your segments are separated by spaces.

• Comma - Your segments are separated by commas (,)

• Semi-colon - Your segments are separated by semi-colons (;)

• Colon - Your segments are separated by colons (:)

• CRLF - Carriage return line feed (CRLF) is a control character that originally referred to moving the carriage on typewriters back to the starting position. In computing, it’s used in classic Mac OS and some Windows-based systems to mark the end of a line. If your input uses \r as the line-ending character, you can set the delimiter to \r to correctly separate values. For example: 100\r200\r300

Regex*

Define the pattern for filtering using a regular expression, which enables more complex matching conditions.

Invert

Set this parameter to true if you want to reverse the filter logic, so it keeps lines that do not match the specified pattern.

---

Example

Suppose you want to filter a series of strings to extract only segments that start with error:

1. In your Pipeline, open the required Action configuration and select the input Field.

2. In the Operation field, choose Filter.

3. Set Delimiter to Line feed

4. Set Regex to ^Error

5. Set Invert to false.

6. Give your Output field a name and click Save. The strings in your input field will be filtered with the specified conditions.

For example, this text:

Error: File not found
Warning: Low memory
Info: Operation completed
Error: Disk full

will be filtered as:

Error: File not found
Error: Disk full

You can try out operations with specific values using the Input field above the operation. You can enter the value in the example above and check the result in the Output field.