# Data Reduction & Optimization Data on the same event is often produced by multiple devices, leading to staggering amounts of duplication and data bloat. You can reduce down data to what you really need and optimize it before sending it on. Filter your data to reduce what you send, remove incomplete or duplicated data, as well as tranform it into any format to match the requirements to make it more actionable. {% embed url="" %} *** ## The use case You receive data regarding firewall activity across an entire platform when you are only interested in threatening IPs. You can reduce unwanted data and send on only the required information. We will use the Parse to give structure and separate into fields. Use message builder to decide which to keep and send on. Let's do this together. Go to the **Pipelines** tab and select **New Pipeline**. Select the pencil icon to rename it *Reduction firewall.paloalto.threat* and click **Enter** to confirm.

### Listen Now, you'll need to find the Listener firewall.paloalto.threat from the list and drag it onto the middle canvas to add it to your Pipeline. This Listener provides information on all paloalto threat activity at the source. ### Reduce We will need to parse the data to separate out the fields to easily identify the desired information. Drag and drop the **Parser** and **Message Builder** actions from the Actions pane. Link the Listener to the Parser by dragging from the out port to the in port of the Parser.

*** #### Parser Click the **Parser** in the canvas and select **Configuration**.

First we must select the field to parse from the Listener in order to separate more specific data. This is the field containing the raw data.

Now we have decided which field, from where, and how to parse, we need to specify how it is output to the next action. Edit the field name. Click **Save**. *** #### Message Builder Click the **Message Builder** in the canvas and select **Configuration**.

This is where we define *what* the final message will be by selecting which fields to send on. Compose the message using [`;`](#user-content-fn-1)[^1] to join it to a coherent message to generate a CSV.

Click **Save**. Select the *out* port (as opposed to the *error* out port) of the Parser and link it to the **Message builder** in the same way. *** We have now successfully reduced the data from the listener to a concise message to be sent on to the end destination. ### Send Finally, we must tell Onum where and how to send the data. Drag the **Syslog** from the **Data sinks** tab and link the *out* output port of the **Message Builder** to the input port. Click **Publish**. Your data has now been optimized. [^1]: --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.onum.com/usecases/reduction/data-reduction-and-optimization.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.