LogoLogo
WebsiteBlogLogin
  • Onum Docs
  • Use Cases
  • Videos
  • Release Notes
  • Use Cases
  • Artificial Intelligence
  • Classifying Anomalies in HTTP Requests
  • Using Amazon GenAI to classify HTTP logs
  • Enrichment
    • Data Enrichment
  • Reduction
    • Data Reduction & Optimization
  • OBSERVABILITY
  • Security
    • Firewall Threat Reduction
    • The Impossible Traveler
Powered by GitBook
On this page
  • Data collection and preparation
  • Data restructuring
  • Filter your data
  • Integrate with replicate models
  • Parser
  • Filter the model output
  • Data sink

Was this helpful?

Export as PDF

Classifying Anomalies in HTTP Requests

PreviousUse CasesNextUsing Amazon GenAI to classify HTTP logs

Last updated 1 month ago

Was this helpful?

Let's say you wish to route HTTP request logs to the appropriate location.

  • Normal logs are stored in S3.

  • Suspicious logs are classified and sent to the corresponding team for further analysis.

  • Anomaly Types:

    1. SQL Injection Attack

    2. Unauthorized Access Attempt

    3. Server Configuration Error

    4. Server Overload due to High Traffic

    5. Unknown

Data collection and preparation

Set up the corresponding Listener to the logs collected from HTTP servers, capturing details such as request methods, URLs, status codes, timestamps, and user-agent strings.

Data restructuring

Using the Parser action, logs are pre-screened and parsed to identify unusual status codes that may indicate potential anomalies.

Filter your data

Using the Conditional action, logs are filtered based on unusual status codes or patterns in request URLs.

Integrate with replicate models

Use the MLreplicate action to filter logs meeting the criteria and send them to public Llama3 model via an API from replicate.com. The model processes the text within these logs to classify the type of anomaly.

Logs meeting the filtering criteria are sent to Llama2 via an API from replicate.com. The model processes the text within these logs to classify the type of anomaly.

Llama3 evaluates each log entry against known patterns and characteristics of the four main categories of anomalies. It assigns a category to each log and provides a rationale based on textual evidence found in the log details.

Parser

Use the Parser action to provide the model output and restructure the data to make it a coherent message.

Filter the model output

Use the Conditional action to filter the model output message.

Data sink

Now we want to redirect our data to Slack channels using the Message builder action and then the Slack data sink.

Input

<134>1 2024-04-16T17:20:25Z 192.168.1.179 webserver - - - GET /api/data HTTP/1.1 500 587 - Mozilla/5.0 (Linux; Android 9; Pixel 3 XL) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.92 Mobile Safari/537.36 - Internal server error - high number of concurrent requests resulted in a failure to process all incoming requests.

output