Stream Rules

Overview

Stream rules evaluate normalized logs during ingestion to immediately detect anomalies or threat activity. Security operators manage rules in Policies > Stream Rules, and detection results can flow into follow-up features such as Event Summary, Tickets, Explanations, and Playbooks.

For example, stream rules can immediately detect events that match specific patterns in IPS, EDR, or firewall logs, or apply multi-step conditions based on normalized field values to quickly identify data exfiltration, C&C communication, or abnormal behavior. Administrators can add, edit, and delete rules and change their enabled state, and operators can review detection policies based on the rule list and status. In Sonar 5, you can link rules to MITRE ATT&CK techniques and tactics to manage detection objectives in a more structured way.

The following diagram shows how a stream rule operates. Stream rules apply to data that has been normalized during the ingestion phase. When you need to enter a field name while writing a stream rule, use the field names normalized by the log schema.

A stream rule performs the following key actions:

• Event detection: Match the detection library against normalized field values.
• Response actions:
  • Issue tickets and send alerts for detected events.
  • Request an explanation if the actor is an employee.
  • Register the IP address associated with the event in an address group.
  • If necessary, send a command to the connected device to block the related traffic.
  • Trigger playbook execution.

In most cases, stream rules issue tickets on event detection or request an explanation from the actor. Excessive ticket creation increases the workload of malware analysts and security operators, so you should minimize duplicate tickets. Logpresso Sonar consolidates events with the same content into a single ticket based on the alert message, ticket subject, or specified duplicate-detection fields.

Search stream rules

You can view or search the list of stream rules in Policies > Stream Rules.

• App: Icon indicating the app associated with the rule. Default rules and user-added rules show the Logpresso icon, and rules installed from apps show the app icon.
• Enabled: Toggle button to enable/disable the rule (: enabled, : disabled)
• Status: Column that shows the rule's current application status.
• Priority: Rule priority (high, medium, low)
• Category: Category assigned to the rule
• Scenario: Name of the stream rule
• Type: Log schema configured for the rule
• Ticket: Whether ticket issuance is configured
• Explanation: Whether explanation request is configured
• Modified At: Date the rule was last modified (or created)

To find a specific stream rule in the list, use the search tool in the toolbar. Enter a keyword to filter the list.

Use the Category, Enabled, Ticket, and Explanation filters in the toolbar to quickly narrow down the rules you need.

Enable/disable a rule

To enable or disable a stream rule, click the toggle button in the Enabled column of the rule (: enabled, : disabled).

Download the list

To save the stream rule list to your local PC, click in the toolbar, then select the file format you want.

Refresh the list

To refresh the stream rule list with the latest data, click in the toolbar.

Set acceptable risk

In environments where the XDR license is activated, you can set an acceptable risk score for a stream rule to estimate the expected number of tickets.

To set acceptable risk:

1. Select the checkbox of the rule you want to configure in the stream rule list.

2. Click in the toolbar, then click Set Acceptable Risk from the menu that appears.

   

3. In the Set Acceptable Risk dialog, enter the Acceptable Risk Score (1–100) and click Estimate Ticket Count to see the estimated daily ticket count. Click Save to apply, or Cancel to discard.

   

Add a stream rule

To add a stream rule:

1. In Policies > Stream Rules, click Add in the toolbar.

2. On the Add Stream Rule screen, configure the basic settings.

   

   • Name: A unique name for the rule (up to 255 characters). Duplicate names are not allowed.

   • Category: The category for the rule. Categories appear in the Category column on the Analysis > Event Summary screen. You can pre-add categories in Rule Categories.

   • Message: The subject assigned to tickets when an event is detected.

     • Enter a field name defined in the log schema as $fieldname or $fieldname$ to pull the field value from the log that triggered detection (e.g., $src_ip). If the field name contains non-ASCII characters, use the $fieldname$ format.
     • If no Duplicate Key Field is set under Advanced Settings, duplicate tickets are suppressed based on whether the generated message (up to 2,000 characters) is identical.

   • Assignee: The user account to assign issued tickets to. You can select one or more accounts. If none is specified, tickets are created without an assignee.

   • Alarm Group: The alarm group to notify when an event is detected.

     Note

     Configure how alarms are received in the alarm group settings.
     

     

   • Signature App: Select the app associated with this rule. Selecting an app makes it easier to identify the source and context of the rule.

   • MITRE Technique: Search for and select one or more MITRE ATT&CK techniques that this rule is designed to detect. You can search by technique ID (e.g., T1078) or technique name. Selected items appear as tags.

   • MITRE Tactic: Search for and select the MITRE ATT&CK tactic this rule addresses. You can search by tactic ID (e.g., TA0001) or tactic name.

   • Description: A detailed description of the rule (up to 2,000 characters).

3. To separately manage the attacker's IP address or the abnormal actor's IP address upon event detection, or to pass address group information to a firewall or IPS, configure the following settings:

   

   • Address Group: The address group to which the value (IP address) of the IP Block Field is added.
   • Block Duration: How long (in minutes) to keep the IP address in the Address Group. The IP address is automatically removed from the address group after the block duration expires. If not specified, the IP address is retained indefinitely (range: 1–100,000,000).
   • IP Block Field: The IP address field to add to the address group when an event is detected. Enter the IP address-related field name defined in the log schema (e.g., src_ip, dst_ip) (up to 50 characters).
     • For block integration settings with third-party security systems, refer to Response Targets.

4. Set the priority of detected events and select the logger model or logger to apply the stream rule to.

   

   • Priority: Priority of detected events (high, medium, low)
   • Log Schema: Select the log schema to apply to the rule. The schema determines which options are available in the scenario builder.
   • Detection Target: Select the data logger model or logger from the list to apply the rule to. You cannot select both a logger model and a logger at the same time — choose one or the other. The list shows only logger models and loggers that reference the selected Log Schema.
   Note

   If a logger model has only "raw" normalization rules, it does not reference any log schema and will not appear in the detection target list. Make sure to define normalization rules that reference a log schema when configuring the logger model.
   

5. Write the threat detection rule. You can write the rule using the Scenario Builder or by entering a query directly in the Query field. For details on the scenario builder, refer to Rules and parameters by field type.

   A rule consists of at least one condition, applied in order from top to bottom. When data arrives through the Detection Target selected in step 4, conditions are evaluated in order. A subsequent condition is only evaluated if the preceding one is satisfied. The execution order can affect system performance. Set simple, fast conditions at the top to filter out irrelevant logs as much as possible.

   If you enter an invalid or non-executable query (e.g., referencing a nonexistent address group or pattern group), the rule cannot be added.

   

   • Scenario Builder: A detection rule editor composed of Condition, Field, and Rule. After selecting all options, click Add to add the rule to the list. You can add one or more rules.
     • Condition: The condition for applying the rule (AND, NOT; default: AND). The next rule is applied when the comparison between the data and this rule evaluates to true.
       • AND: True when matched (true)
       • NOT: True when not matched (true)
     • Field: The field defined in the log schema to compare against the Rule.
     • Rule: The rule to apply to the Field. Shows a list of applicable rules based on the Field data type.
   • Query: Enter a query to add filter rules directly (up to 10,000 characters).
   • Stream Query: Displays the filter rule entered in the Scenario Builder or Query as a stream query.

6. If detected events are related to an employee in the organization, you can request an explanation from that employee.

   

   • Request Explanation: Whether to send an explanation request (default: disabled).

   • Explanation Target Field: The field containing the employee ID of the employee to explain.

     When a threat is detected, an explanation request email is automatically sent to the employee mapped to that ID. The employee information (ID, account, email) must be configured in advance under Users > Employees.

   • Category: Category of the explanation. If needed, click Manage Explanation Categories in Response > Explanations to add categories.

   • First Reviewer, Second Reviewer: The first and second reviewer of the explanation (defaults to department head if not set). Refer to Explanations for details on reviewers.

   • Deadline: Deadline for submitting the explanation (default: 7 days). After the deadline, the explanation target can no longer submit a response.

   • Note: Content to add to the explanation request email. The entered content is inserted where the text macro $user_note appears in the email/SMS template body. If you enter a field name defined in the schema as a macro (e.g., $fieldname), it is replaced with the field value. You can view the note in the explanation detail under Response > Explanations (up to 10,000 characters).

   Note

   Explanation requests require a field named `emp_key`. If that field does not exist, click **Query** in step 5 and write a query that creates the `emp_key` field and maps it to a user account or IP address.
   

7. To issue tickets when events are detected, configure the Advanced Settings.

   

   • Ticket Repository: The ticket repository used to categorize tickets during security operations (default: none selected).

     Note

     You must specify a ticket repository to automatically create a ticket when an event is detected. You can also control ticket creation in a playbook triggered by an event. Do not use this setting if you want the playbook to control ticket creation.
     

   • Duplicate Key Field: Enter the field names to use as the duplicate-detection key, separated by commas (,), in macro format ($fieldname), up to 2,000 characters. If not set, the Message string is used as the basis.

     Note

     For example, if the message is set to "C&C server connection: $src_ip -> $dst_ip" but you want to avoid generating excessive tickets for events from the same host connecting to different C&C servers, set the duplicate key to `$src_ip`.
     

   • Deduplicate Events: When the same event occurs consecutively after a ticket is issued, the duration to ignore duplicate events (default: 0; range: 0–86,400 seconds).

     • When 0: Behavior follows the Deduplicate Tickets setting.
     • When not 0: Ignores the same event for the specified duration, and does not use it as supporting evidence for the existing ticket.

   • Deduplicate Tickets: When the same event occurs after a ticket is issued, the duration during which no new ticket is created but only supporting evidence and occurrence count are added to the existing ticket (default: 3,600; range: 0–86,400 seconds).

     • When 0: A new ticket with the same subject is created each time an event occurs.
     • When not 0: Only the supporting evidence and occurrence count are recorded in the existing ticket for the specified duration.

   • Output Field Order: The order of log data fields to display in the ticket's supporting evidence. Enter field names in display order, separated by commas (,) (up to 2,000 characters; excludes special characters and pipe symbols).

   • Closed Ticket Handling: How to handle a ticket with Deduplicate Tickets set after it is closed (default: Reset deduplication timer).

     • Retain deduplication timer: Even after the ticket is closed, if the same threat event occurs before the deduplication period expires, the evidence and occurrence count are added to the closed ticket.
     • Reset deduplication timer: When the ticket is closed, the deduplication period is treated as expired, and a new ticket is created when the same threat event occurs again.

8. Click Confirm to finish adding the stream rule. The rule is activated immediately upon creation.

Rules and parameters by field type

The scenario builder allows you to configure rules for each field type defined in the log schema. The available rules and parameters for each field data type are as follows.

STRING

BOOL

SHORT/INT/LONG/DOUBLE

DATE

COUNTRY

PORT

IP

MD5/SHA1/SHA256

URL

DOMAIN

Other — BLOB, etc.

Notes on configuring explanation requests

To configure explanation requests, you need a data field that can be mapped to the explanation target. For example, the employee ID, account, or source IP address is required. Most log schemas do not include fields related to explanation targets, so additional configuration is needed.

There are multiple ways to map explanation target information. The following is one representative approach.

1. Configure a lookup table for matching employee accounts and IDs

   • To extract employee IDs by matching account information from collected logs, configure an employee ID table as a lookup table in Analysis > Lookup. The table must contain account (user) and employee ID (emp_key) mapping data. For detailed instructions on adding a lookup table, refer to Analysis > Lookup.

   • When configuring the lookup table, the account field name in the collection table and the account field name in the lookup table must match for the lookup to work correctly.

     

2. Configure a log schema for normalizing employee ID information

   • To add the extracted employee ID as a field and normalize it, edit or add a log schema in Loggers > Log Schemas.

   • In this example, the intrusion detection log schema is used, and the employee ID field is added to that schema.

     

3. Configure a stream query to add the employee ID field by matching against the lookup table

   • To add and store the employee ID field by matching against the lookup table while collecting logs in real time, configure a stream query in Loggers > Logger Models.

   • Add a normalization rule for the employee ID field and configure the stream query for that rule. Example: lookup 'lookup table name' 'account field name' output 'employee ID field name to add'

     

   • If no logger using this logger model has been added, add one.

4. Configure employee information including ID and account

   • To send explanation request emails to employees, configure employee information in Users > Employees.

     

   • When configuring employees, if there is a user account corresponding to an employee, enter the account information in the Login Name field. Also configure the department head to enable completion of threat review through first and second review. For detailed configuration, refer to Users > Employees.

     

   Note

   You can import employee information from a database. For details, contact our technical support team.
   

5. Configure explanation request settings in the detection rule policy

   • To request an explanation from the employee who is the subject of a threat detected by the stream rule, configure the explanation request option.

     

Duplicate a stream rule

To duplicate an existing stream rule:

1. Select the checkbox of the stream rule you want to duplicate in the list.
2. An action menu appears in the toolbar. Click Duplicate in the action menu.
3. In the Duplicate Stream Rule dialog, review the list of rules to duplicate, then click Duplicate. Click Cancel to abort.
   • A new rule named 'original rule name copy' is added in a disabled state.

Edit a stream rule

To edit a stream rule:

1. Click the name of the stream rule you want to edit in the list.
2. On the Edit Stream Rule screen, update the information and click Confirm. For descriptions of each field, refer to Add a stream rule.

Delete stream rules

To delete stream rules:

1. Select the checkboxes of the rules you want to delete in the stream rule list.
2. An action menu appears in the toolbar. Click Delete in the action menu.
3. In the Delete Stream Rule dialog, review the list of rules to delete, then click Delete. Click Cancel to abort.