hbos

Calculates the anomaly score of each record using the Histogram-Based Outlier Score (HBOS) algorithm. For each field, a histogram is built, and the score reflects how rare the value is within its distribution.

Command properties

Syntax

To build a histogram model:

To calculate outlier scores using a histogram model:

Options

op={build|query}

Operation mode

• build: Builds a histogram model from input records and outputs the serialized model data.
• query: Builds or loads a histogram model via subquery, then calculates an outlier score for each record.

k=INT

Number of buckets (bins) in the histogram. Must be a positive integer. (Default: 10)

alpha=FLOAT

Smoothing parameter used to calculate dynamic bucket boundaries. Must be a real number between 0 and 1 (exclusive). (Default: 0.1)

tolerance=FLOAT

Tolerance for bucket boundary determination. Must be a real number between 0 and 1 (exclusive). (Default: 0.1)

Target

FIELD, ...

List of fields to use for outlier score calculation. Separate multiple fields with commas (,). Field values must be numeric.

[by CLAUSE, ...]

Grouping fields. When specified, a separate histogram model is built for each group.

[ SUBQUERY ]

A subquery (enclosed in square brackets [ ]) that retrieves a previously built HBOS model for use with op=query. When specified, outlier scores are calculated in real time.

Output fields

When op=build:

When op=query:

Error codes

Parsing errors

Runtime errors

N/A

Description

The hbos command detects multivariate outliers using the HBOS (Histogram-Based Outlier Score) algorithm. It independently builds a histogram for each field, calculates how rare each value is within its distribution, and sums the scores across all fields.

When run with op=build, it builds a histogram model from input records, serializes the model data, and outputs it as _hbos_bykey and _hbos_model fields. You can store the output in a table using the import command and then use it as a subquery for op=query.

When run with op=query, it builds or loads a histogram model and assigns an outlier score to the _hbos_score field for each input record.

When a by clause is specified, a separate histogram model is built for each group.

Examples

1. Calculate outlier scores in real time

   table duration=1d network_logs
   | eval bytes = long(bytes), pkts = long(pkts)
   | hbos op=query bytes, pkts
   | sort +_hbos_score
   

   Calculates an HBOS outlier score for each record based on the bytes and pkts fields, then sorts by score.

2. Build and save a model

   table duration=7d network_logs
   | eval bytes = long(bytes), pkts = long(pkts)
   | hbos op=build bytes, pkts
   | import hbos_model
   

   Builds an HBOS model from 7 days of data and saves it to the hbos_model table.

3. Outlier detection using a pre-built model

   table duration=1d network_logs
   | eval bytes = long(bytes), pkts = long(pkts)
   | hbos op=query bytes, pkts [ table hbos_model ]
   | search _hbos_score > 10
   

   Calculates outlier scores using the pre-built model in the hbos_model table, and retrieves records with a score greater than 10.

4. Per-group outlier detection

   table duration=1d network_logs
   | eval bytes = long(bytes), pkts = long(pkts)
   | hbos op=query bytes, pkts by src_ip
   

   Builds a separate histogram model for each src_ip and calculates outlier scores.