unpack

Decomposes a binary field's byte array into individual fields by index. Use this command to convert a fixed-length byte array into a numeric feature vector for use as machine learning model input.

Command properties

Syntax

Options

field=FIELD

Name of the binary field to decompose into a byte array.

Output fields

Byte indices are expressed as two-digit decimal numbers. For example, byte 0 is b00, byte 9 is b09, and byte 10 is b10.

Error codes

Parsing errors

Runtime errors

N/A

Description

The unpack command separates the byte array (byte[]) of the specified field by byte position and outputs each byte as an individual field in the format b00, b01, etc. Each byte is converted to an unsigned integer (0–255).

Up to 100 bytes are processed. If the byte array length exceeds 100, only the first 100 bytes are decomposed.

If the specified field is not of type byte[], the record passes through without conversion.

When used with the similar-docs command, you can search for similar documents using byte vectors of TLSH hashes.

Examples

1. Decompose a TLSH hash into a byte vector

   table duration=1d malware_samples
   | eval hash_bytes = fromhex(tlsh_hash)
   | unpack field=hash_bytes
   | fields b00, b01, b02, b03, b04
   

   Converts the hex string in the tlsh_hash field to a byte array and decomposes the first 5 bytes into individual fields.

2. Use a byte vector as anomaly detection model input

   table duration=1d binary_logs
   | unpack field=payload
   | anomalies b00, b01, b02, b03, b04, b05, b06, b07
   

   Decomposes the first 8 bytes of the payload byte array and performs anomaly detection analysis.

Compatibility

The unpack command is available from Sonar 4.0.2308.0.

See also

• anomalies — Run anomaly detection models
• similar-docs — Search for similar documents using TLSH hash
• kmeans — K-means clustering