summary
Outputs statistical summary information for each field of the input records. You can check the field name, type, record count, minimum value, maximum value, and average value at once. This is useful for data exploration and quality verification.
Command properties
| Item | Description |
|---|---|
| Command type | Transforming |
| Required permission | None |
| License usage | N/A |
| Parallel execution | Supported |
| Distributed execution | Runs on Data Node (mapper) |
Syntax
Options
N/A
Output fields
| Field | Type | Description |
|---|---|---|
| name | string | Field name |
| type | string | Field data type |
| count | long | Number of records with a non-null value in the field |
| min | double | Minimum value for numeric fields. Null for non-numeric types. |
| max | double | Maximum value for numeric fields. Null for non-numeric types. |
| avg | double | Average value for numeric fields. Null for non-numeric types. |
Error codes
Parse errors
N/A
Runtime errors
N/A
Description
The summary command consumes all input records and then generates and outputs a statistical summary record for each field. Output records are sorted alphabetically by field name.
For numeric type (integer, floating-point) fields, minimum, maximum, and average values are calculated. The min, max, and avg fields are null for non-numeric fields.
The count field represents the number of records with a non-null value in that field. Therefore, it may differ from the total record count.
Examples
-
Check field statistics for a table
table duration=1h ACCESS_LOG | summaryReads data from the
ACCESS_LOGtable for the past hour and checks the type, count, minimum, maximum, and average values for each field. -
Verify parsing results
table duration=10m FIREWALL_LOG | parsekv | summaryParses firewall logs in key-value format and checks the field list and statistics for each field.
Compatibility
The summary command has been available since before Sonar 4.0.