summary

Outputs statistical summary information for each field of the input records. You can check the field name, type, record count, minimum value, maximum value, and average value at once. This is useful for data exploration and quality verification.

Command properties

ItemDescription
Command typeTransforming
Required permissionNone
License usageN/A
Parallel executionSupported
Distributed executionRuns on Data Node (mapper)

Syntax

summary

Options

N/A

Output fields

FieldTypeDescription
namestringField name
typestringField data type
countlongNumber of records with a non-null value in the field
mindoubleMinimum value for numeric fields. Null for non-numeric types.
maxdoubleMaximum value for numeric fields. Null for non-numeric types.
avgdoubleAverage value for numeric fields. Null for non-numeric types.

Error codes

Parse errors

N/A

Runtime errors

N/A

Description

The summary command consumes all input records and then generates and outputs a statistical summary record for each field. Output records are sorted alphabetically by field name.

For numeric type (integer, floating-point) fields, minimum, maximum, and average values are calculated. The min, max, and avg fields are null for non-numeric fields.

The count field represents the number of records with a non-null value in that field. Therefore, it may differ from the total record count.

Examples

  1. Check field statistics for a table

    table duration=1h ACCESS_LOG | summary
    

    Reads data from the ACCESS_LOG table for the past hour and checks the type, count, minimum, maximum, and average values for each field.

  2. Verify parsing results

    table duration=10m FIREWALL_LOG
    | parsekv
    | summary
    

    Parses firewall logs in key-value format and checks the field list and statistics for each field.

Compatibility

The summary command has been available since before Sonar 4.0.

See also

  • stats — Group-based statistical aggregation using aggregate functions
  • fields — Select or remove output fields
  • eval — Field operations and type checking