forecast

Analyzes the trend of time series data to predict future values. For periodic data, uses STL decomposition (Seasonal and Trend decomposition using Loess) to separate the data into trend, seasonality, and residual components, then produces forecast values based on the trend component. Starting with the 4.0 series' 4.0.2609.0 release, it also assigns each input record an anomaly score based on its deviation from the expected forecast value.

Command properties

Syntax

In the syntax above, predictor, method, and alpha are available starting with 4.0.2609.0.

Options

count=INT

Number of records to forecast. Must be a natural number of 1 or more. (default: 5)

period=INT

Period length of the time series data. If specified, must be a natural number of 3 or more. If not specified, the period is automatically calculated using Spectral Density Estimation, and if the calculated period is less than 2, the data is treated as non-periodic and simple linear regression is applied.

predictor={ets|linear|loess|constant}

When periodic data is decomposed with method=stl (default), the method used to forecast the trend component (default: ets). This option name, and the constant value, are available starting with 4.0.2609.0. In the 5.0 series (through 5.0.2605.2), use the smoother option instead, which accepts only ets, linear, and loess (no constant).

• ets: Exponential smoothing
• linear: Linear regression
• loess: LOESS (Locally Estimated Scatterplot Smoothing)
• constant: Uses the mean value of the trend component as a constant. Suitable for time series without a clear trend. Available starting with 4.0.2609.0.

The option name smoother, used in earlier versions, continues to be recognized as an alias for predictor in the 4.0 series. This option does not apply when method=linear, or when the data is treated as non-periodic.

normalize=BOOL

Whether to apply Box-Cox transformation. When set to true, corrects for non-linearly increasing trends. May have a performance impact due to additional computation. (default: false)

accumulate=BOOL

Whether to accumulate confidence intervals. When set to true, the confidence interval widens as the forecast point moves further into the future. (default: true)

confidence=DOUBLE

Confidence interval percentage used to calculate the upper/lower bounds and the anomaly score. Must be a real number of 0 or more and less than 100. (default: 95) This DOUBLE type and range validation apply starting with 4.0.2609.0. In the 5.0 series (through 5.0.2605.2), this option accepts a whole-number percentage instead and calculates only the confidence interval, since the anomaly score is not available in that series.

seed=INT

Random seed value. Specify a seed value to reproduce the same results for the same input.

time=STR

Name of the time field. Used to calculate the time interval of the time series data. (default: _time)

method={stl|linear}

Time series decomposition algorithm (default: stl). For non-periodic data — where the calculated (or specified) period is less than 2 — simple linear regression is applied regardless of this option. Available starting with 4.0.2609.0.

• stl: Uses STL decomposition to separate the data into trend, seasonality, and residual components, then forecasts the trend component according to the predictor option.
• linear: Forecasts by combining the per-period average (seasonality) with linear regression (trend). The predictor option does not apply with this method.

alpha=DOUBLE

Decay coefficient used to calculate the anomaly score (_score). Must be a real number of 0 or more and 1 or less. Each record's anomaly score is calculated cumulatively as previous record's _score value * alpha + current record's probability-based score. (default: 0.6) Available starting with 4.0.2609.0.

Target

FIELD

Field containing the time series data to forecast

[by FIELD, ...]

Group fields. When a by clause is specified, forecasting is performed independently for records with the same group field value. Specify multiple fields separated by commas (,).

Input fields

Output fields

Error codes

Parse errors

Runtime errors

Description

The forecast command collects all input records before performing time series forecasting. Input records are output as-is with the trend component (_trend) added, and forecasted records are appended after the input records with the _future, _upper, and _lower fields, up to the number specified by the count option. Starting with the 4.0 series' 4.0.2609.0 release, input records also receive the expected forecast value (_expected), confidence interval (_upper, _lower), and anomaly score (_score).

When the period value is 2 or more, the trend and seasonality are separated. In the 4.0 series starting with 4.0.2609.0, this is controlled by the method option, using either STL decomposition (stl, default) or a method that combines the per-period average with linear regression (linear); the predictor option applies only when method=stl, and determines the method used to forecast the trend component. The 5.0 series (through 5.0.2605.2) always uses STL decomposition and does not have the method option; the equivalent smoother option determines the trend forecasting method. When the period is less than 2, the data is treated as non-periodic and simple linear regression is used regardless of these options.

The forecast record time values are calculated based on the interval between the last two records of the time field specified by the time option. It is recommended to prepare data with a consistent time interval using the timechart command before using this command.

Forecasting is performed successfully only when all of the following conditions are met for the input data; otherwise, the original record is output as-is. In the 4.0 series starting with 4.0.2609.0, a _forecast_error field is also added to such records.

• There must be at least 5 records per partition.
• The value of the forecast target field must not be null in the partition's first record.
• If the period option is specified, the number of input records must be at least twice the period value.
• The time field specified by the time option must be present in the partition's last two records and must be of the timestamp type (because the interval between these two records is used to calculate the forecast time values).

When a by clause is used, forecasting is performed independently for records with the same group field value. Temporary files may be used internally for sorting by group.

Examples

1. Output forecast values for time series data

   table duration=30d web_logs
   | timechart span=1d count
   | forecast count
   

   Aggregates web logs from the past 30 days by day, then outputs 5 forecast values for the count field. In the 4.0 series starting with 4.0.2609.0, this also outputs the anomaly score (_score) of each input record.

2. Specify period and forecast count

   table duration=90d web_logs
   | timechart span=1d sum(bytes) as traffic
   | forecast period=7 count=14 traffic
   

   Sets the period to 7 (weekly pattern) and outputs 14 forecast data points.

3. Specify the time field and seed value

   table duration=30d web_logs
   | timechart span=1h sum(bytes) as traffic
   | forecast period=24 time=_time seed=1234 traffic
   

   Sets the period to 24 (daily pattern) and fixes the seed value to 1234 to produce reproducible results for the same input.

4. Forecast by group

   table duration=30d web_logs
   | timechart span=1d count by method
   | forecast count=10 count by method
   

   Independently forecasts 10 values for the count field per group with the same method field value.

5. Apply Box-Cox transformation

   table duration=90d web_logs
   | timechart span=1d sum(bytes) as traffic
   | forecast period=7 normalize=true traffic
   

   Applies Box-Cox transformation to correct for non-linearly increasing trends before forecasting.

6. Using a constant predictor for a time series without a trend (4.0 series, 4.0.2609.0 and later)

   table duration=30d web_logs
   | timechart span=1d count
   | forecast period=7 predictor=constant alpha=0.8 count
   

   Uses a constant predictor (predictor=constant) for a time series without a clear trend, and raises the anomaly score accumulation weight (alpha) to 0.8 to more heavily reflect the influence of the previous record's score. The predictor and alpha options are available starting with 4.0.2609.0.

Change history