Model Evaluation
Assess ARIMA(1,0,1) forecast accuracy using MAE, RMSE, and MAPE metrics.
On average, the ARIMA forecast deviates from actual demand by 24.7402 units. MAE treats all errors equally - no penalty for large errors.
RMSE = 29.9596 units. Penalises large errors more than MAE. Useful for detecting outlier forecasts. Always RMSE ≥ MAE.
Scale-free accuracy metric. MAPE = 4.8292%. Interpretation: Accurate Forecast (MAPE < 10%)
| Metric | Value | Unit |
|---|---|---|
| MAE | 24.7402 | Demand Units |
| RMSE | 29.9596 | Demand Units |
| MAPE | 4.8292% | Percentage |
| Observations | 19 | Periods |
Accurate Forecast (MAPE < 10%)
MAPE guide: <10% = Highly Accurate | 10-20% = Good | 20-50% = Reasonable | >50% = Poor
When errors are normally distributed: $\text{RMSE} \approx \sigma \cdot \sqrt{2/\pi} \cdot \text{MAE}/\text{MAE}$
MAPE is undefined if any $Y_t = 0$; use RMSE in such cases.
The metrics above are in-sample: they compare the fitted values against the very observations used to estimate the coefficients, so they flatter the model. Here the model is re-fitted on only the first 14 periods and then asked to forecast the 5 periods it has never seen. This is the honest measure of forecasting performance.
| Period | Actual Demand | Forecast | Error |
|---|---|---|---|
| 2024-06 | 498.0 | 526.6 | -28.6 |
| 2024-07 | 519.0 | 524.03 | -5.03 |
| 2024-08 | 512.0 | 522.3 | -10.3 |
| 2024-09 | 493.0 | 521.14 | -28.14 |
| 2024-10 | 504.0 | 520.35 | -16.35 |