∫ Mathematical Derivations
Complete mathematical theory behind ARIMA forecasting and inventory optimization - formatted for BSc Mathematics viva presentation.
1. Time Series
A time series is a sequence of observations $\{Y_t\}_{t=1}^n$ indexed by time $t$. In inventory management, $Y_t$ represents demand in period $t$.
Formally, a time series is a collection of random variables ordered by time. The goal is to model the data-generating process and use it for forecasting.
Definition
$$\{Y_t : t \in \mathbb{Z}\} \quad\text{where } Y_t \in \mathbb{R}$$
Components of a time series:
• Trend (T): Long-term increase or decrease
• Seasonality (S): Periodic fluctuations
• Cyclical (C): Non-periodic fluctuations
• Irregular (I): Random noise
Classical decomposition: $Y_t = T_t + S_t + C_t + I_t$ (additive)
• Trend (T): Long-term increase or decrease
• Seasonality (S): Periodic fluctuations
• Cyclical (C): Non-periodic fluctuations
• Irregular (I): Random noise
Classical decomposition: $Y_t = T_t + S_t + C_t + I_t$ (additive)
2. Stationarity
A time series $\{Y_t\}$ is weakly (covariance) stationary if its statistical properties are invariant to shifts in time.
Three Conditions for Weak Stationarity
$$\mathbb{E}[Y_t] = \mu \quad \forall t \quad \text{(constant mean)}$$
$$\text{Var}(Y_t) = \sigma^2 < \infty \quad \forall t \quad \text{(constant variance)}$$
$$\text{Cov}(Y_t, Y_{t-k}) = \gamma(k) \quad \text{(depends only on lag } k\text{)}$$
Why stationarity matters: ARIMA requires stationarity because:
(1) the autocovariance function must be well-defined, and
(2) forecasts would be unbounded for non-stationary processes.
3. Augmented Dickey-Fuller (ADF) Test
The ADF test tests for a unit root in an autoregressive model. A series with a unit root is non-stationary.
ADF Regression Model
$$\Delta Y_t = \alpha + \beta t + \gamma Y_{t-1} + \sum_{i=1}^{k} \delta_i \Delta Y_{t-i} + \varepsilon_t$$
H₀ (Null): $\gamma = 0$ - the series has a unit root → non-stationary
H₁ (Alternative): $\gamma < 0$ - no unit root → stationary
Test Statistic: $\tau = \hat{\gamma} / SE(\hat{\gamma})$ (follows Dickey-Fuller distribution, not standard normal)
Decision Rule: Reject H₀ if $\tau <$ critical value or $p$-value $< \alpha = 0.05$
The augmented terms $\sum \delta_i \Delta Y_{t-i}$ account for higher-order autocorrelation (lags selected by AIC).
H₁ (Alternative): $\gamma < 0$ - no unit root → stationary
Test Statistic: $\tau = \hat{\gamma} / SE(\hat{\gamma})$ (follows Dickey-Fuller distribution, not standard normal)
Decision Rule: Reject H₀ if $\tau <$ critical value or $p$-value $< \alpha = 0.05$
The augmented terms $\sum \delta_i \Delta Y_{t-i}$ account for higher-order autocorrelation (lags selected by AIC).
4. Differencing
Differencing is a transformation applied to a non-stationary series to induce stationarity. It removes deterministic trends.
First and Second Order Differences
$$\Delta Y_t = Y_t - Y_{t-1} \quad \text{(First difference)}$$
$$\Delta^2 Y_t = \Delta Y_t - \Delta Y_{t-1} = Y_t - 2Y_{t-1} + Y_{t-2} \quad \text{(Second difference)}$$
$$\Delta^d Y_t = (1-B)^d Y_t \quad \text{where } B^k Y_t = Y_{t-k}$$
Backshift Operator B: $B Y_t = Y_{t-1}$, $B^2 Y_t = Y_{t-2}$
d is the "I" (Integrated) in ARIMA(p, d, q)
Example: Random walk $Y_t = Y_{t-1} + \varepsilon_t$ → $\Delta Y_t = \varepsilon_t$ (white noise - stationary!)
d is the "I" (Integrated) in ARIMA(p, d, q)
Example: Random walk $Y_t = Y_{t-1} + \varepsilon_t$ → $\Delta Y_t = \varepsilon_t$ (white noise - stationary!)
5. Autoregressive Model - AR(p)
An AR(p) model expresses $Y_t$ as a linear combination of its p previous values plus a white noise term.
AR(p) Definition
$$Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \cdots + \phi_p Y_{t-p} + \varepsilon_t$$
$$\phi(B) Y_t = \varepsilon_t \quad \text{where } \phi(B) = 1 - \phi_1 B - \phi_2 B^2 - \cdots - \phi_p B^p$$
$\phi_i$ = autoregressive coefficients (estimated by least squares or MLE)
$\varepsilon_t$ = white noise: $\varepsilon_t \sim WN(0, \sigma^2)$
Stationarity condition: All roots of $\phi(z) = 0$ must lie outside the unit circle $|z| > 1$
Identification: PACF cuts off sharply at lag $p$ for AR(p) processes
AR(1) Yule-Walker: $\phi_1 = \rho(1)$, AR(2): solve $\begin{pmatrix} 1 & \rho(1) \\ \rho(1) & 1 \end{pmatrix}\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} = \begin{pmatrix}\rho(1)\\\rho(2)\end{pmatrix}$
$\varepsilon_t$ = white noise: $\varepsilon_t \sim WN(0, \sigma^2)$
Stationarity condition: All roots of $\phi(z) = 0$ must lie outside the unit circle $|z| > 1$
Identification: PACF cuts off sharply at lag $p$ for AR(p) processes
AR(1) Yule-Walker: $\phi_1 = \rho(1)$, AR(2): solve $\begin{pmatrix} 1 & \rho(1) \\ \rho(1) & 1 \end{pmatrix}\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} = \begin{pmatrix}\rho(1)\\\rho(2)\end{pmatrix}$
6. Moving Average Model - MA(q)
An MA(q) model expresses $Y_t$ as a linear combination of the current and past q white noise (shock) terms.
MA(q) Definition
$$Y_t = \varepsilon_t + \theta_1 \varepsilon_{t-1} + \theta_2 \varepsilon_{t-2} + \cdots + \theta_q \varepsilon_{t-q}$$
$$Y_t = \theta(B)\varepsilon_t \quad \text{where } \theta(B) = 1 + \theta_1 B + \cdots + \theta_q B^q$$
$\theta_j$ = moving average coefficients
Invertibility condition: All roots of $\theta(z) = 0$ must lie outside the unit circle
Autocorrelation: $\text{Cov}(Y_t, Y_{t-k}) = 0$ for $k > q$ → ACF cuts off at lag $q$
This makes MA(q) always stationary (finite variance regardless of coefficients)
Identification: ACF cuts off sharply at lag $q$ for MA(q) processes
Invertibility condition: All roots of $\theta(z) = 0$ must lie outside the unit circle
Autocorrelation: $\text{Cov}(Y_t, Y_{t-k}) = 0$ for $k > q$ → ACF cuts off at lag $q$
This makes MA(q) always stationary (finite variance regardless of coefficients)
Identification: ACF cuts off sharply at lag $q$ for MA(q) processes
7. ARIMA(p, d, q) Model
ARIMA combines the AR and MA components with differencing to handle non-stationary series. It is the Box-Jenkins methodology.
ARIMA(p, d, q) - General Form
$$\phi(B)(1-B)^d Y_t = \theta(B)\varepsilon_t$$
$$\text{where:} \quad W_t = \Delta^d Y_t = (1-B)^d Y_t$$
$$\phi(B)W_t = \theta(B)\varepsilon_t \quad \text{(ARMA on the differenced series)}$$
p = order of autoregressive part (number of AR lags)
d = degree of differencing (makes series stationary)
q = order of moving average part (number of MA lags)
Parameter Estimation: Maximum Likelihood Estimation (MLE)
$$\hat{\Theta} = \arg\max_\Theta \ln L(\Theta | Y_1, \ldots, Y_n)$$ where $\Theta = (\phi_1,\ldots,\phi_p, \theta_1,\ldots,\theta_q, \sigma^2)$
Model Selection:
$\text{AIC} = -2\ln\hat{L} + 2k$ $\text{BIC} = -2\ln\hat{L} + k\ln n$
where $k$ = number of parameters, $n$ = sample size. Minimise AIC/BIC.
d = degree of differencing (makes series stationary)
q = order of moving average part (number of MA lags)
Parameter Estimation: Maximum Likelihood Estimation (MLE)
$$\hat{\Theta} = \arg\max_\Theta \ln L(\Theta | Y_1, \ldots, Y_n)$$ where $\Theta = (\phi_1,\ldots,\phi_p, \theta_1,\ldots,\theta_q, \sigma^2)$
Model Selection:
$\text{AIC} = -2\ln\hat{L} + 2k$ $\text{BIC} = -2\ln\hat{L} + k\ln n$
where $k$ = number of parameters, $n$ = sample size. Minimise AIC/BIC.
8. Autocorrelation Function (ACF)
The ACF measures the linear dependence between $Y_t$ and $Y_{t-k}$ for lag $k$. It is the primary tool for identifying MA order.
ACF Definition and Estimator
$$\rho(k) = \frac{\gamma(k)}{\gamma(0)} = \frac{\text{Cov}(Y_t, Y_{t-k})}{\text{Var}(Y_t)}$$
$$\hat{\gamma}(k) = \frac{1}{n}\sum_{t=k+1}^n (Y_t - \bar{Y})(Y_{t-k} - \bar{Y})$$
$$\hat{\rho}(k) = \frac{\hat{\gamma}(k)}{\hat{\gamma}(0)} \quad \in [-1, 1]$$
Bartlett's Formula (95% confidence bounds): $\pm 1.96/\sqrt{n}$
Lags where $|\hat{\rho}(k)| > 1.96/\sqrt{n}$ are statistically significant at the 5% level.
Patterns:
• AR(p): ACF decays exponentially or with damped oscillations
• MA(q): ACF has sharp cutoff after lag $q$ (spikes only for $k \leq q$)
• ARMA(p,q): ACF decays exponentially after lag $q$
Lags where $|\hat{\rho}(k)| > 1.96/\sqrt{n}$ are statistically significant at the 5% level.
Patterns:
• AR(p): ACF decays exponentially or with damped oscillations
• MA(q): ACF has sharp cutoff after lag $q$ (spikes only for $k \leq q$)
• ARMA(p,q): ACF decays exponentially after lag $q$
9. Partial Autocorrelation Function (PACF)
PACF measures the correlation between $Y_t$ and $Y_{t-k}$ after removing the linear effects of all intermediate lags $Y_{t-1}, \ldots, Y_{t-k+1}$.
PACF via Projection
$$\phi_{kk} = \text{Corr}(Y_t - \hat{Y}_t,\; Y_{t-k} - \hat{Y}_{t-k})$$
where $\hat{Y}_t$ = projection of $Y_t$ on $\{Y_{t-1}, \ldots, Y_{t-k+1}\}$.
Yule-Walker Equations give the PACF via:
$\begin{pmatrix}\gamma(0) & \gamma(1) & \cdots & \gamma(k-1)\\ \vdots & & & \vdots \\ \gamma(k-1) & \cdots & & \gamma(0)\end{pmatrix} \begin{pmatrix}\phi_{k1}\\ \vdots\\ \phi_{kk}\end{pmatrix} = \begin{pmatrix}\gamma(1)\\ \vdots\\ \gamma(k)\end{pmatrix}$
Patterns:
• AR(p): PACF has sharp cutoff after lag $p$
• MA(q): PACF decays exponentially
• ARMA(p,q): PACF decays after lag $p-q$
Yule-Walker Equations give the PACF via:
$\begin{pmatrix}\gamma(0) & \gamma(1) & \cdots & \gamma(k-1)\\ \vdots & & & \vdots \\ \gamma(k-1) & \cdots & & \gamma(0)\end{pmatrix} \begin{pmatrix}\phi_{k1}\\ \vdots\\ \phi_{kk}\end{pmatrix} = \begin{pmatrix}\gamma(1)\\ \vdots\\ \gamma(k)\end{pmatrix}$
Patterns:
• AR(p): PACF has sharp cutoff after lag $p$
• MA(q): PACF decays exponentially
• ARMA(p,q): PACF decays after lag $p-q$
10. Mean Absolute Error (MAE)
MAE Definition
$$\text{MAE} = \frac{1}{n}\sum_{t=1}^n |Y_t - \hat{Y}_t| = \frac{1}{n}\sum_{t=1}^n |e_t|$$
Properties:
• Unit: same as $Y_t$ (demand units)
• Robust to outliers (linear loss function)
• Minimised by the median forecast
• All errors receive equal weight regardless of magnitude
• Unit: same as $Y_t$ (demand units)
• Robust to outliers (linear loss function)
• Minimised by the median forecast
• All errors receive equal weight regardless of magnitude
11. Root Mean Squared Error (RMSE)
RMSE Definition
$$\text{RMSE} = \sqrt{\frac{1}{n}\sum_{t=1}^n (Y_t - \hat{Y}_t)^2} = \sqrt{\text{MSE}}$$
Properties:
• Unit: same as $Y_t$ (demand units)
• Sensitive to outliers (quadratic loss function)
• Minimised by the mean forecast
• Always: $\text{RMSE} \geq \text{MAE}$ (by Cauchy-Schwarz inequality)
• $\text{RMSE} = \text{MAE}$ only when all forecast errors are equal in magnitude
• Unit: same as $Y_t$ (demand units)
• Sensitive to outliers (quadratic loss function)
• Minimised by the mean forecast
• Always: $\text{RMSE} \geq \text{MAE}$ (by Cauchy-Schwarz inequality)
• $\text{RMSE} = \text{MAE}$ only when all forecast errors are equal in magnitude
12. Mean Absolute Percentage Error (MAPE)
MAPE Definition
$$\text{MAPE} = \frac{100\%}{n}\sum_{t=1}^n \left|\frac{Y_t - \hat{Y}_t}{Y_t}\right|$$
Interpretation Guide:
MAPE < 10%: Highly Accurate | 10-20%: Good | 20-50%: Reasonable | >50%: Inaccurate
Limitation: Undefined when $Y_t = 0$; asymmetric (penalises positive errors more)
Advantage: Scale-free - allows comparison across different demand series
MAPE < 10%: Highly Accurate | 10-20%: Good | 20-50%: Reasonable | >50%: Inaccurate
Limitation: Undefined when $Y_t = 0$; asymmetric (penalises positive errors more)
Advantage: Scale-free - allows comparison across different demand series
13. Economic Order Quantity (EOQ)
The EOQ model minimises total annual inventory cost by finding the optimal order quantity $Q^*$.
EOQ Derivation - Wilson's Formula (1913)
$$\text{Total Cost}(Q) = \underbrace{\frac{D}{Q} \cdot S}_{\text{Annual Ordering Cost}} + \underbrace{\frac{Q}{2} \cdot H}_{\text{Annual Holding Cost}}$$
$$\frac{d(\text{TC})}{dQ} = -\frac{DS}{Q^2} + \frac{H}{2} = 0$$
$$Q^* = \sqrt{\frac{2DS}{H}}$$
Variables:
$D$ = Annual demand (units/year) $S$ = Ordering cost per order (₹)
$H$ = Holding cost per unit per year (₹) $Q^*$ = Optimal order quantity
Second-order condition: $\frac{d^2(\text{TC})}{dQ^2} = \frac{2DS}{Q^3} > 0$ → confirms this is a minimum
Minimum Total Cost: $\text{TC}^* = \sqrt{2DSH}$
$D$ = Annual demand (units/year) $S$ = Ordering cost per order (₹)
$H$ = Holding cost per unit per year (₹) $Q^*$ = Optimal order quantity
Second-order condition: $\frac{d^2(\text{TC})}{dQ^2} = \frac{2DS}{Q^3} > 0$ → confirms this is a minimum
Minimum Total Cost: $\text{TC}^* = \sqrt{2DSH}$
14. Safety Stock
Safety stock is a buffer maintained to protect against demand variability during the lead time.
Safety Stock Formula
$$\text{SS} = Z_\alpha \cdot \sigma_{LT}$$
$$\sigma_{LT} = \sigma_d \cdot \sqrt{LT} \quad \text{(demand std dev over lead time)}$$
$$\therefore \text{SS} = Z_\alpha \cdot \sigma_d \cdot \sqrt{LT}$$
Derivation: If $D_i \sim \mathcal{N}(\mu_d, \sigma_d^2)$ are i.i.d. daily demands,
Total lead time demand: $D_{LT} = \sum_{i=1}^{LT} D_i \sim \mathcal{N}(LT \cdot \mu_d,\; LT \cdot \sigma_d^2)$
We set safety stock so that: $P(D_{LT} \leq \text{ROP}) = \text{Service Level}$
$\Rightarrow$ SS = $Z_\alpha \cdot \sigma_{LT}$ where $Z_\alpha = \Phi^{-1}(\text{SL})$
Z-score table: SL 90% → Z=1.282 | SL 95% → Z=1.645 | SL 99% → Z=2.326
Total lead time demand: $D_{LT} = \sum_{i=1}^{LT} D_i \sim \mathcal{N}(LT \cdot \mu_d,\; LT \cdot \sigma_d^2)$
We set safety stock so that: $P(D_{LT} \leq \text{ROP}) = \text{Service Level}$
$\Rightarrow$ SS = $Z_\alpha \cdot \sigma_{LT}$ where $Z_\alpha = \Phi^{-1}(\text{SL})$
Z-score table: SL 90% → Z=1.282 | SL 95% → Z=1.645 | SL 99% → Z=2.326
15. Reorder Point (ROP)
The ROP is the inventory level at which a replenishment order should be triggered to avoid stockout.
ROP Formula
$$\text{ROP} = \bar{d} \cdot LT + \text{SS}$$
$$= \bar{d} \cdot LT + Z_\alpha \cdot \sigma_d \cdot \sqrt{LT}$$
Interpretation:
$\bar{d} \cdot LT$ = Expected demand during lead time (cycle stock component)
$Z_\alpha \cdot \sigma_d \cdot \sqrt{LT}$ = Buffer for demand uncertainty (safety stock component)
Decision Rule: When $\text{Inventory} \leq \text{ROP}$, immediately place an order of size $Q^*$ (EOQ).
Complete Policy: $(Q^*, \text{ROP})$ policy - orders of fixed size $Q^*$ placed when inventory hits $\text{ROP}$.
$\bar{d} \cdot LT$ = Expected demand during lead time (cycle stock component)
$Z_\alpha \cdot \sigma_d \cdot \sqrt{LT}$ = Buffer for demand uncertainty (safety stock component)
Decision Rule: When $\text{Inventory} \leq \text{ROP}$, immediately place an order of size $Q^*$ (EOQ).
Complete Policy: $(Q^*, \text{ROP})$ policy - orders of fixed size $Q^*$ placed when inventory hits $\text{ROP}$.
For Your Viva: All mathematical derivations on this page use rigorous notation consistent with standard textbooks. Key references: Box & Jenkins (1976), Montgomery et al., Wilson (1913). The ARIMA methodology follows the three-stage Box-Jenkins iterative procedure: Identification → Estimation → Diagnostic Checking.