🧠 Classical ML · Tier 2 🔴 Advanced MODULE 08

Time Series Basics

⏱️ 3 hours

📈 ARIMA · Lag Features · Decomposition

🧩 5 Quiz Questions

Classical ML — Module 8 of 100%

🎯 What you'll learn: Why time-ordered data violates the assumptions of standard ML, how to decompose a series into trend + seasonality + residual, ARIMA model parameters, why shuffling is forbidden for time series cross-validation, creating lag and rolling-window features for sklearn models, and a complete sales forecasting project.

8.1 · What Makes Time Series Different

Temporal Order Cannot Be Ignored

Standard ML assumes samples are independent and identically distributed (IID). Time series violates this fundamentally — tomorrow's sales depend on today's sales. Shuffling training data destroys temporal information, and using future data to predict the past is look-ahead bias (a form of leakage).

The three structural components of most time series are: Trend (long-term upward/downward direction), Seasonality (repeating patterns at fixed periods — weekly, yearly), and Residual (unexplained noise after removing trend and seasonality).

Seasonal decomposition with statsmodelsPYTHON

from statsmodels.tsa.seasonal import seasonal_decompose
import pandas as pd

# Parse dates, set as index with monthly frequency
df['date'] = pd.to_datetime(df['date'])
df = df.set_index('date').asfreq('M')

# Multiplicative: trend * seasonality * residual (use when amplitude grows)
# Additive:       trend + seasonality + residual (use when amplitude is constant)
result = seasonal_decompose(df['sales'], model='multiplicative', period=12)
result.plot()  # shows 4 subplots: original, trend, seasonal, residual

# Access components individually
trend     = result.trend
seasonal  = result.seasonal
residual  = result.resid

⚠️

Never shuffle time series data

If you use sklearn's train_test_split(shuffle=True) on time series, you create look-ahead bias — the model trains on data from 2024 to predict data from 2022. Future information contaminates the past. Always split chronologically: the last N% of time steps as the test set.

8.2 · ARIMA — The Classical Forecasting Model

Auto-Regression, Integration, Moving Average

ARIMA(p,d,q) is the workhorse of classical forecasting. It combines three mechanisms to model stationary time series:

AR — Auto-Regression

Use past p observations as predictors. ACF plot shows if needed.

I — Integration

Differencing order to make series stationary. d=1 for most trends.

MA — Moving Average

Model residual errors using past q error terms. PACF plot guides choice.

ARIMA with statsmodels + auto_arimaPYTHON

from statsmodels.tsa.arima.model import ARIMA

# Manual ARIMA(1,1,1) — AR=1, difference once, MA=1
model = ARIMA(train, order=(1, 1, 1))
result = model.fit()
forecast = result.forecast(steps=12)
print(result.summary())

# Auto ARIMA: tests combinations and picks best by AIC
from pmdarima import auto_arima  # pip install pmdarima
auto_model = auto_arima(
    train,
    seasonal=True, m=12,        # monthly seasonality → SARIMA
    stepwise=True,               # faster search strategy
    information_criterion='aic'  # Akaike IC for model selection
)
print(f"Best order: {auto_model.order}")

8.3 · Lag Features & TimeSeriesSplit

Using sklearn Models on Time Series

You don't have to use ARIMA. Any sklearn model (XGBoost, Random Forest) can forecast time series if you convert the problem into supervised learning by creating lag features: the value at time t-1, t-2, t-7, etc. This approach often outperforms ARIMA on complex series with external predictors.

For cross-validation, use TimeSeriesSplit instead of KFold — it creates expanding windows where validation always follows training chronologically.

Lag features + rolling stats + TimeSeriesSplitPYTHON

from sklearn.model_selection import TimeSeriesSplit
from xgboost import XGBRegressor

# ── Create lag and rolling features ─────────────────────
df = df.sort_values('date')

for lag in [1, 2, 3, 7, 14, 28]:  # 1–4 weeks back
    df[f'lag_{lag}'] = df['sales'].shift(lag)

# Rolling statistics (shift 1 to avoid using current value)
df['roll_7_mean']  = df['sales'].shift(1).rolling(7).mean()
df['roll_28_std']  = df['sales'].shift(1).rolling(28).std()

# Drop rows with NaN from lagging
df = df.dropna()

# ── Correct CV: always train before test chronologically ─
tss = TimeSeriesSplit(n_splits=5)
scores = []
for train_idx, val_idx in tss.split(X):
    X_tr, X_val = X.iloc[train_idx], X.iloc[val_idx]
    y_tr, y_val = y.iloc[train_idx], y.iloc[val_idx]
    model = XGBRegressor(n_estimators=200)
    model.fit(X_tr, y_tr)
    scores.append(model.score(X_val, y_val))
print(f"Mean R²: {sum(scores)/len(scores):.3f}")

📌

Shift before rolling to prevent leakage

.shift(1).rolling(7).mean() computes the 7-day mean of the 7 days before the current row. Without .shift(1), the rolling mean includes the current value — your model learns from the value it is trying to predict, which is perfect on training data but impossible in production.

8.4 · Sales Forecasting Project

Full Pipeline: Decompose → Feature → Forecast

Combining everything into a real forecasting pipeline using the Rossmann Store Sales pattern (daily store sales with promotions and holidays).

End-to-end sales forecasting pipelinePYTHON

import pandas as pd
import numpy as np
from xgboost import XGBRegressor
from sklearn.metrics import mean_absolute_percentage_error

# 1. Load and sort by date
df['Date'] = pd.to_datetime(df['Date'])
df = df.sort_values('Date').reset_index(drop=True)

# 2. Date features
df['year']       = df['Date'].dt.year
df['month']      = df['Date'].dt.month
df['dayofweek']  = df['Date'].dt.dayofweek
df['dayofyear']  = df['Date'].dt.dayofyear
df['week']       = df['Date'].dt.isocalendar().week.astype(int)

# 3. Lag features (group by store to avoid store cross-contamination)
for lag in [1, 7, 14, 28]:
    df[f'sales_lag_{lag}'] = (
        df.groupby('Store')['Sales'].shift(lag)
    )
df['sales_roll7'] = df.groupby('Store')['Sales'].shift(1).rolling(7).mean()
df = df.dropna()

# 4. Chronological split — last 6 weeks = test
cutoff = df['Date'].max() - pd.Timedelta(weeks=6)
train  = df[df['Date'] <= cutoff]
test   = df[df['Date'] >  cutoff]

features = ['year', 'month', 'dayofweek', 'Promo',
             'sales_lag_1', 'sales_lag_7', 'sales_roll7']

model = XGBRegressor(n_estimators=500, learning_rate=0.05)
model.fit(train[features], train['Sales'])
y_pred = model.predict(test[features])

mape = mean_absolute_percentage_error(test['Sales'], y_pred)
print(f"Test MAPE: {mape:.2%}")  # typically 5–15% on this dataset

🧩 Module 8 Check

5 questions · instant feedback

1. Why is random shuffling of time series data before train/test split a problem?

2. In ARIMA(p,d,q), what does the 'd' parameter control?

3. You create a feature: df['roll_7_mean'] = df['sales'].rolling(7).mean(). What is wrong with this for forecasting?

4. Why should you use TimeSeriesSplit instead of KFold for cross-validation on time series?

5. Your sales series shows an upward trend and peaks every December. Which decomposition model should you choose?

← Module 7: Model Tuning Module 9: Recommenders →