🧠 Classical ML · Tier 2
🟢 Beginner
MODULE 01
The scikit-learn Workflow
Classical ML — Module 1 of 100%
🎯 What you'll learn: The sklearn estimator API contract (fit/transform/predict), preprocessing data with scalers and encoders, chaining steps into a Pipeline that prevents data leakage, applying different transformations to different columns with ColumnTransformer, and saving/loading trained models with joblib.
1.1 · The Estimator API
Every sklearn Object Follows the Same Contract
scikit-learn's power comes from a uniform API. Whether you're using a decision tree, a scaler, or a neural network, every sklearn object shares the same three methods:
| Method | What it does | Who has it |
|---|---|---|
| fit(X, y) | Learn parameters from training data. Stores state (e.g., mean/std for a scaler, coefficients for a model). | All estimators |
| transform(X) | Apply a learned transformation to data. Outputs a new dataset. | Transformers (scalers, encoders) |
| predict(X) | Use a fitted model to predict labels/values on new data. | Predictors (classifiers, regressors) |
| fit_transform(X, y) | Convenience: fit then transform in one call. Only safe on training data. | Transformers |
This contract is what makes Pipelines possible. Every step exposes the same interface, so sklearn can call them in sequence automatically.
The core pattern — fits on train, transforms both
PYTHON
from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.model_selection import train_test_split # 1. Split first — always X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=42 ) # 2. Fit scaler on TRAIN only scaler = StandardScaler() X_train_scaled = scaler.fit_transform(X_train) # fit + transform X_test_scaled = scaler.transform(X_test) # transform only (no fit!) # 3. Fit model on scaled train model = LogisticRegression() model.fit(X_train_scaled, y_train) # 4. Evaluate on scaled test accuracy = model.score(X_test_scaled, y_test) print(f"Accuracy: {accuracy:.3f}")
⚠️
The most common sklearn mistake
Never call
fit_transform(X_test) — that would compute a new mean/std from the test set, leaking information about it into your preprocessing. Always fit on training data only, then use transform() on everything else. This is why Pipelines matter so much.1.2 · Preprocessing
Scaling, Normalization, and Encoding
Most ML algorithms are sensitive to the scale of input features. A feature ranging 0–100,000 will dominate a feature ranging 0–1 unless you normalize. sklearn provides dedicated transformers for this.
StandardScaler vs MinMaxScaler
PYTHON
from sklearn.preprocessing import StandardScaler, MinMaxScaler # StandardScaler: mean=0, std=1 (z-score normalization) # Best for: algorithms assuming Gaussian distribution (LogReg, SVM, PCA) std_scaler = StandardScaler() X_std = std_scaler.fit_transform(X_train) # mean_ and scale_ attributes store what was learned: print(std_scaler.mean_) # per-feature means print(std_scaler.scale_) # per-feature std deviations # MinMaxScaler: rescales to [0, 1] # Best for: neural networks, image data, bounded ranges mm_scaler = MinMaxScaler() X_mm = mm_scaler.fit_transform(X_train) # Formula: (x - min) / (max - min)
For categorical features, you need encoders instead of scalers. The right choice depends on whether the categories have a natural order:
OneHotEncoder for nominal categories
PYTHON
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder import numpy as np colors = np.array([['red'], ['blue'], ['red'], ['green']]) # OneHotEncoder: creates a binary column per category # Use when categories have NO natural order (city, color, gender) ohe = OneHotEncoder(sparse_output=False) encoded = ohe.fit_transform(colors) # Result: [[0,0,1], [1,0,0], [0,0,1], [0,1,0]] # Columns: blue, green, red # OrdinalEncoder: integers 0..N-1 # Use when categories HAVE order (low/medium/high, cold/warm/hot) sizes = np.array([['small'], ['large'], ['medium']]) oe = OrdinalEncoder(categories=[['small', 'medium', 'large']]) oe.fit_transform(sizes) # → [[0.], [2.], [1.]]
ℹ️
Drop the first column to avoid multicollinearity
For linear models, pass
drop='first' to OneHotEncoder. With K categories, you only need K-1 columns (the last one is implied). Tree-based models don't care — skip the drop for them.1.3 · Pipelines
Chain Everything Together — Safely
A Pipeline bundles preprocessing steps and a model into a single object. The critical benefit: when you call pipeline.fit(X_train, y_train), sklearn automatically calls fit_transform on each preprocessing step using only the training data — no leakage possible. When you call pipeline.predict(X_test), it applies the transformations that were learned from training.
Basic Pipeline — scaler + model
PYTHON
from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.svm import SVC pipe = Pipeline([ ('scaler', StandardScaler()), ('classifier', SVC(kernel='rbf')) ]) # fit() calls fit_transform on scaler, then fit on SVC pipe.fit(X_train, y_train) # predict() calls transform on scaler, then predict on SVC preds = pipe.predict(X_test) score = pipe.score(X_test, y_test) # Access individual steps learned_scaler = pipe.named_steps['scaler'] print(learned_scaler.mean_)
Real datasets have mixed column types — some numeric, some categorical. ColumnTransformer applies different transformations to different subsets of columns, then stacks the results horizontally.
Full Pipeline with ColumnTransformer — the real-world pattern
PYTHON
from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.impute import SimpleImputer from sklearn.ensemble import RandomForestClassifier numeric_features = ['age', 'salary', 'score'] categorical_features = ['city', 'department'] # Numeric pipeline: fill missing → scale num_pipe = Pipeline([ ('imputer', SimpleImputer(strategy='median')), ('scaler', StandardScaler()) ]) # Categorical pipeline: fill missing → one-hot encode cat_pipe = Pipeline([ ('imputer', SimpleImputer(strategy='most_frequent')), ('encoder', OneHotEncoder(handle_unknown='ignore')) ]) # ColumnTransformer applies each sub-pipeline to its columns preprocessor = ColumnTransformer([ ('num', num_pipe, numeric_features), ('cat', cat_pipe, categorical_features) ]) # Final pipeline: preprocessing + model full_pipe = Pipeline([ ('preprocessor', preprocessor), ('model', RandomForestClassifier(n_estimators=100, random_state=42)) ]) full_pipe.fit(X_train, y_train) print(f"Test score: {full_pipe.score(X_test, y_test):.3f}")
💡
Pipelines work with cross_val_score too
Pass the whole pipeline as the estimator to
cross_val_score(full_pipe, X, y, cv=5). Each fold correctly fits the preprocessing only on training folds and applies it to the validation fold. This is the only leak-free way to cross-validate.1.4 · Saving & Loading Models
Persist Your Trained Pipeline
Training a model is expensive. joblib serializes fitted sklearn objects — including entire Pipelines — to disk. It's faster than Python's built-in pickle for large NumPy arrays (the learned weights/parameters) because it uses memory-mapped arrays.
Save and reload a trained pipeline
PYTHON
import joblib # Save the entire fitted pipeline (preprocessing + model) joblib.dump(full_pipe, 'model_v1.pkl') # Later — reload and predict on raw data # (the pipeline handles all preprocessing automatically) loaded_pipe = joblib.load('model_v1.pkl') predictions = loaded_pipe.predict(X_new_raw) # compress=3 shrinks file size ~5x for large models joblib.dump(full_pipe, 'model_v1_compressed.pkl', compress=3)
⚠️
sklearn version compatibility
A model pickled with sklearn 1.3 may not load correctly on sklearn 1.0. Always record the library versions when saving (
sklearn.__version__). In production, pin your sklearn version in requirements.txt and store it alongside the model file.🧩 Module 1 Check
5 questions · instant feedback
1. You have a fitted StandardScaler. Which method should you call on your test set?
2. Which sklearn class applies different transformers to different subsets of columns in one step?
3. What is the primary advantage of wrapping a scaler and model inside a Pipeline before cross-validation?
4. You have a feature "size" with values: small, medium, large. Which encoder is most appropriate?
5. Which library is recommended for saving sklearn models, and why is it preferred over Python's built-in pickle?