Imagine you're predicting house prices. Two features:
Without scaling, a distance-based model like k-NN or SVM will treat a 1-unit difference in area as equivalent to a 1-unit difference in rooms — but 1 m² is nothing while 1 room is huge. The model is dominated by whichever feature has the largest absolute values.
The most common scaler. For each feature, it subtracts the mean and divides by the standard deviation:
After scaling: every feature has mean = 0 and standard deviation = 1. Values are now in units of "how many standard deviations from the mean."
from sklearn.preprocessing import StandardScaler import numpy as np X_train = np.array([[25], [50], [75], [150]]) X_test = np.array([[40], [100]]) scaler = StandardScaler() scaler.fit(X_train) # learn μ and σ from train X_train_sc = scaler.transform(X_train) X_test_sc = scaler.transform(X_test) # use SAME μ and σ print(scaler.mean_) # [75.] print(scaler.scale_) # [47.17] print(X_train_sc) # [[-1.06] [-0.53] [ 0. ] [ 1.60]]
When to use it: When you have no strong outliers and your data is roughly bell-shaped. Default choice for most models.
Scales each feature to a specific range, usually [0, 1]:
Use when: your model needs values in [0, 1] specifically — neural networks with sigmoid outputs, image pixel values (0–255 → 0–1).
Weakness: One extreme outlier changes the scale for every other value. A salary of $10M in a dataset of $40k–$150k salaries compresses everything else into a tiny range near zero.
Uses the median and the interquartile range (IQR) instead of mean and std. Outliers have much less influence:
Use when: your data has meaningful outliers you can't (or shouldn't) remove — income data, fraud amounts, medical measurements.
| Scaler | Centers on | Scale unit | Outlier resistant | Best for |
|---|---|---|---|---|
| StandardScaler | Mean | Std deviation | No | Most models (default) |
| MinMaxScaler | Min value | Value range | No | Neural nets, image data |
| RobustScaler | Median | IQR | Yes | Data with real outliers |
| None | — | — | — | Tree-based models (RF, XGBoost) |
Models work with numbers. A column containing "red", "green", "blue" needs to become numbers. There are two main approaches, and choosing wrong causes subtle bugs.
Assigns an integer to each category: small=0, medium=1, large=2. Only use this when the order is real and meaningful.
from sklearn.preprocessing import OrdinalEncoder import numpy as np # ✅ CORRECT: size has a real order X = np.array([['small'], ['large'], ['medium'], ['small']]) enc = OrdinalEncoder(categories=[['small', 'medium', 'large']]) enc.fit(X) print(enc.transform(X)) # [[0.] [2.] [1.] [0.]] ← small=0, medium=1, large=2 # ❌ WRONG: color has no order — encoding 'red'=0, 'green'=1, 'blue'=2 # tells the model green > red, blue > green. False signal.
Creates one binary column per category. "color" with values red/green/blue becomes three columns: is_red, is_green, is_blue. Each row has exactly one 1 and the rest 0.
from sklearn.preprocessing import OneHotEncoder import numpy as np X = np.array([['red'], ['green'], ['blue'], ['red']]) enc = OneHotEncoder(sparse_output=False) enc.fit(X) X_encoded = enc.transform(X) print(enc.categories_) # [['blue', 'green', 'red']] print(X_encoded) # [[0. 0. 1.] ← red # [0. 1. 0.] ← green # [1. 0. 0.] ← blue # [0. 0. 1.]] ← red # drop='first' avoids dummy variable trap (multicollinearity) enc2 = OneHotEncoder(drop='first', sparse_output=False) enc2.fit(X) print(enc2.transform(X)) # [[0. 1.] # blue is the dropped reference category # [1. 0.] # [0. 0.] # [0. 1.]]
drop='first' or drop='if_binary' to drop one column. Tree-based models don't need this.
This is the single most common mistake in ML preprocessing. It produces results that look good during development but fail in production.
from sklearn.preprocessing import StandardScaler scaler = StandardScaler() # ❌ Bug: fitting on all data before splitting X_scaled = scaler.fit_transform(X_all) # sees test data! X_train, X_test = train_test_split(X_scaled, ...) # ❌ Bug: fitting scaler on test set scaler.fit(X_test) # overwrites train stats! X_test_scaled = scaler.transform(X_test)
from sklearn.preprocessing import StandardScaler from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2) scaler = StandardScaler() scaler.fit(X_train) # ✅ learn stats from TRAIN ONLY X_train_sc = scaler.transform(X_train) # ✅ apply to train X_test_sc = scaler.transform(X_test) # ✅ apply to test with SAME stats # Or even better: use a Pipeline — it handles this automatically
In real data you have both numeric and categorical columns. ColumnTransformer applies different preprocessing to each column type, then a Pipeline wraps everything:
from sklearn.pipeline import Pipeline from sklearn.compose import ColumnTransformer from sklearn.preprocessing import StandardScaler, OneHotEncoder from sklearn.ensemble import RandomForestClassifier import pandas as pd # Example dataframe data = pd.DataFrame({ 'age': [25, 32, 47, 19], 'salary': [40000, 75000, 95000, 30000], 'city': ['London', 'Paris', 'London', 'Berlin'], 'dept': ['Eng', 'HR', 'Eng', 'Finance'], }) y = [1, 0, 1, 0] # Define which columns get which treatment numeric_features = ['age', 'salary'] categorical_features = ['city', 'dept'] # Build a transformer for each type numeric_transformer = StandardScaler() categorical_transformer = OneHotEncoder(handle_unknown='ignore') # ColumnTransformer applies each transformer to its columns preprocessor = ColumnTransformer(transformers=[ ('num', numeric_transformer, numeric_features), ('cat', categorical_transformer, categorical_features), ]) # Wrap preprocessor + model in a single Pipeline pipe = Pipeline([ ('preprocessor', preprocessor), ('model', RandomForestClassifier(random_state=42)) ]) pipe.fit(data, y) print(pipe.predict(data)) # [1 0 1 0]
handle_unknown='ignore' makes the encoder output all zeros for unknown categories instead of crashing.
city column: London, Paris, Berlin (no natural order). Which encoder should you use?fit() on training data only. transform() on everything. A Pipeline makes this automatic.