1.3 — Building a Full Pipeline Object

A Pipeline chains a sequence of steps — typically several transformers followed by one final estimator — into a single object exposing the same fit / predict interface as any other estimator.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('clf', LogisticRegression())
])
pipe.fit(X_train, y_train)
pipe.predict(X_test)

Calling pipe.fit runs fit_transform on every step except the last, then fit on the last. Calling pipe.predict runs transform on every step except the last, then predict on the last — automatically respecting the fit/transform split from lesson 1.2.

ColumnTransformer for mixed-type data

Most real datasets mix numeric columns (needing scaling) with categorical columns (needing encoding). ColumnTransformer applies different transformers to different column subsets and concatenates the results into one feature matrix:

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder

preprocessor = ColumnTransformer([
    ('num', StandardScaler(), numeric_cols),
    ('cat', OneHotEncoder(handle_unknown='ignore'), categorical_cols)
])

full_pipe = Pipeline([
    ('preprocess', preprocessor),
    ('clf', LogisticRegression())
])

This is normally the first step inside a larger pipeline — the model never sees a mismatch between numeric and categorical handling because ColumnTransformer resolves it before the estimator is reached.

Why pipelines matter beyond convenience

They make cross-validation safe by construction. If you preprocess your whole dataset once and then run cross-validation, each fold's "test" portion was preprocessed using statistics that included it — that's leakage. If you instead put preprocessing inside the pipeline and cross-validate the pipeline, scikit-learn refits the preprocessing from scratch on each fold's training portion alone.

from sklearn.model_selection import cross_val_score

scores = cross_val_score(full_pipe, X, y, cv=5)
# Each fold refits preprocessor + model on that fold's train split only

Takeaways

• Pipeline chains transformers + a final estimator into one object with a uniform fit/predict interface.
• ColumnTransformer handles mixed numeric/categorical data inside a single preprocessing step.
• Cross-validating a pipeline (not pre-transformed data) is the only way to guarantee no leakage across folds.