Saving & Loading Models with Pickle & Joblib
.fit(), get a model object, and the moment the script ends, it's gone. Section 5 is about turning a trained model into something that actually SHIPS. This lesson is step one: persisting a trained model to disk with Python's built-in pickle module and with joblib, scikit-learn's recommended alternative — so training happens once, and every prediction after that just loads the saved file.
Why You Need to Persist a Trained Model
Training a model is often the most expensive part of the whole ML workflow — it can mean minutes of gradient boosting on a large dataset, or a grid search from Lesson 11 trying dozens of hyperparameter combinations. None of that work should happen again just because a web server restarted or a script got run twice. The trained model — its learned coefficients, tree splits, or cluster centroids — needs to be saved somewhere so it can be reloaded instantly.
.fit() every time.The general idea is called serialization — converting a live Python object sitting in memory (a fitted RandomForestRegressor, a Pipeline, a StandardScaler) into a sequence of bytes that can be written to a file, and later read back into an equivalent live object. Python has two common ways to do this for ML work: the standard-library pickle module, and joblib.
Python's pickle Module
pickle ships with Python itself — no extra install needed. It can serialize almost any Python object (lists, dicts, custom classes, and fitted scikit-learn models) into a binary format using pickle.dump(), and reconstruct that object later using pickle.load(). Files must be opened in binary mode — "wb" to write, "rb" to read — because pickle produces binary data, not text.
import pickle from sklearn.linear_model import LogisticRegression from sklearn.datasets import load_iris X, y = load_iris(return_X_y=True) model = LogisticRegression(max_iter=200) model.fit(X, y) # Save the fitted model to disk — note the "wb" (write, binary) mode with open("iris_model.pkl", "wb") as f: pickle.dump(model, f) print("Model saved to iris_model.pkl")
import pickle # Load the model back — "rb" (read, binary) mode with open("iris_model.pkl", "rb") as f: loaded_model = pickle.load(f) # The loaded object behaves exactly like the original fitted model prediction = loaded_model.predict(X[:1]) print(f"Prediction: {prediction}")
.pkl files that your own training pipeline produced, or that come from a source you fully trust — the same caution applies to joblib files in Section 3, since joblib uses a similar underlying mechanism for arbitrary Python objects.joblib — The Scikit-learn-Recommended Alternative
joblib is a separate package (installed automatically as a scikit-learn dependency, or via pip install joblib) that's specifically optimized for objects containing large NumPy arrays — exactly what a fitted scikit-learn model is full of: coefficient arrays, tree structures, cluster centroids. For that kind of object, joblib is generally more efficient to write and read than plain pickle, which is why the scikit-learn documentation itself recommends it for persisting models. The API is deliberately simple — joblib.dump() and joblib.load() — and unlike pickle, it doesn't require manually opening a file object.
import joblib from sklearn.ensemble import RandomForestClassifier from sklearn.datasets import load_iris X, y = load_iris(return_X_y=True) model = RandomForestClassifier(n_estimators=100, random_state=42) model.fit(X, y) # joblib.dump() takes the object and a filename directly — no "with open()" needed joblib.dump(model, "iris_model.joblib") print("Model saved to iris_model.joblib")
import joblib loaded_model = joblib.load("iris_model.joblib") prediction = loaded_model.predict(X[:1]) print(f"Prediction: {prediction}")
compress=3 (an integer from 0–9) to joblib.dump(model, "iris_model.joblib", compress=3) shrinks the saved file size, trading a little extra CPU time on save/load for a smaller file — genuinely useful for large ensemble models with many trees.pickle vs. joblib — When to Use Which
Both work for saving a scikit-learn model, and both carry the same "don't load untrusted files" caveat from Section 2. The practical difference is mostly about efficiency and convenience for NumPy-array-heavy objects.
| Aspect | pickle | joblib |
|---|---|---|
| Availability | Built into Python standard library | Separate package (ships as a scikit-learn dependency) |
| Best for | General Python objects — dicts, lists, custom classes | Objects with large NumPy arrays — fitted sklearn models especially |
| Efficiency on big models | Works, but less optimized for large arrays | Generally faster & more memory-efficient for large arrays |
| API style | with open(...) as f: pickle.dump(obj, f) | joblib.dump(obj, "file.joblib") — no manual file handling |
| Built-in compression option | Manual (e.g. wrap with gzip) | Built in via compress= parameter |
In practice: for a scikit-learn Pipeline, RandomForestClassifier, or any other fitted model from Sections 2–4 of this course, reach for joblib first — it's what the scikit-learn documentation itself recommends, and it's what this lesson's worked example and Lesson 22's API both use.
Full Worked Example — Train, Save, Load, Predict
A realistic version of this workflow rarely saves a bare model on its own — it saves the WHOLE pipeline, preprocessing included. Here's a small "Student Score Predictor" example (features: study hours, attendance, previous score, assignments completed → predicted final score) that this course reuses again in Lesson 22's API and Lesson 24's capstone.
| StudentID | StudyHours | Attendance% | PrevScore | AssignmentsDone | FinalScore |
|---|---|---|---|---|---|
| 1 | 4.5 | 92 | 78 | 9 | 84 |
| 2 | 1.2 | 65 | 52 | 4 | 49 |
| 3 | 6.0 | 98 | 88 | 10 | 93 |
| 4 | 2.8 | 74 | 61 | 6 | 60 |
import pandas as pd import joblib from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.ensemble import RandomForestRegressor from sklearn.pipeline import Pipeline # 1. Load data df = pd.read_csv("student_scores.csv") features = ["StudyHours", "Attendance", "PrevScore", "AssignmentsDone"] X = df[features] y = df["FinalScore"] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # 2. Build ONE pipeline — scaler and model saved together as a single object pipeline = Pipeline([ ("scaler", StandardScaler()), ("model", RandomForestRegressor(n_estimators=200, random_state=42)) ]) pipeline.fit(X_train, y_train) print(f"Test R²: {pipeline.score(X_test, y_test):.3f}") # 3. Save the WHOLE pipeline with joblib joblib.dump(pipeline, "student_score_model.joblib") print("Pipeline saved to student_score_model.joblib")
scaler and model were saved as two separate files, a later mistake — loading the wrong scaler version, or forgetting to apply it at all — would silently produce wrong predictions. Wrapping both in a single sklearn.pipeline.Pipeline and saving THAT one object removes the mismatch risk entirely: loading it back always applies scaling and prediction together, in the right order.import joblib import pandas as pd # Load the saved pipeline — this is often a totally separate script/process pipeline = joblib.load("student_score_model.joblib") # A brand-new student, never seen during training new_student = pd.DataFrame([{ "StudyHours": 5.0, "Attendance": 88, "PrevScore": 70, "AssignmentsDone": 8 }]) predicted_score = pipeline.predict(new_student)[0] print(f"Predicted final score: {predicted_score:.1f}")
Notice that load_and_predict.py never imports RandomForestRegressor or re-fits a StandardScaler — the loaded pipeline object already knows how to do both, because it was saved AFTER fitting. This is exactly the mechanism Lesson 22's FastAPI app relies on to serve predictions.
Versioning & Naming Considerations
A file called model.joblib works fine for a single experiment, but it breaks down the moment there's more than one trained version around — which happens almost immediately in any real project. A few practical habits keep this manageable:
student_score_model_v1.joblib, _v2.joblib — simple, greppable, and it's immediately obvious which file a deployed API is pointing at.model_v1_metadata.json) recording the feature names/order, training date, and library versions used — makes it possible to sanity-check a model months later.features list) — document that order next to the saved file, not just in a script that might change later.The Complete Script, Start to Finish
Training, saving, and a fresh-process load-and-predict, combined into one reference script.
import pandas as pd import joblib from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.ensemble import RandomForestRegressor from sklearn.pipeline import Pipeline # 1. Load & split df = pd.read_csv("student_scores.csv") features = ["StudyHours", "Attendance", "PrevScore", "AssignmentsDone"] X, y = df[features], df["FinalScore"] X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) # 2. Fit a pipeline pipeline = Pipeline([("scaler", StandardScaler()), ("model", RandomForestRegressor(n_estimators=200, random_state=42))]) pipeline.fit(X_train, y_train) print(f"Test R²: {pipeline.score(X_test, y_test):.3f}") # 3. Save with joblib, versioned filename MODEL_PATH = "student_score_model_v1.joblib" joblib.dump(pipeline, MODEL_PATH) # 4. Simulate a fresh process: load and predict loaded_pipeline = joblib.load(MODEL_PATH) new_student = pd.DataFrame([{"StudyHours": 5.0, "Attendance": 88, "PrevScore": 70, "AssignmentsDone": 8}]) print(f"Predicted score: {loaded_pipeline.predict(new_student)[0]:.1f}")
Use the pipeline, X_test, and y_test objects from Section 5/7 as your starting point.
After saving and reloading
pipeline with joblib, compare pipeline.predict(X_test) (before saving) against loaded_pipeline.predict(X_test) (after loading) using numpy.allclose(). Confirm they match perfectly.
Using Python's
datetime module, build a filename like student_score_model_20260811.joblib that embeds today's date, and save the pipeline under that name instead of a fixed _v1.
Save the same fitted
pipeline once with pickle and once with joblib (try compress=3 too). Use Python's os.path.getsize() to compare the three resulting file sizes on disk.
💡 Show hints if you're stuck
- Task 1:
import numpy as np; np.allclose(pred_before, pred_after)should returnTrue. - Task 2:
from datetime import date; f"student_score_model_{date.today():%Y%m%d}.joblib". - Task 3:
import os; os.path.getsize("student_score_model_v1.joblib")returns a size in bytes.