🐼 Section 2 · Pandas
🟡 Intermediate
MODULE 11
GroupBy, Aggregation & Pivot Tables
Your progress in Section 267%
🎯 What you'll learn: How
.groupby() works through the split-apply-combine pattern, the standard aggregation functions (.sum(), .mean(), .count()), running several aggregations at once with .agg(), building a spreadsheet-style summary with pd.pivot_table(), and counting category frequencies with .value_counts().
Section 1
GroupBy Mechanics: Split-Apply-Combine
.groupby() is arguably the most powerful single method in Pandas — it's how you answer questions like "what's the average sales per region?" in one line. It works in three conceptual stages:
1
Split
The DataFrame is broken into groups, one per unique value in the column(s) you group by — every "North" row goes in one bucket, every "South" row in another, and so on.
2
Apply
A function (like
sum or mean) runs independently on each group's data.3
Combine
The per-group results are stitched back together into a single new DataFrame or Series, indexed by the group labels.
groupby_basics.py
PYTHON
import pandas as pd df = pd.DataFrame({ "region": ["North", "South", "North", "East", "South", "North"], "product": ["Laptop", "Mouse", "Monitor", "Keyboard", "Webcam", "Mouse"], "sales": [1200, 45, 890, 65, 150, 55], }) grouped = df.groupby("region")["sales"].sum() print(grouped) # region # East 65 # North 2145 # South 195 # Name: sales, dtype: int64
📝
Nothing happens until you apply a function
df.groupby("region") by itself returns a DataFrameGroupBy object — Pandas has planned the split but hasn't computed anything yet. It's only once you chain on .sum(), .mean(), or another aggregation that the apply-and-combine steps actually run.Section 2
Aggregation Functions
Any function that reduces a group of values down to a single summary number works after .groupby() — the same statistical functions from NumPy and Section 1, plus a few Pandas-specific ones.
agg_functions.py
PYTHON
df.groupby("region")["sales"].sum() # total sales per region df.groupby("region")["sales"].mean() # average sales per region df.groupby("region")["sales"].count() # number of rows per region df.groupby("region")["sales"].min() # smallest sale per region df.groupby("region")["sales"].max() # largest sale per region # Grouping by MULTIPLE columns — one group per unique (region, product) pair df.groupby(["region", "product"])["sales"].sum() # Grouping without picking a column first summarizes every numeric column df.groupby("region").mean(numeric_only=True)
⚠️
count() vs. size()
.groupby("region")["sales"].count() counts non-null values in that column per group. .groupby("region").size() counts rows per group regardless of missing values. If a group has NaNs in the column you're aggregating, the two numbers can differ — worth remembering when counts don't match what you expect.Section 3
Running Multiple Aggregations with .agg()
Rather than call .sum(), then .mean(), then .count() separately, .agg() computes several statistics for a group in a single pass, and can apply different functions to different columns.
agg_method.py
PYTHON
# Multiple functions on ONE column summary = df.groupby("region")["sales"].agg(["sum", "mean", "count"]) print(summary) # sum mean count # region # East 65 65.000000 1 # North 2145 715.000000 3 # South 195 97.500000 2 # DIFFERENT functions on DIFFERENT columns, using a dict summary = df.groupby("region").agg({ "sales": ["sum", "mean"], "product": "count", }) # Naming the resulting columns yourself summary = df.groupby("region")["sales"].agg( total_sales=("sum"), avg_sales=("mean"), )
✨
.agg() replaces a lot of repeated code
Instead of building three separate Series with three separate
.groupby() calls and joining them together, .agg([...]) or .agg({...}) gets you one tidy summary DataFrame directly.Section 4
pd.pivot_table()
A pivot table reshapes data into a spreadsheet-style summary — one set of values becomes row labels, another set becomes column labels, and the cells hold an aggregated value. It's conceptually a groupby with a reshaping step built in.
pivot_table.py
PYTHON
# regions become rows, products become columns, cells = total sales pivot = pd.pivot_table( df, values="sales", index="region", columns="product", aggfunc="sum", fill_value=0, # show 0 instead of NaN where a region/product pair never occurs ) print(pivot) # product Keyboard Laptop Monitor Mouse Webcam # region # East 65 0 0 0 0 # North 0 1200 890 55 0 # South 0 0 0 45 150
pivot_table(values="sales", index="region", columns="product", aggfunc="sum")
| region | Keyboard | Laptop | Monitor | Mouse | Webcam |
|---|---|---|---|---|---|
| East | 65 | 0 | 0 | 0 | 0 |
| North | 0 | 1200 | 890 | 55 | 0 |
| South | 0 | 0 | 0 | 45 | 150 |
📝
pivot_table vs. groupby
df.groupby(["region", "product"])["sales"].sum() would compute the exact same numbers, but stacked into a tall, multi-index Series. pd.pivot_table() lays that same result out wide, like a real spreadsheet pivot table — the choice is about which shape is more useful for what comes next.Section 5
.value_counts()
For a quick "how many of each category do I have?" question on a single column, .value_counts() is faster to write than a full groupby.
value_counts.py
PYTHON
df["region"].value_counts() # region # North 3 # South 2 # East 1 # Name: count, dtype: int64 — sorted descending by default df["region"].value_counts(normalize=True) # as proportions instead of raw counts (sums to 1.0)
✨
value_counts() is a shortcut for a common groupby
df["region"].value_counts() is functionally equivalent to df.groupby("region").size().sort_values(ascending=False) — but shorter to write, and the go-to first check when exploring any categorical column.Section 6
Lesson Summary
Let's recap everything you learned in this lesson:
✅.groupby() follows split → apply → combine — nothing computes until you chain on an aggregation.
✅
.sum(), .mean(), .count(), .min()/.max() summarize each group.✅
.agg([...]) or .agg({...}) runs several aggregations at once.✅
pd.pivot_table() reshapes a groupby-style summary into a wide, spreadsheet-style table.✅
.value_counts() is the fast path for counting category frequencies in one column.🧩 Knowledge Check — Lesson 11
Answer all 5 questions to test your understanding. Instant feedback on every answer.
1. What are the three stages of the groupby pattern?
2. What does
df.groupby("region") return, on its own, with no aggregation chained on?