Matrix Operations: Multiplication,
Transpose & Inverse

⏰ 26 min 📚 Beginner ➕ Math

Section 1

Matrix Multiplication — The Rule That Trips Everyone Up Once

Matrix multiplication is not what most people guess on first encounter (multiplying each matching position together, like you would with two vectors). It's a different, specific operation, and getting the rule straight now will save you from a very common bug later: shape mismatch errors, which are probably the single most common error message you'll see across every lab in this entire course track.

⚠️

The rule: to multiply matrix A (shape m×n) by matrix B, B's number of rows must equal A's number of columns (n). The result has shape m×p, where p is B's number of columns. If the inner numbers don't match, the multiplication is simply undefined — not "wrong," literally impossible.

Section 2

A Worked Example

Take a small 2×2 matrix A and a 2×1 vector b (a vector is just a matrix with one column):

A = [ 1  2 ]      b = [ 5 ]
    [ 3  4 ]          [ 6 ]

A × b = [ (1×5 + 2×6) ]   = [ 17 ]
        [ (3×5 + 4×6) ]     [ 39 ]

Notice what actually happened: each row of A got dot-producted with the column of b. That's the entire rule for matrix multiplication — it's just a grid of dot products. If Lesson 2.2's dot product made sense, you already understand the mechanics here; this is that same operation, repeated systematically across rows and columns.

💡

This exact A × b pattern — a matrix of data multiplied by a vector of weights — is how linear regression makes predictions for an entire dataset of houses all at once, instead of looping through each house individually. This is the computational trick that makes ML on large datasets fast.

Section 3

The Transpose

The transpose flips a matrix over its diagonal — rows become columns, columns become rows. Written as A^T.

A = [ 1  2  3 ]        Aᵀ = [ 1  4 ]
    [ 4  5  6 ]              [ 2  5 ]
                              [ 3  6 ]

A shape of (2, 3) becomes (3, 2) after transposing. This shows up constantly in practice for a very mundane reason: you'll frequently have data in one orientation but need it in the other to satisfy a matrix multiplication's shape rule from Section 1. When you hit a shape-mismatch error in a lab later in this course, "try transposing one of the matrices" is a genuinely common, correct first thing to check.

Section 4

The Inverse — Intuition Over Computation

For regular numbers, the inverse of 5 is 1/5, because 5 × (1/5) = 1. Matrices have a similar idea: the inverse of matrix A, written A⁻¹, is the matrix that "undoes" A — multiplying A by A⁻¹ gives you the identity matrix (the matrix equivalent of the number 1).

💡

You will almost never compute a matrix inverse by hand in real ML work — NumPy and scikit-learn do it for you, and for large matrices, computing an exact inverse is often avoided entirely in favor of more numerically stable alternatives. What matters here is the concept: solving "what weights make this equation true" in linear regression is mathematically equivalent to using an inverse (or an inverse-like operation) to isolate the unknown weights.

⚠️

One important fact worth knowing: not every matrix has an inverse. A matrix without one is called "singular," and you'll occasionally see this exact word in an error message when a model's underlying math hits a numerical issue — usually caused by features in your dataset that are perfect duplicates or perfectly correlated with each other.

🎉 Lesson 2.3 complete! Next: eigenvalues and eigenvectors.