Vector Operations: Dot Product,
Norms & Projections

⏰ 24 min 📚 Beginner ➕ Math

Section 1

The One Operation That Shows Up Everywhere: The Dot Product

If this entire course track has one single most-used piece of math, it's the dot product. It's the operation underneath linear regression (Module 7), the core computation inside every neural network layer (Tier 3), and the way similarity gets measured in search and recommendation systems (Tier 2 and Tier 4). Understanding it well here pays off constantly later.

The dot product takes two vectors of the same length and combines them into a single number: multiply corresponding positions together, then add up all the results.

Dot product, by formula a \cdot b = a₁b₁ + a₂b₂ + a₃b₃ + ... + aₙbₙ

Worked example. Say a = [2, 3, 1] and b = [4, 0, 5]:

a · b = (2×4) + (3×0) + (1×5)
    = 8 + 0 + 5
    = 13

That single number, 13, is the dot product. It has a genuinely useful interpretation: it captures how much two vectors "agree" in direction, scaled by their magnitude. Two vectors pointing in similar directions produce a large positive dot product; vectors pointing in opposite directions produce a negative one; vectors at a perfect right angle produce exactly zero.

Section 2

Why the Dot Product Matters for ML, Specifically

In linear regression (Module 7), a model's prediction is literally a dot product: you take the feature vector for a house (square footage, bedrooms, etc.) and dot it with a "weights" vector the model has learned, producing a single predicted price. That's not a metaphor — it's the actual computation happening under scikit-learn's .predict() call.

💡

This is also exactly how recommendation systems (Tier 2, Module 9) measure how similar two users' tastes are, and how vector databases in Tier 4's RAG systems find the most relevant document for a question. Same operation, different application — this is a recurring theme in ML: a small number of core mathematical tools get reused across wildly different-looking problems.

Section 3

Vector Norms — "How Big Is This Vector?"

A norm measures a vector's length or magnitude. The most common one, the L2 norm (also called Euclidean norm), is the straight-line distance from the origin to the point the vector represents — it's just the Pythagorean theorem, extended to as many dimensions as you need.

L2 norm ||v|| = \sqrt(v₁² + v₂² + ... + vₙ²)

For v = [3, 4]: ||v|| = √(9 + 16) = √25 = 5. If you've ever used the 3-4-5 right triangle in school, this is the exact same calculation, just relabeled.

📊

Why Norms Matter Here

Regularization techniques (Tier 2, Module 7 and Tier 3, Module 4) directly penalize large vector norms in a model's learned weights — this is literally how L1/L2 regularization gets its name.

📊

Distance Between Points

The norm of the difference between two vectors, ||a − b||, gives you the distance between them — the exact calculation K-Means clustering (Tier 2, Module 4) uses to group similar points.

Section 4

Projections — Intuition Only

A projection answers: "if I have to represent vector a using only the direction of vector b, what's the closest I can get?" Picture shining a flashlight straight down onto vector a from directly above vector b — the shadow it casts onto b's line is the projection.

💡

You won't compute projections by hand often in this course, but the idea — "compressing information onto the most useful direction while losing as little as possible" — is precisely the intuition behind PCA (Principal Component Analysis), the dimensionality reduction technique you'll learn properly in Tier 2, Module 5. Keep this flashlight-and-shadow picture in mind; it'll make PCA click much faster later.

🎉 Lesson 2.2 complete! Next: matrix multiplication and why shapes matter.