01 Key Concepts
Loss Function
A function that measures how wrong a model's predictions are compared to the actual values; training a model means trying to minimize this function.
Gradient Descent
An optimization algorithm that repeatedly adjusts a model's parameters in the direction that most reduces the loss function, using its gradient (derivative).
Learning Rate
A small constant that controls how big each step is during gradient descent; too large risks overshooting, too small makes training very slow.
Vectors and Matrices in ML
Data (like an image or a row of features) is typically represented as a vector, and datasets as matrices, so that models can use fast matrix operations to make predictions.
Overfitting
When a model learns the training data too precisely, including its noise and quirks, causing it to perform poorly on new, unseen data.
02 Key Formulas
- Gradient descent update: parameter_new = parameter_old - (learning_rate * gradient)
03 Solved Examples
- Apply the update rule: w_new = w_old - (learning_rate * gradient).
- = 5 - (0.1*4).
- A learning rate too large can cause the algorithm to overshoot the minimum and even diverge.
- A learning rate too small can make training take an impractically long time to converge.
- A large gap between training performance and new-data performance is a classic warning sign.
- The model has likely memorized quirks specific to the training set rather than learning generalizable patterns.
04 Practice Questions
📄 Machine Learning Mathematics — Downloadable Worksheet
10 questions with a full answer key. Grab the PDF to print, or try the interactive version in your browser.