MODULE 3 — SUPPORT VECTOR MACHINES
3.4 — When to Use SVMs vs Trees vs Linear Models
A practical decision guide rather than a vague "it depends."
| Family | Wins on |
|---|
| SVMs | Small-to-medium datasets where quadratic/cubic training cost is manageable · high-dimensional sparse data (text classification, TF-IDF vectors is the classic example) · problems needing a clean, margin-maximized decision boundary |
| Trees / ensembles | Tabular data with mixed types (numeric, categorical, missing values handled natively by some implementations) · large N — scales much better than kernel SVMs · interpretability requirements (single trees readable directly, ensembles offer feature importance / SHAP) |
| Linear models | Very large datasets where speed matters more than capturing nonlinearity · cases where the relationship is genuinely close to linear (or made so via feature engineering) · regularized regression (Ridge/Lasso) giving prediction plus feature-selection-like effects for free |
The meta-lesson: there's rarely a universally "best" algorithm. Match the algorithm's assumptions and computational profile to your data's shape and your constraints, rather than defaulting to whatever performed best on the last unrelated project.
A quick mental checklist
- How many rows? Tens of thousands+ and a kernel SVM is already getting slow — lean trees or linear.
- How many features, and how sparse? High-dimensional and sparse (text) → SVM territory.
- Mixed types, missing values? Trees handle this most gracefully out of the box.
- Do you need to explain individual predictions to a non-technical stakeholder? Trees and linear models are easier to justify than SVMs.
- Do you suspect the true relationship is close to linear? Don't reach for a complex model just because it's available — start linear, and only add complexity once you've confirmed it's needed.
Takeaways
- Dataset size and dimensionality are usually the first filter — they rule options in or out before accuracy even enters the conversation.
- Interpretability requirements often matter as much as raw performance in production settings.
- This is the close of Module 3 — Module 4 shifts from supervised to unsupervised learning, starting with K-Means.