Regularization (L1 & L2)
Machine Learning · Intermediate
Definition
Regularization adds a penalty on large weights to the loss to reduce overfitting.
Keeps models simpler and better at generalizing.
Diagram
Flowchart
Example: Penalties
w, lam = [3, -4], 0.1
print(round(lam * sum(abs(x) for x in w), 2), round(lam * sum(x * x for x in w), 2))Output: 0.7 2.5
Key points
- L1 (Lasso) drives weights to zero → feature selection
- L2 (Ridge) shrinks weights smoothly
Interview questions
Effect of very large λ?
Weights shrink too much and the model underfits.