Foundations of Machine LearningRidge, Lasso, ElasticNet & The Bias-Variance Tradeoff
Written Guide21 minutes

Ridge, Lasso, ElasticNet & The Bias-Variance Tradeoff

Lesson Summary: Combating overfitting through L1/L2 penalty constraints, feature sparsity, and cross-validation.

Overfitting occurs when a model captures random noise in training data rather than true underlying relationships. Regularization constrains model capacity:

  • Ridge Regression (L2): Adds λ∑θj2\lambda \sum \theta_j^2 penalty; shrinks weights smoothly without setting them strictly to zero.
  • Lasso Regression (L1): Adds λ∑∣θj∣\lambda \sum |\theta_j| penalty; produces sparse models by driving coefficients to zero (automatic feature selection).
Previous
Gradient Descent, Convex Optimization & Learning Rates
Next Lesson
Interactive Loss Surface & Gradient Descent Optimizer Lab