Introduction In this post, we will explore frequentist and Bayesian analogues of regularized/penalized linear regression models (e.g., LASSO [L1 penalty], Ridge regression [L2 penalty]), which are an extention of traditional linear regression models of the form:
[y = \beta_{0}+X\beta + \epsilon\tag{1}] where (\epsilon) is the error, which is normally distributed as:
[\epsilon \sim \mathcal{N}(0, \sigma)\tag{2}] Unlike these traditional linear regression models, regularized linear regression models produce biased estimates for the (\beta) weights.