LinearRegression.md (1648B)
1 # Linear Regression 2 3 ML L2 - Also referred to as ordinary least squares 4 5 **Definition:** Fitting a straight line to data which allows for arbitrary inputs in the valid domain but not necessarily in the training set, to get accurate outputs. 6 7 The goal is to find a $\theta$ (parameters) that minimizes $J(\theta)=\frac{1}{2}\sum_{i=1}^{m}(h(x_i) - y_i)^2$. This is called the cost function. 8 9 To load linear regression using SkiLearn do as follows: 10 11 ```python3 12 13 from sklearn.linear_model import LinearRegression 14 model = LinearRegression() 15 16 ``` 17 18 Note that the constant term for a linear regression model is referred to as the bias term or the intercept term. 19 20 The normal equation for linear regression (closed-form solution) is as follows: 21 22 Theta = (X transpose * X) ^ -1 * X transpose * y 23 24 Where y is an m x 1 vector of target values and X is in some way related to inputs as a matrix with a column of ones for the intercept term... 25 26 This way of linear regression, the closed form way, is better when there are not a massive number of features, but if there are lots of features or the training instances aer too vast to fit into memory, then the [GradientDescent](GradientDescent.md) way is better. 27 28 See [RidgeRegression](RidgeRegression.md), [[LassoRegression.md]], and [[ElasticNetRegression.md]] for some ways to constrain linear models (decrease degrees of freedom to avoid overfitting). 29 30 As it relates to linear regression, it is good to add some regularization and when we know only a few features matter elastic regression is good. Otherwise, in most cases, ridge regression is a good option when we don't think there are useless features.