Regression Techniques
By: Manoj Reddy
Tags: Ridge Regression, Regression Analysis, Logistic Regression, Linear Regression

When Regression is chosen?

Regression analysis is used once you want to predict dependent variable from variety of independent variables, like “salary” or “weight”. Many various models can be used, but linear regression is simpliest regression among all.

Regression Analysis is a predictive modelling technique. It estimates the relationship between a dependent and an independent variable. The conditional expectation of the criterions based on predictors where the average value of the dependent variables is given when the independent variables are changed. The major uses for regression analysis are determining the strength of predictors, forecasting an effect, and trend forecasting.

Types of Regression –

Linear Regression is employed for predictive analysis. Linear regression is a linear approach for modeling the connection between the dependent variable and the independent variable. It needs relationship between dependent variable and independent variable to be linear. The linearity assumptions are often tested by scatter plots. For linear regression , there’s a danger of overfitting.

Logistic Regression is similar as linear regression except, logistic regression produces results in binary format which is used to predict the outcome of a categorical dependent variable. So outcome should be discrete i.e true or false, yes or no, 0 or 1 instead of predicting something continuous like size. Logistic regression is usually used for classification.

Polynomial Regression is used for curvilinear data. Polynomial regression is fit with the method of least squares. Polynomial Regression is a regression algorithm that models the relationship between a dependent(y) and independent variable(x) as nth degree polynomial.

Stepwise regression will be used for fitting regression models with predictive models. It is carried out automatically. With each step, the variable is added or subtracted from the set of explanatory variables. The approaches for stepwise regression are forward selection, backward elimination.

Ridge regression may be a technique for analyzing multiple regression data. When multicollinearity occurs, least squares estimates are unbiased. Ridge regression uses ridge trace to determine how correlation between predictors affect coefficient estimates using k. A degree of bias is added to the regression estimates, ridge regression reduces the standard errors.

Lasso regression may be a regression analysis method that performs both variable selection and regularization. Lasso regression uses soft thresholding. Lasso regression selects only a subset of the provided covariates for use in the final model.

Elastic Net regression may be a regularized regression method that linearly combines the penalties to the loss function during training of the lasso and ridge methods. Elastic Net regression is used for support vector machines, metric learning, and portfolio optimization.

CONCLUSION:

This dealed with Regression Techniques, Types of Regression, when regression is used and what is regression. Hope you had a good read. Kindly share your views.