Blog

UNDERSTANDING LOGISTIC REGRESSION

DATE: April 9, 2017

  • Logistic Regression is used to model the outcomes of a categorical target variable
  • Input features are scaled just as with linear regression, however result is fed as an input to the logistic function.
  • In linear regression, coefficients are found by minimizing the sum of squared error terms
  • In logistic regression, we do this by maximizing the likelihood of the data
  • The likelihood of an observation is the probability of seeing that observation in a particular model. This is called MAXIMUM LIKELIHOOD ESTIMATION

Interpreting Coefficients in Logistic Regression:

  • Similar to interpretation of coefficients in Linear Regression
  • A unit increase in feature Xi results in multiplying the odds ratio by an amount
  • When a coefficient βi is positive, the we multiply the odds ratio by a number greater than 1, so increasing the feature Xi will effectively increase the probability of the output being labeled as class 1.
  • Increasing the feature with a negative coefficient shifts the balance toward predicting the class as 0.

Where Logistic Model differs from Linear Model:

  • When we change the value of an input feature, the effect is a multiplication on the odds ratio and not on the model output itself, which is the probability of predicting the class 1.
  • The change In the output of our model as a result of a change in the input is not constant throughout, but depends on the current value of our input features.
  • This is different from linear regression, where no matter what the values of the input features, the regression coefficients always represent a fixed increase in the output per unit increase of an input feature.