Home

Notes on Linear Regression

The logistic least squares implements the logistic regression using the ordinary least squares as the regression method. The logistic regression provides the estimate of $P(y|X)$ which determines the probability that the data point is a member of either class $y \in \{ 0,1 \}$ for the predictor variables. $$ P(y = 1|X) = \frac{1}{\left( 1 + e^{(-\beta_0 + \sum_{k} \beta_k x_k )} \right)} $$ conversely $$ P(y = 0|X) = \frac{ e^{(-\beta_0 + \sum_k \beta_k x_k )} }{ { ( 1 + e^{ (-\beta_0 + \sum_{k} \beta_k x_k ) } ) } } $$ The logit function for the odds ratio is the transformation of the above and is estimated as $$ log \frac{P(y=0|X)}{P(y=1|X)} = \beta_0 + \beta^T X $$ The logistic regression proceeds by estimating the logit function, once estimated the class probability can be estimated using the logistic function.