A supervised ML algorithm for classification — output is categorical (usually binary). Unlike Linear Regression, it predicts the probability of a class label and maps it to a discrete output via the Sigmoid function. Coefficients are estimated using Maximum Likelihood Estimation (MLE). Input = Independent variable (X) · Output = Categorical label Y ∈ {0, 1}.
ShapeS-curve (not straight line like Linear Regression)
Log-Odds (Logit Function) — log[ P(Y=1) / P(Y=0) ] = β₀ + β₁X₁ + … + βₚXₚ · This transformation maps probabilities (0–1) onto the full real line (−∞, +∞), creating a linear relationship between input features and the log-odds of the outcome. Logistic Regression is linear in log-odds space.
Evaluation metrics & confusion matrix
1
Accuracy
% of all correctly predicted instances.
(TP + TN) / (TP+TN+FP+FN)
Misleading on imbalanced data.
2
Precision
Of all predicted positives, how many were correct?
TP / (TP + FP)
Use when false positives are costly.
3
Recall
Of all actual positives, how many were caught?
TP / (TP + FN)
Use when false negatives are costly.
4
F1 Score
Harmonic mean of Precision & Recall.
2 × P×R / (P + R)
Best for imbalanced datasets.
Confusion Matrix
Predicted Positive
Predicted Negative
Actual Positive
TP True Positive
FN False Negative (Type II)
Actual Negative
FP False Positive (Type I)
TN True Negative
■ Correct predictions
■ Incorrect predictions
Linear regression vs logistic regression
Aspect
Linear Regression
Logistic Regression
Key Difference
Output
Continuous (e.g., price)
Probability → class label
LR predicts a number; LogR predicts a category
Function
Y = β₀ + β₁X (line)
σ(z) = 1/(1+e⁻ᶻ) (curve)
Straight line vs S-shaped sigmoid curve
Range
−∞ to +∞
0 to 1
LogR output is bounded — interpretable as probability
Loss Function
MSE / Least Squares
Log Loss / Cross-Entropy
Optimization method differs fundamentally
Use Case
Regression tasks
Classification tasks
Choose based on whether target is continuous or categorical