?embed=1 for iframe mode We start with a simple binary dataset: study hours (x) mapped to pass/fail labels (0/1). Logistic regression is built for this kind of yes/no prediction.
What kind of target does logistic regression model?
A straight regression line can output any value, including numbers below 0 or above 1. Those numbers cannot be interpreted as probabilities.
Why is plain regression unsuitable here?
Instead of raw scores, we want a probability between 0 and 1 for each input. That probability is later converted to a class decision.
The sigmoid squashes any input into the 0–1 range. Its S-shape makes it perfect for turning linear combinations into probabilities.
Our model predicts p = 1 / (1 + exp(-(w₀ + w₁ × x))). The weights w₀ and w₁ shift and tilt the curve.
What does w₁ control in the logistic model?
With weights near zero, the model predicts ~0.5 for every x. That makes poor decisions because it ignores the structure of the data.
Cross-entropy penalises confident wrong predictions heavily. This pushes the model to assign high probability to the correct class.
What happens to loss when the model is confidently wrong?
Gradients show how to adjust w₀ and w₁ to reduce loss. Each update nudges the sigmoid to better separate the classes.
After many updates, the sigmoid aligns with the data: low x predicts fail (0), high x predicts pass (1).
The decision boundary is the x where p = 0.5. Points to the right are classified as 1; to the left as 0.
What probability marks the decision boundary?
By sampling different x-values you can see how the predicted probability climbs smoothly from 0 toward 1.
To classify, we threshold the probability: if p ≥ 0.5, predict class 1; otherwise predict class 0.
How do we convert probability to a class?
Accuracy tells us what fraction of labels we got right. The confusion matrix counts correct positives/negatives and mistakes.
On an XOR-style dataset, labels alternate along x. A single sigmoid can only make one boundary, so it misclassifies alternating sections.
Because logistic regression uses one boundary, it cannot follow multiple flips between 0 and 1. Misclassifications stay even after training.
Why does one sigmoid fail on XOR?
To handle shapes with multiple boundaries we need models like kNN that adapt locally instead of forcing a single global split.