Introduction

Logistic Regression is a fundamental classification algorithm in Machine Learning. Unlike Linear Regression, which predicts continuous values, Logistic Regression is used to predict categories such as yes/no, true/false, or 0/1.

In this lesson, you will learn how Logistic Regression works, its formula, and how it is used in real-world classification problems.

What is Logistic Regression?

Logistic Regression is a supervised learning algorithm used for classification problems.

It predicts the probability that a given input belongs to a particular category.

Example

Predict whether an email is spam or not spam.

Logistic Function (Sigmoid Function)

σ(z)=11+e−z\sigma(z) = \frac{1}{1 + e^{-z}}σ(z)=1+e−z1​

The sigmoid function converts any value into a range between 0 and 1.

Output Interpretation

• Value close to 1 → Class 1
• Value close to 0 → Class 0

How Logistic Regression Works

1. Takes input features
2. Applies a linear combination
3. Passes the result through the sigmoid function
4. Outputs a probability
5. Classifies based on a threshold (usually 0.5)

Logistic Regression Equation

z=w1x1+w2x2+…+bz = w_1x_1 + w_2x_2 + … + bz=w1​x1​+w2​x2​+…+b

The linear equation is passed into the sigmoid function to get probabilities.

Types of Logistic Regression

Binary Logistic Regression

Used for two classes
Example: Spam vs Not Spam

Multiclass Logistic Regression

Used for more than two classes
Example: Classifying types of fruits

Decision Boundary

Logistic Regression creates a decision boundary that separates different classes.

• If probability ≥ 0.5 → Class 1
• If probability < 0.5 → Class 0

Cost Function (Log Loss)

J=−1n∑[ylog⁡(y^)+(1−y)log⁡(1−y^)]J = -\frac{1}{n} \sum \left[y \log(\hat{y}) + (1 – y) \log(1 – \hat{y})\right]J=−n1​∑[ylog(y^​)+(1−y)log(1−y^​)]

This function measures how well the model performs in classification tasks.

Advantages of Logistic Regression

• Simple and easy to implement
• Works well for binary classification
• Provides probability outputs
• Efficient and fast

Limitations of Logistic Regression

• Assumes linear relationship
• Not suitable for complex datasets
• Sensitive to outliers

Implementation in Python

from sklearn.linear_model import LogisticRegression

X = [[1], [2], [3], [4]]
y = [0, 0, 1, 1]

model = LogisticRegression()
model.fit(X, y)

prediction = model.predict([[2.5]])
print(prediction)

Real-World Applications

• Email spam detection
• Disease prediction
• Customer churn prediction
• Credit risk analysis

Conclusion

Logistic Regression is a powerful yet simple classification algorithm. It is widely used in real-world applications and forms the foundation for more advanced classification techniques.

In the next lesson, you will learn about K-Nearest Neighbors (KNN), another important classification algorithm.

FAQs

What is Logistic Regression used for?

It is used for classification problems like spam detection and disease prediction.

What is the sigmoid function?

It is a function that converts values into probabilities between 0 and 1.

Is Logistic Regression linear or non-linear?

It is linear in terms of decision boundary but uses a non-linear sigmoid function.

What is a decision boundary?

It is a line or boundary that separates different classes.

Can Logistic Regression handle multiple classes?

Yes, it can be extended to multiclass classification.

Internal Link

To explore more courses and improve your skills, click here for more free courses