Logistic Regression for Binary Classification

Building a logistic regression model from Scratch in Python using gradient descent

Photo by Scott Graham on Unsplash

Logistic regression is a commonly used algorithm in machine learning for binary classification problems. Let’s walk through the steps of building a logistic regression model from scratch in Python.

First, we will define the logistic function and then move on to implementing the algorithm using gradient descent. Finally, we will train the model on a dataset and evaluate its performance.

Let’s begin.

Defining the Logistic Function

The logistic function, also known as the sigmoid function, is a function that maps any real-valued number to a value between 0 and 1. It is commonly used in logistic regression to model the probability of a binary outcome. The logistic function is defined as:

$$\sigma(z) = \frac{1}{1 + e^{-z}}$$

where $z$ is the input value.

Let’s define the logistic function in Python:

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

Implementing the Algorithm with Gradient Descent