独家优惠奖金 100% 高达 1 BTC + 180 免费旋转

Logistic Regression

After learning the fundamentals of regression, it’s time to learn the fundamentals of classification. And what could be simpler than Logistic Regression!

It is recommended that you read this Linear Regression Tutorial before beginning this tutorial. This is a complete guide to Linear Regression, and Logistic Regression uses several ideas that are related to Linear Regression. With that said, let’s begin!

It is a classification algorithm that is applied in situations when the output variable is categorical. The goal of Logistic Regression is to discover a relationship between features and the probability of a specific outcome.

For example,

Email: Spam / Not Spam?

Online Transaction: Fraudulent (Yes/No)

Here,

In this post, we will build a logistic regression model to predict whether a student gets admitted into a university.

In short Linear Regression, plots all the data onto a graph (of x and y), fits all the data to a best-fit line, and then makes predictions for inputs as the corresponding y. Logistic Regression on the other hand fits all the data to an S-curve and there are only two possible outputs (two classifications), which are represented as the top and bottom lines.

The S curve does not refer to the shape of the letter S; instead, it stands for the sigmoid function. This is because the sigmoid function perfectly meets our purpose of categorising data into two groups. The sigmoid formula is as follows, where x is the number of inputs.

In English, the sigmoid is just the calculation of probability based on the weighted sum of the input features. The formula for the weighted sum is as follows,

Before coding up our sigmoid function, let us initialize our dataset.

Here ‘0’ represents students mark in 1ˢᵗ test, ‘1’ represents students mark in 2ⁿᵈ test and ‘2’ represents whether a student gets admitted(1) or not(0).

Let us visualize our data,

In the case of Logistic regression the Hypothesis (h) is represented by the following equation:

Pictorial representation of hypothesis:

From the above figures we can tell that,

Since our data set has two features: test1 and test2 the logistic regression hypothesis is the following:

The logistic regression classifier will predict “Admitted” if:

This is because the logistic regression “threshold” is set at g(z)=0.5, see the plot of the logistic regression function above for verification.

Let’s start by defining the logistic regression cost function for the two points of interest: y=1, and y=0, that is, when the hypothesis function predicts Admitted or Not Admitted,

Simplified cost function is as follows,

Then, we take a convex combination in y of these two terms to come up with the logistic regression cost function:

Before using the data to compute cost, we should normalize our data(To know more about normalization Click here.)

Logistic Regression’s gradient descent algorithm will look identical to Linear Regression’s gradient descent algorithm. For the case of gradient descent, the search direction is the negative partial derivative of the logistic regression cost function with respect to the parameter θ. In its most basic form, gradient descent will iterate along the negative gradient direction of θ (known as a minimizing sequence) until reaching convergence.

Here, the decision boundary is as follows:

From the above output, it shows that the student with marks 45 and 85 has 80% probability of getting admitted in the university.

Today, we saw the concepts behind hypothesis, cost function, and gradient descent of Logistic regression. Then it was created from scratch using python’s numpy, pandas and matplotlib. The dataset and final code is uploaded in github.

If you enjoyed my article, a clap 👏 and a follow would be absolute badass and it is helpful for medium to promote this article so that others may read it. I am Jagajith and I will catch you in the next one.

Add a comment

What is your view?

Send

Related posts:

Implementing the Simple FB Connect Module!

Hello dear readers! I’m Bagha, a participant in the Google Code-In competition. I’ve chosen the Drupal organization and have recently accepted a task which involve completing the title above. Here is…