The cost function for linear regression is simple: it calculates the average difference between predicted and actual values. However, using the same cost function for logistic regression leads to non-convex functions with many local minimums.
For logistic regression, a specific cost function exists:
Cost(f(x), y) = -y*log(f(x)) - (1-y)*log(1-f(x)),
where h(x) represents the predicted probability that the output y is 1 given the input x and model parameters.
It is important to note that y is always either 0 or 1.
If y=1, we use only -log(f(x)), which approaches 0 as f(x) approaches 1.
If y=0, we use -log(1-f(x)), which approaches 0 as f(x) approaches 0.
Despite its complexity, the cost function is still straightforward. Furthermore, by using this function, we can create a convex function that we can optimize using gradient descent.

The cost function for linear regression is simple: it calculates the average difference between predicted and actual values. However, using the same cost function for logistic regression leads to non-convex functions with many local minimums.

For logistic regression, a specific cost function exists:

Cost(f(x), y) = -y\*log(f(x)) - (1-y)\*log(1-f(x)),

where h(x) represents the predicted probability that the output y is 1 given the input x and model parameters.

It is important to note that y is always either 0 or 1.

If y=1, we use only -log(f(x)), which approaches 0 as f(x) approaches 1.

If y=0, we use -log(1-f(x)), which approaches 0 as f(x) approaches 0.

Despite its complexity, the cost function is still straightforward. Furthermore, by using this function, we can create a convex function that we can optimize using gradient descent.

Cost function for logistic regression