Significance of the derivative
By: Adrià Serra
Tags: Machine Learning, Artificial Intelligence, Deep Learning, Calculus, Data Science

CALCULUS DERIVATIVES

Significance of the derivative

The process of finding critical values

After derivative theory posts, we will start to see some of the applications that make this technique one of the most important in mathematics and therefore at machine learning.

Maximum and Minimum of a function

The theorem of the local maximum that we already introduced, can be extrapolated to the whole function domain.

Let f be a function and A a set of numbers contained in the domain of f. A point x in A is a maximum point for f on A if f(x)≥f(y) for every y in A. The number f(x) itself is called the maximum value of f on A.

Notice that there can be multiple maximum points for the same function at distinct x values. The minimum point definition is obtained inverting the previous definition, ergo changing f(x)≥f(y) for f(x)≤f(y).

Derivatives at maximum and minimum points

As you can expect, maximum and minimum points will always be a change in the derivative of the function, that allows us to demonstrate that:

Let f be any function defined on (a,b). If f is a maximum or a minimum point for f on (a,b), and f is differentiable at x, then f’(x)=0.

Local maximums and minimums

Let f be a function, and A a set of numbers contained in the domain of f. A point x in A is a local maximum[minumum] point for f on A of there is some δ > 0 such that x is a maximum[minumum] point for f on A ⋂ (x-δ,x+δ ).

Critical points

Not all x that makes f’(x) = 0 will be maximums or minimums, they are types of critical points:

A critical point of a function f is a number x such that f’(x)=0. The number f(x) is called a critical value of f.

In order to find the maximum and minimum of f, we have to check the following values:

The critical points of f in [a,b] .

. The endpoints, a and b .

and . Points x in [a,b] such that f is not differentiable at.

Some important theorems for critical point detection

The Rolle’s Theorem

If f is continuous on [a,b] and differentiable on (a,b), and f(a)=f(b), then there is a number x in (a,b) such that f’(x)=0.

This can lead to a constant function or to a function that changes the gradient between both values, so f’(x)=0.

The mean value theorem

Rolle theorem allows us to demonstrate the following:

If f is continuous on [a,b] and differentiable on (a,b), then there is a number x in (a,b) such that

Mean value theorem, self-generated.

Classifying critical points

Increasing and decreasing functions

A function f is increasing on an interval if f(a)<f(b) whenever a and be are two numbers in the interval with a<b. The function f is decreasing on an interval if f(a) > f(b) for all a and b in the interval with a<b.

Second derivatives for critical point classification

Now we know how to find a critical point with the first derivative and check the type of them using the left and right derivative values, we can use the second derivative to skip the lateral limit calculations.

Suppose f’(a)=0. If f’’(a)>0, then f has a local minimum at a; if f’’(a)<0 the nf has a local maximum at a.

Two strong theorems for simplification

The Cauchy Mean value theorem

If f and g are continuous on [a,b] and diffrentiable on (a,b), then there is a number x such that [f(b)-f(a)]g’(x)=[g(b)-g(a)]f’(x). If g(b)≠g(a), and g’(x)≠0, this can be written as:

Cauchy mean value theorem, self-generated.

L’Hôpital’s rule

L’hôpital Rule, self-generated.

Conclusion

In this post, we introduced how to use derivatives to find local maximums and minimums, they allow us to find possible solutions to our cost function optimization. This will allow us to determine the gradient easier.