Extrema

Extrema, or extreme points, in multivariable calculus are similar to those in single variable calculus, except they're even prettier. At least if we don't go beyond . If we add more dimensions, they start getting really hard to visualize at all, so we'll stay in two variables for most of this topic.

The theory extends to higher dimensions, and in some cases the techniques we'll show do as well. Unfortunately, though, investigating extrema turns out to be a bit more complicated in several variables than in one.

There are three types of extrema. A point in the domain of is an extrema if and only if:

We will see more of all these three types in the coming lecture notes. And a small reminder: do not forget to check the boundary as you look for extrema. It's a classical but unnecessary mistake.

As we classify critical points, we talk about local and global min and max points, like in single variable calculus.

The function has a local maximum at in the domain of , if for all points in the domain that are sufficiently close to . If the inequality holds for all the points in the domain of , then it's a global maximum.

The graph below has a global maximum at the point , and a local maximum at :

Local and global minima are defined in the same way, just switching to in the definition.

Do all functions have extrema? No. There are a few requirements which need to be met.

Sufficient conditions for the existence of a maximum and minimum value of a function are:

Then, the range of will be a bounded set as well, and there exist points in the domain for which will have global max and min values.

These requirements are quite natural. Try to come up with counter-examples for each one of them, where the global max or min would be undefined.

A hint for the first one: what happens around if ?

Singular points are points where the gradient is not defined. These can be points where is discontinuous, like here:

This is the function . In the case of discontinuous functions, it may, as we said in the last note, be that the function does not have a global max or min.

Specifically, the function above goes to infinity as goes to . So there exists no global maximum.

Singular points can also be points where is not differentiable. One example is the at bottom of the cone made up by

As we cannot take the partial derivatives there, we need to check the function value at this point separately to determine if it's a local or global max or min.

For a function , the critical points come in three versions: maxima, minima and saddle points. A point is a critical point if:

This means both partial derivatives must be zero at .

At a critical point, the tangent plane is horizontal. This is true for maxima:

... and for minima:

It also goes for saddle points. This type of critical point is somewhat more loosely defined; basically, all that is neither maximum nor minimum but still has a zero-gradient is a saddle point.

Below, the typical saddle point is displayed on the right, together with a more dubious fellow on the left, which actually has an endless line of saddle points.

If we want to know where the critical points of are, we are often not satisfied before we've also determined which type of critical point we've found. After all, there is a big difference in knowing we have a mountain to climb or a valley to stroll down into.

There are different ways of doing the classification. One is to proceed as follows: knowing that the function has a critical point at , study the function difference:

As we take a small step away from the point, does the function increase or decrease?

If is positive for all , the function increases in all directions around the point, and we're dealing with a minimum. If is negative, is decreasing in all directions, and the point must be a maximum. If the function is decreasing in some directions and increasing in others, we're standing face to face with a saddle point.

This method is brute force, and if the function is not nice, it may be hard to conclude without doubt that what we've found isn't a saddle point.

Classify the critical point in of the function

We begin by finding

since can be both negative or positive for different values of and , is a saddle point.

If not tempted by the idea of analysing the function behaviour in all directions around some point, our saviour is the second derivative test.

Like in single variable calculus, the sign of the second derivative of a function tells what concavity the function has. However, as there are multiple second derivatives, the concavity is harder to read.

In two variables, the second derivative test gives us the answer. For understanding it, and calculating the concavity in higher dimensions, we'll need some serious linear algebra to help us out with the business.

Today, we'll content ourselves with stating the formula for the aforementioned second derivative test. Given all the second partial derivatives of a function , plug them into the following formula:

Now, say that has a critical point at . Calculate . If

The following lecture note relies heavily on concepts from linear algebra, so feel free to skip it if you are not comfortable with it. That being said, we will here build a bridge between calculus and linear algebra which can be useful both practically and to aid your understanding.

Recall the second derivative test from the previous lecture note. Here, we examined the sign of the following function in order to classify critical points:

There is no coincidence that we use the letter to denote this function. The reason is that it stands for a determinant. In particular, the one of a matrix called the Hessian matrix.

The determinant for a -matrix is given by:

Now let us see what happens as we insert the second partial derivatives of the function as the elements in a matrix this way:

What we get is the Hessian matrix of a function in two variables. Assuming that all second partial derivatives are continuous, we have that

so that the determinant becomes

Hence, what we used as our function to classify critical points of a function at a point according to the second derivative test is the determinant of the Hessian matrix at .

Now the reason this technique works is because the Hessian matrix, with its elements being all second partial derivatives, contains all the necessary information for the task, which is the function's concavity in the various directions.

This can be useful not only to classify critical points, but to examine the concavity at any other point as well. For example when sketching a surface in 3D.

The Hessian matrix exists for twice differentiable functions of any number of variables, with it looks as follows:

Now the Hessian matrix can be used to classify critical points in any dimension, but it is not quite as simple as to look at the determinant. Instead, we have the following:

Recall from linear algebra that a matrix is