When does a stationary point really minimize?

Source scope: §4.7, PDF pages 29, 30. Numerical examples, experiments, added derivations, and practice are teaching supplements unless identified as source results.

Zero slope is a candidate, not an answer

The origin is stationary for $x^2+y^2$ , $x^2-y^2$ , $x^3$ , and $-x^4$ . Their behavior differs: a minimum, a saddle, a stationary inflection, and a strict maximum. Conditions must distinguish them without reading too much into a zero derivative.

All results below concern unconstrained interior points. Gradients and Hessians are taken in an open neighborhood. A constrained boundary minimum such as $x^2$ at $x=1$ on $[1,3]$ need not have zero gradient.

First-order necessity

Proposition 13. If $x^*$ is a local minimum and $f$ is continuously differentiable near $x^*$ , then $\nabla f(x^*)=0$ .

Source proof. Suppose instead $\nabla f(x^*)\ne0$ and set $h=-\nabla f(x^*)$ . Continuity gives

\lim_{t\to0}-\nabla f(x^*+th)=h,\qquad \lim_{t\to0}h^T\nabla f(x^*+th)=-\|h\|_2^2<0.

Thus there is $T>0$ such that $h^T\nabla f(x^*+th)<0$ for $t\in[0,T]$ , inside the neighborhood. By Taylor’s mean-value formula, for every sufficiently small $t\in(0,T]$ there is $t'\in(0,t)$ with

f(x^*+th)=f(x^*)+t h^T\nabla f(x^*+t'h)<f(x^*).

These lower values occur arbitrarily close to $x^*$ , contradicting local minimality. Therefore the gradient vanishes.

This proof explains a descent direction: when $\nabla f(x)\ne0$ , a sufficiently small positive $\alpha$ gives $f(x-\alpha\nabla f(x))<f(x)$ . Points with zero gradient are stationary points. They need not be extrema. For the source’s $f(x,y)=x^2-y^2$ , the origin minimizes the $y=0$ slice and maximizes the $x=0$ slice, so it is neither a local minimum nor a local maximum. It is a saddle point.

Second-order necessity

Proposition 14. If $x^*$ is a local minimum and $f$ is twice continuously differentiable near it, then $\nabla^2f(x^*)$ is positive semidefinite.

Source proof. If it were not, there would be a direction $h$ with $h^T\nabla^2f(x^*)h<0$ . Continuity gives

\lim_{t\to0}\nabla^2f(x^*+th)=\nabla^2f(x^*),\qquad \lim_{t\to0}h^T\nabla^2f(x^*+th)h<0.

Choose $T>0$ so this directional curvature stays negative for $t\in[0,T]$ within the neighborhood. Proposition 13 gives $\nabla f(x^*)=0$ . For any sufficiently small $t\in(0,T]$ , second-order Taylor gives a $t'\in(0,t)$ such that

f(x^*+th)=f(x^*)+t h^T\underbrace{\nabla f(x^*)}_{0} +\tfrac12t^2h^T\nabla^2f(x^*+t'h)h<f(x^*).

Again this contradicts local minimality. Hence the Hessian is positive semidefinite.

A sufficient neighborhood condition

Proposition 15. Suppose $f$ is twice continuously differentiable, $\nabla^2f$ is positive semidefinite throughout a neighborhood of $x^*$ , and $\nabla f(x^*)=0$ . Then $x^*$ is a local minimum. If $\nabla^2f(x^*)$ is positive definite, then it is a strict local minimum.

Source proof. Choose an open ball $B_r(x^*)$ contained in the neighborhood. For $\|h\|_2<r$ , Taylor gives some $t\in(0,1)$ with

f(x^*+h)=f(x^*)+h^T\underbrace{\nabla f(x^*)}_{0} +\tfrac12h^T\nabla^2f(x^*+th)h\ge f(x^*).

The intermediate point stays in the ball because $\|th\|_2=t\|h\|_2<r$ for $h\ne0$ , and the $h=0$ case is equality. The Hessian there is positive semidefinite, proving the inequality and local minimality.

If $\nabla^2f(x^*)$ is positive definite, continuity lets us choose a smaller ball $B_{r'}(x^*)$ where every Hessian is positive definite. The same calculation gives $f(x^*+h)>f(x^*)$ for $0<\|h\|_2<r'$ , proving strict local minimality. Added detail: the smallest eigenvalue at $x^*$ is some $\mu>0$ ; continuity makes the operator norm of the Hessian change smaller than $\mu/2$ nearby, so every unit directional curvature remains at least $\mu/2$ .

Why checking one semidefinite Hessian fails

For $f(x)=x^3$ , $f'(0)=0$ and the one-by-one Hessian is $[f''(0)]=[0]$ , positive semidefinite. Yet negative inputs give negative values and positive inputs give positive values: zero is no extremum. The source calls this a saddle; in one-variable calculus it is commonly called a stationary inflection point.

For $f(x)=-x^4$ , the gradient and Hessian also vanish at zero, but $f(x)<0=f(0)$ for all $x\ne0$ . Zero is a strict local maximum. These are both source counterexamples. Their Hessians, $6x$ and $-12x^2$ , fail to remain positive semidefinite in any open neighborhood of zero.

The neighborhood condition in Proposition 15 can be hard to verify computationally. Sometimes one proves the Hessian positive semidefinite for all inputs analytically. Positive definiteness at the point, together with continuity of the Hessian, already ensures a positive definite neighborhood. Strict minima need not have a positive definite Hessian: $x^4$ at zero is an added counterexample to that converse.

Transfer

For $f(x,y)=x^4+y^2$ , the gradient vanishes at zero and the Hessian there is $\operatorname{diag}(0,2)$ . Its Hessian is $\operatorname{diag}(12x^2,2)$ everywhere, so Proposition 15 guarantees a local minimum. Directly, $x^4+y^2>0$ away from zero, proving it is strict even though the Hessian at zero is not positive definite.

Explore the relationship

Explore it yourself

Test stationary points

Can zero slope and zero curvature distinguish a minimum from a maximum?

Preparing interactive controls…

Slice direction in the xy-plane (degrees)0

Gradient at 0 = 0. Hessian at 0: diag(2, 2).

f(−0.1v) = 0.01; f(0) = 0; f(0.1v) = 0.01. Positive definite Hessian: strict minimum.

The graph is the slice t ↦ f(tv) for unit v = (cos θ, sin θ). Sampling alone does not prove local minimality; use the lesson's neighborhood conditions.

Check your reasoning

Make a prediction

What guarantees a local minimum for a C\u00b2 function at an interior point?

Answer: A zero gradient and a PSD Hessian throughout a neighborhood.. Taylor samples curvature along short segments, so neighborhood semidefiniteness controls the remainder.