When is every nonzero direction uphill?

Source scope: §3.11. Worked numerical examples, experiments, and practice are teaching supplements unless identified as source results.

Check all directions

For $A=\operatorname{diag}(2,0)$ , the form is $2x_1^2$ . It is never negative, but the entire vertical direction has value zero. Replacing zero by one makes the value positive at every nonzero input.

A symmetric matrix is positive semidefinite, written $A\succeq0$ , if $x^TAx\ge0$ for every $x$ . It is positive definite, written $A\succ0$ , if $x^TAx>0$ for every nonzero $x$ . Positive definite implies positive semidefinite; the converse fails. These definitions use symmetry throughout.

Proposition 5: eigenvalue test

Source statement. A real symmetric matrix is positive semidefinite iff all eigenvalues are nonnegative, and positive definite iff all eigenvalues are positive.

Source proof. For an eigenvector $x$ with eigenvalue $\lambda$ ,

x^TAx=\lambda x^Tx=\lambda\|x\|_2^2.

Since $x\ne0$ , divide by its positive squared length. Nonnegativity of the form forces $\lambda\ge0$ ; strict positivity forces $\lambda>0$ . Conversely if all eigenvalues are nonnegative, the Rayleigh bound gives $R_A(x)\ge\lambda_{\min}\ge0$ for $x\ne0$ . The numerator has the same sign as the quotient, so the form is nonnegative, including zero input. If all eigenvalues are positive, the bound is strictly positive at nonzero inputs, proving positive definiteness.

Proposition 6: a common source of semidefinite matrices

For any $A\in\mathbb R^{m\times n}$ , $A^TA$ is positive semidefinite. It is positive definite if $\operatorname{null}(A)=\{0\}$ .

Source proof. Symmetry follows by transposing. For every $x$ ,

x^T(A^TA)x=(Ax)^T(Ax)=\|Ax\|_2^2\ge0.

If the nullspace is trivial, nonzero $x$ implies $Ax\ne0$ , giving strict positivity. Conversely, as a teaching observation, any nonzero null vector prevents strict positivity. A positive definite matrix is invertible because it has no zero eigenvalue; a positive semidefinite matrix may be singular.

Proposition 7: lift a flat direction

If $A\succeq0$ and $\epsilon>0$ , then $A+\epsilon I\succ0$ .

Source proof. For $x\ne0$ ,

x^T(A+\epsilon I)x=\underbrace{x^TAx}_{\ge0}+\underbrace{\epsilon\|x\|_2^2}_{>0}>0.

Combining Propositions 6 and 7 shows that $A^TA+\epsilon I$ is positive definite, hence invertible, for every real rectangular $A$ and $\epsilon>0$ . The positivity of $\epsilon$ is essential.

Explore it yourself

Turn a unit vector

Predict the quadratic value before aligning x with an eigenvector.

Preparing interactive controls…

Eigenvalue λ₁3

Eigenvalue λ₂1

Unit-vector angle (degrees)30

A = [[2.500, 0.866], [0.866, 1.500]]. xᵀAx = R(x) = 3.000; bounds [1, 3]. trace = 4.000; determinant = 3.000. Positive definite.

Axes: first and second real coordinates, in unit steps. Endpoints and labels identify each vector.

Explore in three dimensions

Explore it yourself

A bowl, a flat direction, or a saddle?

Change the eigenvalues of a two-variable quadratic form. Then add εI and watch both curvatures shift.

Preparing interactive controls…

Teaching supplement · interactive 3D projection · one grid interval = one coordinate unit

Change the mathematics

Base eigenvalue λ₁2

Base eigenvalue λ₂1

Diagonal shift ε0

f(x,y) = 2.000 u² + 1.000 v², where (u,v) = Qᵀ(x,y) and Q turns by 30°. Shifted eigenvalues = (2.000, 1.000). Positive definite: a bowl.

The green surface samples −0.8 ≤ x,y ≤ 0.8. To fit the view its vertical coordinate is f(x,y)/2; x and y remain in ordinary units. When positive definite, the blue base-plane curve is the contour f = 1, including portions outside the sampled square. Adding ε shifts both eigenvalues; an indefinite matrix needs a large enough shift.

Camera controls · change only the view

Drag the diagram with a mouse, or use these sliders with a keyboard or touch. Projection can make perpendicular directions look oblique; the coordinate readout is unchanged.

Camera azimuth (degrees)35

Camera elevation (degrees)25

Camera zoom0.9

Red: input or first direction. Blue: second direction or reference. Green: result. Amber: residual, normal, or third direction. Labels distinguish overlapping objects.

Transfer

Does adding $0.1I$ make $\operatorname{diag}(-2,1)$ positive definite?

Check the hypothesis

No. The shifted eigenvalues are $-1.9$ and $1.1$ . Proposition 7 starts with a positive semidefinite matrix. For an arbitrary symmetric matrix, the shift must exceed the negative of its smallest eigenvalue.