Which direction gives the largest quadratic value?

Source scope: §3.10.1. Worked numerical examples, experiments, and practice are teaching supplements unless identified as source results.

Keep length fixed, change direction

For $A=\operatorname{diag}(3,1)$ and a unit vector $x=(\cos\theta,\sin\theta)$ ,

x^TAx=3\cos^2\theta+\sin^2\theta.

The two nonnegative weights sum to one. Turning toward the first axis increases the weight on three. The quadratic value lies between one and three.

For real symmetric $A$ , $x^TAx$ is a quadratic form. Its Rayleigh quotient, defined only for $x\ne0$ , is

R_A(x)=\frac{x^TAx}{x^Tx}.

The source asks readers to verify two properties: for $\alpha\ne0$ , $R_A(\alpha x)=R_A(x)$ because numerator and denominator both scale by $\alpha^2$ ; and if $Ax=\lambda x$ , then $R_A(x)=\lambda x^Tx/(x^Tx)=\lambda$ .

Proposition 4: unit-vector bound and proof

Source statement. For $\|x\|_2=1$ ,

\lambda_{\min}(A)\le x^TAx\le\lambda_{\max}(A),

with equality at either end exactly when $x$ is an eigenvector of the corresponding eigenvalue.

Source proof, expanded. Decompose $A=Q\Lambda Q^T$ . Set $y=Q^Tx$ . This change is one-to-one and preserves unit length, so

\max_{\|x\|_2=1}x^TAx=\max_{\|y\|_2=1}y^T\Lambda y=\max_{\sum_i y_i^2=1}\sum_i\lambda_i y_i^2.

Let $I=\{i:\lambda_i=\lambda_{\max}(A)\}$ . The sum cannot exceed $\lambda_{\max}\sum_i y_i^2=\lambda_{\max}$ . Equality holds precisely when $y_j=0$ for every $j\notin I$ , so $\sum_{i\in I}y_i^2=1$ . Then the value is $\sum_{i\in I}\lambda_i y_i^2=\lambda_{\max}$ . Moreover,

x=QQ^Tx=Qy=\sum_i y_iq_i=\sum_{i\in I}y_iq_i.

Those columns form an orthonormal basis for the maximal eigenspace. Their nonzero combination is therefore an eigenvector for $\lambda_{\max}$ . Conversely every unit vector in that eigenspace attains the bound. Replacing “maximum” by “minimum” gives the lower bound and its equality condition. This includes repeated extreme eigenvalues; a single preferred eigenvector is not required.

Theorem 3: min-max characterization

For arbitrary $x\ne0$ , normalize by its length. Scale invariance and Proposition 4 give the source’s Min-max Theorem:

\lambda_{\min}(A)\le R_A(x)\le\lambda_{\max}(A),

with the same equality conditions. Equivalently,

\lambda_{\min}(A)=\min_{x\ne0}R_A(x),\qquad \lambda_{\max}(A)=\max_{x\ne0}R_A(x).

This is a variational characterization: an eigenvalue is described as the optimum of a scalar function of direction.

Explore it yourself

Turn a unit vector

Predict the quadratic value before aligning x with an eigenvector.

Preparing interactive controls…

Eigenvalue λ₁3

Eigenvalue λ₂1

Unit-vector angle (degrees)30

A = [[2.500, 0.866], [0.866, 1.500]]. xᵀAx = R(x) = 3.000; bounds [1, 3]. trace = 4.000; determinant = 3.000. Positive definite.

Axes: first and second real coordinates, in unit steps. Endpoints and labels identify each vector.

Explore in three dimensions

Explore it yourself

Search directions on a unit sphere

Move x over the sphere. Can xᵀAx leave the interval between the smallest and largest eigenvalues?

Preparing interactive controls…

Teaching supplement · interactive 3D projection · one grid interval = one coordinate unit

Change the mathematics

Direction longitude (degrees)45

Direction latitude (degrees)25

λ₁2

λ₂1

λ₃1

x = (0.641, 0.641, 0.423); ‖x‖ = 1.000. Ax = (1.399, 1.079, 0.423). R(x) = xᵀAx = 1.766. Eigenvalue bounds = [1.000, 2.000]. Q rotates the first two eigenvectors by 30°.

The blue wireframe is the unit sphere, not the image of that sphere under A. Red x stays on it; green Ax need not. At an eigenvector the two vectors share a line, or Ax vanishes for a zero eigenvalue. Equal extreme eigenvalues allow multiple maximizing directions.

Camera controls · change only the view

Drag the diagram with a mouse, or use these sliders with a keyboard or touch. Projection can make perpendicular directions look oblique; the coordinate readout is unchanged.

Camera azimuth (degrees)35

Camera elevation (degrees)25

Camera zoom0.9

Red: input or first direction. Blue: second direction or reference. Green: result. Amber: residual, normal, or third direction. Labels distinguish overlapping objects.

Transfer

If $A=\operatorname{diag}(4,4,1)$ , which unit vectors maximize the quadratic form?

Keep the whole maximal eigenspace

Every unit vector $(a,b,0)$ with $a^2+b^2=1$ attains four. Restricting the answer to the two coordinate axes would miss infinitely many maximizers.