How much can a matrix amplify a vector?

Source scope: §3.14, PDF pages 22, 23. Numerical examples, experiments, added derivations, and practice are teaching supplements unless identified as source results.

Size depends on the question

For $A=\begin{bmatrix}1&2\\0&1\end{bmatrix}$, the maximum absolute column sum is $3$, and the maximum absolute row sum is also $3$. Its Frobenius norm is $\sqrt6$. These are different ways to measure one matrix, not inconsistent answers.

For a linear map $T:V\to W$ between normed spaces, the source defines

Technical clarification. For arbitrary normed spaces use a supremum, which may be infinite for an unbounded linear map. In the finite-dimensional matrix setting below the maximum exists: normalize to a compact unit sphere and maximize a continuous function.

For $A\in\mathbb R^{m\times n}$ using the same vector $p$-norm in both spaces, $p\ge1$,

The first two are the maximum absolute column and row sums. Added justification: the triangle inequality bounds $\|Ax\|_1$ by the largest column sum times $\|x\|_1$; a signed coordinate vector attains that column sum. For the infinity norm, bound every output coordinate by its row sum times $\|x\|_\infty$. Choose the signs of $x_j$ to match a maximizing row to attain equality.

The source derives the spectral norm by squaring a nonnegative ratio:

The last expression is a Rayleigh quotient. Its maximum is $\lambda_{\max}(A^TA)=\sigma_1^2$, attained by a top right-singular vector. Taking the square root gives $\|A\|_2=\sigma_1$. In the example, $A^TA=\begin{bmatrix}1&2\\2&5\end{bmatrix}$, so $\|A\|_2=\sqrt{3+2\sqrt2}=1+\sqrt2$.

Bound a chain of maps

By definition $\|Ax\|_p\le\|A\|_p\|x\|_p$, including $x=0$.

Proposition 8. For compatible matrices, $\|AB\|_p\le\|A\|_p\|B\|_p$.

Source proof. For every $x$, apply the amplification bound twice:

Divide by $\|x\|_p$ for $x\ne0$, then maximize. The right side after division is constant, giving the stated bound.

Count all squared entries

The Frobenius norm (written $\|\cdot\|_f$ in the PDF) is

Expanding the diagonal gives $\operatorname{tr}(A^TA)=\sum_j\sum_iA_{ij}^2$, proving the first equality. For the second, the source uses

The spectral norm measures the greatest stretch; the Frobenius norm aggregates all squared stretches.

Which sizes ignore rotations?

A norm is unitarily invariant when $\|UAV\|=\|A\|$ for appropriately sized orthogonal $U,V$ in this real setting. Consequently $\|A\|=\|U\Sigma V^T\|=\|\Sigma\|$: only singular values matter.

Proposition 9. The spectral and Frobenius norms are unitarily invariant.

Source proof for Frobenius. Orthogonality and cyclic trace give

Take square roots. Source proof for spectral norm, with corrected substitution:

Here $y=Vx$ preserves length and is a bijection carrying nonzero vectors to nonzero vectors. The PDF’s final sentence writes $y=V^Tx$; that does not match the displayed $AVx$. We correct that substitution explicitly.

Transfer

For a diagonal matrix with singular values $4,3$, the spectral norm is $4$ and the Frobenius norm is $5$. Rotating its input or output preserves both values. The induced 1- and infinity-norms generally change under rotations.

Explore the relationship