Why is the perpendicular foot the closest point?

Source scope: §3.5.3. Worked numerical examples, experiments, and practice are teaching supplements unless identified as source results.

Separate what a line explains

Let $S$ be the horizontal axis and $v=(2,3)$. The component in $S$ is $(2,0)$; the remaining component is $(0,3)$. Moving along the line cannot remove that vertical residual.

For any subset $S$ of an inner product space $V$, define

Even if $S$ is not a subspace, $S^\perp$ is: zero is orthogonal to everything, and inner-product linearity preserves the zero conditions under addition and scaling.

Proposition 1: unique orthogonal decomposition

Source statement. If $S$ is a finite-dimensional subspace of $V$, every $v\in V$ can be written uniquely as $v=v_S+v_\perp$, with $v_S\in S$ and $v_\perp\in S^\perp$.

Source proof, with steps expanded. Choose an orthonormal basis $u_1,\ldots,u_m$ for $S$. Set

The first vector belongs to $S$ by its span. For each basis vector,

Linearity extends orthogonality from the basis to all of $S$. For uniqueness, take another decomposition, including one constructed from a different orthonormal basis. Then $v_S-v'_S=v'_\perp-v_\perp$ lies in both spaces. Therefore

Thus $v_S=v'_S$, and subtracting from $v$ gives $v_\perp=v'_\perp$. Consequently $V=S\oplus S^\perp$ under the stated finite-dimensional-subspace assumption. The source follows its proposition with “whenever S is a subspace”; retain the finite-dimensional hypothesis here rather than extending that sentence to arbitrary infinite-dimensional subspaces.

Proposition 2: eight properties and their proofs

Define the orthogonal projection $P=P_S:V\to S$ by $Pv=v_S$.

1. Basis formula: $Pv=\sum_i\langle v,u_i\rangle u_i$ for any orthonormal basis of $S$. This is the construction above; uniqueness makes it independent of basis.
2. Residual: $v-Pv\perp S$. This was proved by taking its inner product with every basis vector.
3. Linearity: Write $x=x_S+x_\perp$ and $y=y_S+y_\perp$. Their sum splits as $(x_S+y_S)+(x_\perp+y_\perp)$ in the two subspaces; uniqueness gives $P(x+y)=Px+Py$. Likewise $\alpha x=\alpha x_S+\alpha x_\perp$, giving $P(\alpha x)=\alpha Px$.
4. Identity on S: For $s\in S$, $s=s+0$ is its decomposition, so $Ps=s$.
5. Range and nullspace: The definition gives $\operatorname{range}(P)\subseteq S$; property 4 gives the reverse inclusion by choosing input $s$. If $Pv=0$, its decomposition gives $v=v_\perp\in S^\perp$. Conversely $v\in S^\perp$ has decomposition $0+v$ and maps to zero. Hence $\operatorname{range}(P)=S$ and $\operatorname{null}(P)=S^\perp$.
6. Idempotence: $P^2v=P(Pv)=Pv$ because $Pv\in S$ and property 4 applies. Thus $P^2=P$.
7. No length increase: Pythagoras gives $\|v\|^2=\|Pv\|^2+\|v-Pv\|^2\ge\|Pv\|^2$. Take nonnegative square roots.
8. Unique closest point: For $s\in S$, the residual is perpendicular to $Pv-s\in S$. Thus

Taking square roots proves $\|v-Pv\|\le\|v-s\|$. Equality requires $\|Pv-s\|^2=0$, equivalently $s=Pv$. Therefore $Pv=\operatorname*{arg\,min}_{s\in S}\|v-s\|$.

Any linear map satisfying $P^2=P$ is called a projection. Orthogonal projections have the additional perpendicular-residual property; idempotence alone does not establish it. The experiment replaces the source’s closest-point sketch with the same point, subspace, foot, and residual, plus a movable candidate.

Write the projection as a matrix

For $S\subseteq\mathbb R^n$ and orthonormal columns $u_i$ assembled in $U$,

Hence $P_S=\sum_i u_i u_i^T=UU^T$, using the sum-of-outer-products identity. Orthonormality matters; arbitrary basis columns do not give this formula.

Explore in three dimensions

Transfer

Project $(2,0)$ onto the line spanned by $(1,1)$.