Can you undo a map that loses information?

Source scope: §3.16, PDF pages 25. Numerical examples, experiments, added derivations, and practice are teaching supplements unless identified as source results.

Undo only what survives

Let $A=\begin{bmatrix}2&0&0\\0&0&0\end{bmatrix}$. Its output contains twice the first input coordinate and no information about the other two. The map $A^\dagger(y_1,y_2)^T=(y_1/2,0,0)^T$ recovers the visible component and sets unrecoverable input coordinates to zero. It cannot recover a discarded value from nothing.

For every $A\in\mathbb R^{m\times n}$ the Moore–Penrose pseudoinverse $A^\dagger\in\mathbb R^{n\times m}$ exists and is uniquely characterized by

The source states existence and uniqueness without a uniqueness proof. If $A$ is invertible, $A^\dagger=A^{-1}$: the inverse satisfies all four conditions. A nonsquare matrix has no two-sided inverse, but still has this pseudoinverse.

Reverse the SVD

For $A=U\Sigma V^T$,

Transpose the $m\times n$ diagonal array $\Sigma$ to an $n\times m$ array and replace every positive $\sigma_i$ by $1/\sigma_i$. Keep zero entries zero. For the example, $A^\dagger=\begin{bmatrix}1/2&0\\0&0\\0&0\end{bmatrix}$. A zero singular value is never divided into one.

Verify the four conditions

The source leaves verification as an exercise. The following solution is a teaching supplement. Let $D_m=\Sigma\Sigma^\dagger$ and $D_n=\Sigma^\dagger\Sigma$. Both are diagonal masks with $r$ leading ones and zeros elsewhere. Hence

Insert $U^TU=I$ and $V^TV=I$ into the products to get $AA^\dagger A=U\Sigma V^T=A$ and $A^\dagger AA^\dagger=V\Sigma^\dagger U^T=A^\dagger$. Also $AA^\dagger=UD_mU^T$ and $A^\dagger A=VD_nV^T$ are symmetric because the masks are symmetric. All four defining properties hold.

For the running example the two masks are $\operatorname{diag}(1,0)$ and $\operatorname{diag}(1,0,0)$, so the same verification can be done entry by entry. These products are orthogonal projections onto the column and row spaces. In particular $A^\dagger Ax$ returns the row-space component of $x$, not necessarily $x$ itself.

Transfer

For $B=\begin{bmatrix}0&0\\0&3\end{bmatrix}$, $B^\dagger=\operatorname{diag}(0,1/3)$. As a positive singular value approaches zero its reciprocal grows; at zero the pseudoinverse assigns zero. This explains why the pseudoinverse can be sensitive to small measurement errors near a change of rank.

Explore the relationship

Check your reasoning