## Eigendecomposition¶

## Singular value decomposition¶

The singular value decomposition (SVD) gives a similar factorization to
the eigendecomposition, but can be applied to non-square matrices. It
also gives convenient solutions to find a matrix's rank and null space.

The SVD of an $m \times n$ matrix $A$ takes the form:
$$A = U \Sigma V^T$$ where $U$ is an $m \times m$ orthogonal matrix, $V$
is an $n \times m$ matrix with orthogonal columns, and $\Sigma$ is an
$m \times m$ matrix with zero entries on all of the off-diagonals, and
the diagonal containing only nonnegative entries.

The SVD can be used for many purposes, but in this book we use it
primarily for calculating the matrix pseudoinverse $A^+$. Once the SVD
is computed, the pseudoinverse $\Sigma^+$ of $\Sigma$ can be computed by
taking the reciprocal of all nonzero diagonal entries of $\Sigma$, and
leaving the zero entries. Then the pseudoinverse of $A$ is
$A^+ = V \Sigma^+ U^T$. It is an exercise to convince yourself that this
solution indeed satisfies the properties of the pseudoinverse.