Relation between vector norms, eigenvalues, and condition numbers.

Consider the matrix A with eigenvalues $\lambda_i$ and corresponding eigenvectors ${\bf v}^{(i)}$, $i=1,2,\ldots,n$. Assume the eigenvalues are sorted in descending order, then we have

$$A {\bf v}^{(i)} = \lambda_i {\bf v}^{(i)}, \quad {\rm with... ...\geq \vert\lambda_2\vert \geq \cdots \geq \vert\lambda_n\vert.$$ (1)

We relate the norms of matrices with vector norms in the following definition:

$$\vert\vert A \vert\vert = \max_{\vert\vert{\bf x}\vert\vert = 1} \vert\vert A {\bf x}\vert\vert$$ (2)

With eigenvalues of A sorted in descending order, we can prove that for symmetric matrices A: $\vert\vert A\vert\vert = \vert\lambda_1\vert$.

From (1), assuming A is invertible, we derive

$${\bf v}^{(i)} = A^{-1} \lambda_i {\bf v}^{(i)} \quad {\rm wh... ...uad \frac{1}{\lambda_i} {\bf v}^{(i)} = A^{-1} {\bf v}^{(i)}.$$ (3)

Thus A^-1 has the same eigenvectors as A with eigenvalues $\frac{1}{\lambda_i}$, so we can write

$$A^{-1} {\bf v}^{(i)} = \frac{1}{\lambda_i} {\bf v}^{(i)}, ... ...t \leq \cdots \leq \left\vert\frac{1}{\lambda_n} \right\vert.$$ (4)

So, $\vert\vert A^{-1}\vert\vert = \left\vert \frac{1}{\lambda_n} \right\vert$.

The condition number of matrix A is ${\rm cond}(A)$. If A is symmetric:

$${\rm cond}(A) = \vert\vert A\vert\vert \vert\vert A^{-1}\vert\vert = \left\vert \frac{\lambda_1}{\lambda_n} \right\vert.$$ (5)