Eigenvalues and Eigenvectors

Donovan D Chang

28 Eigenvalues and Eigenvectors

So far we have discussed the linear transformation $\vec{x}\mapsto A\vec{x}$ , where $\vec{x}\in\mathbb{R}^j$ and $A\vec{x}\in \mathbb{R}^i$ . Now, we consider a special case $\vec{x}$ is mapped into a scalar multiple of itself, i.e. for scalar $\lambda$ ,

$A\vec{x}=\lambda \vec{x}$

Naturally, $A$ is a $n\times n$ square matrix and it is a mapping from $\mathbb{R}^n$ onto itself for $n\in \mathbb{N}$ . To formally define relevant terms,

Definition. (Eigenvalue and Eigenvector)
Given a matrix equation $A\vec{x}=\lambda\vec{x}$ , where $A$ is a $n\times n$ matrix and $\lambda$ is a scalar,

$\vec{x}$ is an eigenvector of $A$ satisfying
$A\vec{x}=\lambda\vec{x}$

where $\vec{x}\neq \vec{0}$ .
Such $\lambda$ is called an eigenvalue.

Note that the set of eigenvectors is in fact $\mathrm{Nul}(A-\lambda I)$ , where $\mathrm{Nul}(A-\lambda I)\neq \left\lbrace\vec{0}\right\rbrace$ . Therefore, this set, a subspace of $\mathbb{R}^n$ , is called the eigenspace of $A$ corresponding to $\lambda$ . Let us illustrate with an example.

Example. Examine whether 3 is an eigenvalue of $A=\begin{pmatrix} 3 & 1 & 0\\ 2 & -1 & 7\\ 1 & 1 & 1 \end{pmatrix}$ .

Let us find the $\mathrm{Nul}(A-3I)$ by letting $A\vec{x}=3\vec{x}$ .
Then,

$\begin{align*} A-3I &= \begin{pmatrix} 0 & 1 & 0\\ 2 & -4 & 7\\ 1 & 1 & -2 \end{pmatrix}\\ & \sim \begin{pmatrix} 2 & 0 & 7\\ 1 & 0 & -2\\ 0 & 1 & 0 \end{pmatrix}\\ & \sim \begin{pmatrix} 0 & 0 & 11\\ 1 & 0 & -2\\ 0 & 1 & 0 \end{pmatrix}\\ & \sim \begin{pmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{pmatrix} =I \end{align*}$

Therefore, $A-3I$ is invertible and $\mathrm{Nul}(A) = \left\lbrace \vec{0}\right\rbrace$ .
Therefore, 3 is not an eigenvalue of $A$ .

Characteristic Equation

We saw, for non-trivial solutions of $A\vec{x}=\lambda\vec{x}$ to exist, $A-\lambda I$ has to be singular, i.e not invertible. That is,

$\mathrm{det}\left( A-\lambda I\right)=0$

We call this a characteristic equation of $A$ . And by solving this characteristic equation, we can efficiently identify eigenvalues.

Diagonalization

Now we turn our attention to decomposition of a matrix, factorizing into multiple matrices. Suppose we want to compute $A^{n}$ , the $n$ -th power of a $n\times n$ matrix $A$ , where the value of $n\in\mathbb{N}$ is large. Then, it would be a computationally expensive task to do it straightforward. However, if it is the case a given matrix can be decomposed, this cumbersome task can be substantially lightened. In this section, we introduce a special case of decomposition or factorization of a matrix called diagonalization.

Definition. (Diagonalization)
A $n\times n$ matrix $A$ is diagonalizable if there exists an invertible matrix $P$ and a diagonal matrix $D$ such that

$A=PDP^{-1}$

Note how easy and lightweighted the involved computation of the $n$ -th power of $A^n$ becomes when $A$ is diagonalizable as $PDP^{-1}$ .

$\begin{align*} A^n &= \left( PDP^{-1}\right)^n\\ &= \left( PDP^{-1}\right)\left( PDP^{-1}\right)\cdots\left( PDP^{-1}\right)\\ &= PD^{n}P^{-1} \end{align*}$

Let us illustrate with an example.

Example. Compute $A^{100}$ of a diagonalizable matrix $A=\begin{pmatrix} 4 & 0 & 1\\ -2 & 1 & 0\\ -2 & 0 & 1 \end{pmatrix}$ , whose eigenvalues are $\lambda=1,2,3$ .

1. Let us first find the eigenvectors of $A$ , where non-trivial solutions of $\left(A-\lambda I\right)\vec{x}=0$ .

(a) $A-I = \begin{pmatrix} 3 & 0 & 1\\ -2 & 0 & 0\\ -2 & 0 & 0 \end{pmatrix} \sim \begin{pmatrix} 1 & 0 & 0\\ 0 & 0 & 1\\ 0 & 0 & 0 \end{pmatrix}$ .
Therefore, $\vec{x} =\begin{pmatrix} x_1\\ x_2\\ x_3 \end{pmatrix} = \begin{pmatrix} 0\\ x_2\\ 0 \end{pmatrix} = x_2\begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}$ , for $x_2\in \mathbb{R}$ .

(b) $A-2I = \begin{pmatrix} 2 & 0 & 1\\ -2 & -1 & 0\\ -2 & 0 & -1 \end{pmatrix} \sim \begin{pmatrix} 1 & 0 & \frac{1}{2}\\ 0 & 1 & -1\\ 0 & 0 & 0 \end{pmatrix}$ .\\
Therefore, $\vec{x} =\begin{pmatrix} x_1\\ x_2\\ x_3 \end{pmatrix} = \begin{pmatrix} -\frac{1}{2}x_3\\ x_3\\ x_3 \end{pmatrix} = x_3\begin{pmatrix} -\frac{1}{2}\\ 1\\ 1 \end{pmatrix}$ , for $x_3\in \mathbb{R}$ .

(c) $A-3I = \begin{pmatrix} 1 & 0 & 1\\ -2 & -2 & 0\\ -2 & 0 & -2 \end{pmatrix} \sim \begin{pmatrix} 1 & 0 & 1\\ 0 & 1 & -1\\ 0 & 0 & 0 \end{pmatrix}$ .\\
Therefore, $\vec{x} =\begin{pmatrix} x_1\\ x_2\\ x_3 \end{pmatrix} = \begin{pmatrix} -x_3\\ x_3\\ x_3 \end{pmatrix} = x_3\begin{pmatrix} -1\\ 1\\ 1 \end{pmatrix}$ , for $x_3\in \mathbb{R}$ .

2. Construct $P=\begin{pmatrix} 0 & -1 & -\frac{1}{2}\\ 1 & 1 & 1\\ 0 & 1 & 1 \end{pmatrix}$ from linearly independent eigenvectors identified above, where the order is irrelevant.

Then, identify $P^{-1}$ .

$\begin{align*} \begin{pmatrix} P & I \end{pmatrix} &= \begin{pmatrix} 0 & -1 & -\frac{1}{2} & 1 & 0 & 0\\ 1 & 1 & 1 & 0 & 1 & 0\\ 0 & 1 & 1 & 0 & 0 & 1 \end{pmatrix}\\ &\sim \begin{pmatrix} 1 & 0 & 0 & 0 & 1 & -1\\ 0 & 1 & 0 & -2 & 0 & -1\\ 0 & 0 & 1 & 2 & 0 & 2 \end{pmatrix} = \begin{pmatrix} I & P^{-1} \end{pmatrix} \end{align*}$

Therefore, $P^{-1} =\begin{pmatrix} 0 & 1 & -1\\ -2 & 0 & -1\\ 2 & 0 & 2 \end{pmatrix}$ .

3. Construct $D = \begin{pmatrix} 1 & 0 & 0\\ 0 & 3 & 0\\ 0 & 0 & 2 \end{pmatrix}$ so that the order of eigenvalues matches the order $P$ was constructed.

4. Compute $A^{100}$

$\begin{align*} A^{100} &= \left(PDP^{-1}\right)^{100}\\ &= PD^{100}P^{-1}\\ &= \begin{pmatrix} 0 & -1 & -\frac{1}{2}\\ 1 & 1 & 1\\ 0 & 1 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0\\ 0 & 3 & 0\\ 0 & 0 & 2 \end{pmatrix}^{100} \begin{pmatrix} 0 & 1 & -1\\ -2 & 0 & -1\\ 2 & 0 & 2 \end{pmatrix}\\ &= \begin{pmatrix} 0 & -1 & -\frac{1}{2}\\ 1 & 1 & 1\\ 0 & 1 & 1 \end{pmatrix} \begin{pmatrix} 1^{100} & 0 & 0\\ 0 & 3^{100} & 0\\ 0 & 0 & 2^{100} \end{pmatrix} \begin{pmatrix} 0 & 1 & -1\\ -2 & 0 & -1\\ 2 & 0 & 2 \end{pmatrix}\\ &= \begin{pmatrix} 2\cdot 3^{100}-2^{100} & 1 & -1-3^{100}+2^{101}\\ -2\cdot 3^{100}+2^{101} & 1 & -1-3^{100}+2^{101}\\ -2\cdot 3^{100}+2^{101} & 0 & -3^{100}+2^{101} \end{pmatrix} \end{align*}$

Therefore,

$A^{100} = \begin{pmatrix} 2\cdot 3^{100}-2^{100} & 1 & -1-3^{100}+2^{101}\\ -2\cdot 3^{100}+2^{101} & 1 & -1-3^{100}+2^{101}\\ -2\cdot 3^{100}+2^{101} & 0 & -3^{100}+2^{101} \end{pmatrix}$

Note that the computation of $A^{n}$ of a diagonalizable matrix $A=PDP^{-1}$ is substantially simplified by the property of a diagonal matrix $D$ :

$D^n = \begin{pmatrix} d_1 &0 &\cdots &0\\ 0 &d_2 &\cdots &\vdots\\ \vdots &\cdots &\ddots &0\\ 0 &\cdots &0 &d_n\\ \end{pmatrix}^n = \begin{pmatrix} d_1^n &0 &\cdots &0\\ 0 &d_2^n &\cdots &\vdots\\ \vdots &\cdots &\ddots &0\\ 0 &\cdots &0 &d_n^n\\ \end{pmatrix}$

28 Eigenvalues and Eigenvectors

Diagonalization

License

Share This Book