Click here to go back to the previous page
So, what exactly are eigenvalues and eigenvectors? To understand them, we should view matrices as linear transformations. Think of it like a function. They have inputs and outputs. In calculus, you have a variable \(x\) as the input and the output is \(f(x)\). In linear algebra, the idea is the same. You have a vector \(x\) that gets transformed by the matrix \(\textbf{A}\) to yield a new vector \(\textbf{A}x\).
The eigenvectors of a matrix \(\textbf{A}\) are the vectors that when transformed by \(\textbf{A}\) are pararell to the initial vector.
Algebraically, this is written
\[ \textbf{A}x = \lambda x \] where \(\lambda\) is an eigenvalue.Eignvalues are useful for solving many dynamic problems.
How do we find the eigenvalues and eigenvectors of a matrix? First, know that only square matrices have eigenvalues and eignvectors.
Manipulating the equation above
\[ \textbf{A}x - \lambda x = (\textbf{A} - \lambda \textbf{I})x = 0 \]Analyzing this result, we find that \(\textbf{A} - \lambda \textbf{I}\) must be singular for there to be nontrival solutions. This equation is called the characteristic equation much like the one related to differential equations.
Remember that singular matrices have a zero determinant. So
\[ \det{(\textbf{A} - \lambda \textbf{I})} = \det{ \left( \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{bmatrix} - \begin{bmatrix} \lambda & 0 & \cdots & 0 \\ 0 & \lambda & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda \end{bmatrix} \right) } = \begin{vmatrix} a_{11} - \lambda & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} - \lambda & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} - \lambda \end{vmatrix} = 0 \]Finding the eigenvalues is now much easier.
An interesting fact about the eigenvalues of a matrix is that is that they add up to the trace of a matrix.
Well, what is the trace? The trace of a square matrix is the sum of the entries along the main diagonal.
Given an \(n\) by \(n\) matrix \(\textbf{B}\), the trace
\[ \textrm{tr} \ \textbf{B} = \sum_{i=1}^{n} b_{ii} \]There are a couple of common trace properties.
Let \(\textbf{A}\) and \(\textbf{B}\) be \(n\) by \(n\) matrices.
\[ \textrm{tr}(\textbf{A + B}) = \textrm{tr}(\textbf{A}) + \textrm{tr}(\textbf{B}) \] \[ \textrm{tr}(\textbf{A}) = \textrm{tr}(\textbf{A}^{T}) \] \[ \textrm{tr}(k\textbf{A}) = k\textrm{tr}(\textbf{A}) \] for some constant \(k\). \[ \textrm{tr}(\textbf{AB}) = \textrm{tr}(\textbf{BA}) \]There is a general formula that can be used to find the eignvalues of a 2 by 2 matrix.
We will use a matrix \( \textbf{A} = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \).
Remember that we will need to use the equation \(\det{(\textbf{A} - \lambda \textbf{I})} = 0 \).
With this
\[ \det{(\textbf{A} - \lambda \textbf{I})} = \begin{vmatrix} a - \lambda & b \\ c & d - \lambda \end{vmatrix} = (a - \lambda)(d - \lambda) - bc \] \[ (a - \lambda)(d - \lambda) - bc = ad - \lambda a - \lambda d + \lambda^{2} - bc = 0 \]Factoring and reordering the terms, we get
\[ ad - \lambda a - \lambda d + \lambda^{2} - bc = \lambda^{2} - \lambda(a + d) + ad - bc = 0 \]Now we have a simple quadratic equation. All we have to do is solve it and we have our eigenvalues.
Notice that
\[ \lambda^{2} - \lambda(a + d) + (ad - bc) = \lambda^{2} - \lambda(\textrm{tr}(\textbf{A})) + \det{\textbf{A}} \]We can conclude that for any 2 by 2 matrix, the eigenvalues can be found using the following formula.
\[ \lambda^{2} - \lambda(\textrm{tr}(\textbf{A})) + \det{\textbf{A}} = 0 \]Once we have the eigenvalues of a matrix, finding the eigenvectors becomes a much easier task. All we have to do is plug the eigenvalues back into the initial equation.
\[ (\textbf{A} - \lambda \textbf{I})x = 0 \]The eigenvectors make up the nullspace of \(\textbf{A} - \lambda \textbf{I}\). Another interesting fact about eigenvalues that should be noted is that the product of the eigenvalues of a matrix is equal to the determinant of that matrix.
Eigenvalues can be repeated much like the roots of a polynomial can appear more than once. When this happens, there is a change that the eigenvectors of the matrix will not be distinct. Here is an instance of this:
Find the eigenvalues for \(\textbf{M} = \begin{bmatrix} 3 & 1 \\ 0 & 3 \end{bmatrix}\)
\[ \det{(\textbf{M} - \lambda \textbf{I})} = 0 \] \[ \begin{vmatrix} 3 - \lambda & 1 \\ 0 & 3 - \lambda \end{vmatrix} = (3-\lambda)(3-\lambda) = 0 \] \[ \lambda_{1} = 3 \quad \lambda_{2} = 3 \]Fact
If a square matrix is singular, then \(\lambda = 0\) will always be an eigenvalue.
This is pretty easy to show. The characteristic polynomial of any \(n\) by \(n\) matrix is in the form
\[ (-1)^{n}\lambda^{n} + (-1)^{n-1}\textrm{tr}\textbf{A}\lambda^{n-1} + \cdots + \det{\textbf{A}} = 0 \]When the matrix is singular, the determinant is 0 and the polynomial becomes
\[ (-1)^{n}\lambda^{n} + (-1)^{n-1}\textrm{tr}\textbf{A}\lambda^{n-1} + \cdots = 0 \]All of the other terms have \(\lambda\) which means that it can be factored out.
\[ \lambda((-1)^{n}\lambda^{n-1} + (-1)^{n-1}\textrm{tr}\textbf{A}\lambda^{n-2} + \cdots) = 0 \]The solution that immediately emerges is \(\lambda = 0\).
When dealing with eigenvalues, there is a change that they can be repeated much like the roots of a polynomial. We will continue to solve for the eigenvectors anyways and see what happens.
\[ (\textbf{M} - \lambda_{1}\textbf{I})x_{1} = 0 \] \[ \begin{bmatrix} 3 - \lambda_{1} & 1 \\ 0 & 3 - \lambda_{1} \end{bmatrix} \begin{bmatrix} a_{1} \\ a_{2} \end{bmatrix} = \begin{bmatrix} 3 - 3 & 1 \\ 0 & 3 - 3 \end{bmatrix} \begin{bmatrix} a_{1} \\ a_{2} \end{bmatrix} = \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} a_{1} \\ a_{2} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \]With just a glance, a solution can be spotted.
\[ x_{1} = \begin{bmatrix} a_{1} \\ a_{2} \end{bmatrix} = s\begin{bmatrix} 1 \\ 0 \end{bmatrix} \]Now let's think about what happens when we try to solve for the other eigenvector \(\lambda_{2}\). Since it is the same as the previous eigenvalue, we will get the same exact matrix.
\[ \begin{bmatrix} 0 & 1 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} a_{1} \\ a_{2} \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} \]But the problem is that this system only has one solution. This means that there is no distinct eigenvector for the second eigenvalue.
\[ x_{1} = s\begin{bmatrix} 1 \\ 0 \end{bmatrix} \qquad x_{2} = \textrm{There is no other eigenvector} \] where \(s\) is an arbitrary constant.So in summary, given that there are repeated eigenvalues, there is a chance that there will be a shortage of eivenvectors. This can in turn affect the process of matrix diagonalization because this means there are no longer enough linearly independent eigvenectors. Therefore, matrices that do not have unique eigenvalues may or may not be diagonalizable.
However, it is important to note that there are cases where repeated eigenvalues can still yield independent eigenvectors. A very simple instance of this is the identity matrix \(\textbf{I}\). We'll look at the \(4\) by \(4\) identity matrix.
\[ \det{(\textbf{I} - \lambda \textbf{I})} = \begin{bmatrix} 1 - \lambda & 0 & 0 & 0 \\ 0 & 1 - \lambda & 0 & 0 \\ 0 & 0 & 1 - \lambda & 0 \\ 0 & 0 & 0 & 1 - \lambda \end{bmatrix} = (1-\lambda)^{4} = 0 \]The only eigenvalue is \(\lambda = 1\), but the eigenvectors are
\[ x_{1} = \begin{bmatrix} 1 \\ 0 \\ 0 \\ 0 \end{bmatrix} \quad x_{2} = \begin{bmatrix} 0 \\ 1 \\ 0 \\ 0 \end{bmatrix} \quad x_{3} = \begin{bmatrix} 0 \\ 0 \\ 1 \\ 0 \end{bmatrix} \quad x_{4} = \begin{bmatrix} 0 \\ 0 \\ 0 \\ 1 \end{bmatrix} \]As we saw with the examples, repeated eigenvalues may either have distinct eigenvectors or shared eigenvectors. This is why it is preferable to work with matrices that have unique eigenvectors. When they are all unique, the matrix will always have independent eigenvectors.
There is one last piece of information that is critical to know. Not only may eigenvectors be repeated, but they can also be complex numbers. One theorem that can be applied to this is complex conjugate root theorem. This theorem essentially says that complex roots always come in pairs. We can verify this fact with the knowledge that we already know. The trace of the matrix is equal the sum of the eigenvalues. If we have a matrix of real numbers and an eigenvector that is a complex number with a nonzero imaginary part, there must be some other complex number that cancels out the imaginary part. This other complex number is its complex conjugate.
There is also a relationship between the eigenvectors. "An eigenvector for the conjugate eigenvalue is simply the conjugate eigenvector (the eigenvector obtained by conjugating each entry of the first eigenvector). This is always true."
\[ x_{1} = \begin{bmatrix} a_{1} + b_{1}i \\ a_{2} + b_{2}i \\ \vdots \\ a_{n} + b_{n}i \end{bmatrix} \qquad x_{2} = \overline{x_{1}} = \begin{bmatrix} a_{1} - b_{1}i \\ a_{2} - b_{2}i \\ \vdots \\ a_{n} - b_{n}i \end{bmatrix} \]Given a matrix of real number entries, if \(\lambda_{*}\) is a complex eigenvalue with a nonzero imaginary part, then there exists another eigenvalue \(\overline{\lambda_{*}}\) such that \(\overline{\lambda_{*}}\) is the complex conjugate of \(\lambda_{*}\).
\[ z = a + bi \qquad \bar{z} = a - bi \]Example:
The matrix \(\textbf{R} = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}\) represents a 90 degree rotation. Find its eigenvalues and eigenvectors.
\[ \det(\textbf{R} - \lambda\textbf{I}) = \begin{bmatrix} -\lambda & -1 \\ 1 & -\lambda \end{bmatrix} = \lambda^{2} + 1 = 0 \] \[ \lambda^{2} + 1 = 0 \quad \Rightarrow \quad \lambda^{2} = -1 \quad \Rightarrow \quad \lambda = \pm \sqrt{-1} = \pm i \] \[ \lambda_{1} = i \qquad \lambda_{2} = -i\]As expected, since one eigenvalue is a complex number, the other is its complex conjugate. Now we can solve for the eigenvectors with \((\textbf{R} - \lambda\textbf{I})x = 0\). In doing so, we get
\[ x_{1} = \begin{bmatrix} i \\ 1 \end{bmatrix} \qquad x_{2} = \begin{bmatrix} 1 \\ i \end{bmatrix} \]Complex eigenvalues that have an absolute value of 1 can hint towards the periodic nature of a matrix. If a given matrix \(\textbf{A}\) is diagonalizable (the very next topic covers diagonalization and powers of A), then it can be written as
\[ \textbf{A} = \textbf{S}\Lambda\textbf{S} \] where \(\textbf{S}\) is a square matrix of the independent eigenvectors of \(\textbf{A}\) and \(\Lambda\) a the diagonal matrix of eigenvalues.When a matrix is diagonal, then we can easily compute its powers.
\[ \textbf{A}^{k} = \textbf{S}\Lambda^{k}\textbf{S} \]Suppose the eigenvalues of \(\textbf{A}\) are complex numbers with absolute value 1. That means they lie on the unit circle within the complex plane. Numbers like this can be expressed with the complex exponential.
\[ e^{i\theta} \]When the exponent of the exponential is a mutliple of \(2\pi\), we get 1.
\[ e^{0} = e^{i2\pi} = e^{i4\pi} = e^{i6\pi} = \cdots \]This means that in order to determine the periodicity of the matrix, we must figure out when the exponent goes back to 0.
\[ \lambda = e^{i\theta} \] \[ i\theta T = i2\pi \] \[ \theta T = i2\pi \] \[ T = \frac{2\pi}{\theta} \]With this equation, we see that the period is given by \(T\).
For a matrix to be diagonalizable, there must be \(n\) linearly independent eigenvectors. Putting those eigenvectors into the columns of a matrix \(\textbf{S}\)
\[ \textbf{A}\textbf{S} = \begin{bmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{n1} & \cdots & a_{nn} \end{bmatrix} \begin{bmatrix} & & \\ x_{1} & \cdots & x_{n} \\ & & \end{bmatrix} = \begin{bmatrix} & & \\ \textbf{A}x_{1} & \cdots & \textbf{A}x_{n} \\ & & \end{bmatrix} = \begin{bmatrix} & & \\ \lambda_{1}x_{1} & \cdots & \lambda_{n}x_{n} \\ & & \end{bmatrix} = \begin{bmatrix} & & \\ x_{1} & \cdots & x_{n} \\ & & \end{bmatrix} \begin{bmatrix} \lambda_{1} & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & \lambda_{n} \end{bmatrix} = \textbf{S}\Lambda \] where \(x_{n}\) is the \(n\)-th eigenvector and \(\Lambda\) is a diagonal matrix composed of the eigenvalues of \(\textbf{A}\).One reason for diagaonlization being really useful is the fact that it is much easier to compute the powers of a diaganal matrix rather than a non-diagonal matrix.
Example:
\[ \begin{bmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix}^{3} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 3 \end{bmatrix} = \begin{bmatrix} 1^{3} & 0 & 0 \\ 0 & 2^{3} & 0 \\ 0 & 0 & 3^{3} \end{bmatrix} = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 8 & 0 \\ 0 & 0 & 27 \end{bmatrix} \]Taking a look at the equation \(\textbf{A}\textbf{S} = \textbf{S}\Lambda\), we can rewrite it into
\[ \textbf{A} = \textbf{S}\Lambda\textbf{S}^{-1} \]Notice what happens when we square \(\textbf{A}\).
\[ \textbf{A}^{2} = \textbf{S}\Lambda(\textbf{S}^{-1}\textbf{S})\Lambda\textbf{S}^{-1} = \textbf{S}\Lambda^{2}\textbf{S}^{-1} \]In general
\[ \textbf{A}^{k} = \textbf{S}\Lambda^{k}\textbf{S}^{-1} \]So when we raise the matrix \(\textbf{A}\) to a power \(k\), the eigenvalues of the matrix are also raised to that power. We could have also seen this pattern with the equation \(\textbf{A}x = \lambda x\). Watch what happens when we square \(\textbf{A}\) in this equation.
\[ \textbf{A}x = \lambda x \] \[ \textbf{A}^{2}x = \textbf{A}\textbf{A}x = \textbf{A} \lambda x \] \[ \textbf{A}^{2}x = \lambda(\textbf{A} x) = \lambda^{2}x \]And once again, in general
\[ \textbf{A}^{k}x = \lambda^{k}x \]Eigenbasis — set of basis vectors that happen to also be the eigenvectors.
Similar matrices — two matrices that have the same rank, trace, determinant, and eigenvalues.
Similarity transformation
Given a matrix \(\textbf{B}\) with eigenvalues \(\lambda_{1}, \lambda_{2}, \ldots, \lambda_{n}\), the eigenvalues of \(\textbf{B}^{-1}\) are the reciprocals of the eigenvalues of \(\textbf{B}\), \(\frac{1}{\lambda_{1}}, \frac{1}{\lambda_{2}}, \ldots, \frac{1}{\lambda_{n}}\).
Reasoning:
\[ \textbf{B}x = \lambda_{i} x \quad \Rightarrow \quad x = \textbf{B}^{-1}\lambda_{i} x \quad \Rightarrow \quad \textbf{B}^{-1}x = \frac{1}{\lambda_{i}}x \]Another great use of eigenvalues is for solving systems of linear differential equations. Some information on differential equations is on this page.
Matrices can tell us useful information.
Stability of \(2\) by \(2\) matrix.
Here is an example of solving a system of first order linear homogeneous differential equations using matrices.
\[ x' = ax + by \] \[ y' = cx + dy \]We can represent this system with matrices. The functions that we are differentating are with respect to \(t\).
\[ x' = \frac{dx}{dt} \qquad y' = \frac{dy}{dt} \] \[ \begin{pmatrix} x \\ y \end{pmatrix}' = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} \]Differentiating a vector is the same as differentiating its components.
\[ \begin{pmatrix} x \\ y \end{pmatrix}' = \begin{pmatrix} x' \\ y' \end{pmatrix} \]When solving systems of differential equations of this type, solutions will have the form
\[ \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix} e^{\lambda t} \] where \(a_{1}\) and \(a_{2}\) are arbitrary constants. This is just a vector of coefficients.Using this knowledge, we can plug this into the system.
\[ \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix}\lambda e^{\lambda t} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix} e^{\lambda t} \]Canceling the \(\lambda e^{\lambda t}\) terms on both sides, we see
\[ \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix}\lambda = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix} \]For convenience, let \( v = \begin{pmatrix} a_{1} \\ a_{2} \end{pmatrix} \) and \(\textbf{A} = \begin{pmatrix} a & b \\ c & d \end{pmatrix} \)
Now we can make the important connection. Our system is
\[ \textbf{A}u = \lambda u \]This is the same equation that we started with when computing were eigenvalues. To solve this, we can use the characteristic equation.
\[ \det(\textbf{A} - \lambda \textbf{I}) = 0 \]This equation will give the characteristic polynomial for the system. Once we have the eigenvalues for this matrix, we can must find the eigenvectors and substitude them back into the solution form.
\[ \begin{pmatrix} x \\ y \end{pmatrix} = c_{1}x_{1} e^{\lambda_{1}t} + c_{2}x_{2} e^{\lambda_{2}t} \] where \(x_{1}\) and \(x_{2}\) are eigenvectors.The vector \(u\) ended up becoming the eigenvectors of the matrix. All we are left to do is find the values of \(c_{1}\) and \(c_{2}\) using the initial conditions of the system. That is a relatively easy process.
Now we will go through a slightly different approach that utilizes matrix diagonalization using eigenvectors. We are no longer considering the system prior, but instead a system of \(n\) first order linear homogeneous differential equations with \(n\) unknowns. The only thing that we must assume is that there are \(n\) linearly independent eigenvectors corresponding to the \(n\) by \(n\) matrix \(\textbf{A}\). We will put these eigenvectors into a matrix \(\textbf{S}\). Hopefully the next equation is familiar from the section before.
\[ \textbf{A}\textbf{S} = \textbf{S}\Lambda \quad \Rightarrow \quad \textbf{A} = \textbf{S}\Lambda\textbf{S}^{-1} \]Recall that \(\Lambda\) is the diagonal matrix with the eigenvalues of \(\textbf{A}\). Using this neat equation, we can rewrite the equation for our system. We had
\[ u' = \textbf{A}u \]Let \(u = \textbf{S}v \)
Due to the fact that we assumed \(\textbf{S}\) has \(n\) linearly independent columns and it is an \(n\) by \(n\) matrix, \(\textbf{S}\) is full-rank. This means that it spans the whole space, \(\mathbb{R}^{n}\), and that therefore there is a combination that gives the vector \(u\). The vector \(v\) is the vector that gives this combination. It also has the variables \(x\) and \(y\). Let's substitute it into the equation.
\[ u' = \textbf{A}u \quad \textrm{becomes} \quad (\textbf{S}v)' = \textbf{A}\textbf{S}v \]Because \(\textbf{S}\) has only constants and no variables, it can be factored out leaving \(v'\).
\[ \textbf{S}v' = \textbf{A}\textbf{S}v \] \[ v' = \textbf{S}^{-1}\textbf{A}\textbf{S}v \]Now we will use the equation that we mentioned above to performed a substitution.
\[ v' = \textbf{S}^{-1}(\textbf{A}\textbf{S})v = \textbf{S}^{-1}(\textbf{S}\Lambda)v = (\textbf{S}^{-1}\textbf{S})\Lambda v = \Lambda v\]The equation is now
\[ v' = \Lambda v\]This equation gives us the solutions immediately. Remember that \(\Lambda\) is a diaganal matrix. In general, the equation looks like this:
\[ v' = \begin{pmatrix} v_{1}' \\ v_{2}' \\ \vdots \\ v_{n}' \end{pmatrix} = \begin{pmatrix} \lambda_{1} & 0 & \cdots & 0 \\ 0 & \lambda_{2} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & \lambda_{n} \end{pmatrix} \begin{pmatrix} v_{1} \\ v_{2} \\ \vdots \\ v_{n} \end{pmatrix} = \Lambda v \]We can construct equations
\[ \frac{dv_{1}}{dt} = \lambda_{1}v_{1} \qquad \frac{dv_{2}}{dt} = \lambda_{2}v_{2} \quad \cdots \quad \frac{dv_{n}}{dt} = \lambda_{n}v_{n} \]These are just seperable equations which means these can be solved fairly simply.
\[ \frac{dv_{n}}{dt} = \lambda_{n}v_{n} \] \[ \int \frac{1}{v_{n}}dv_{n} = \int \lambda_{n}dt \] \[ \ln{v_{n}} = \lambda_{n}t + K \] \[ e^{\ln{v_{n}}} = v_{n} = e^{(\lambda_{n}t + K)} \] \[ v_{n} = e^{\lambda_{n}t} e^{K} \]Let \(K = C_{n}\)
\[ v_{n} = C_{n}e^{\lambda_{n}t} \]We can replace \(C_{n}\) with \(v_{n}(0)\). So we see that the general solution to these equations is \( v_{n} = e^{\lambda_{n}t}v_{n}(0) \). There is a more general solution can be written for all of \(v\).
\[ v(t) = e^{\Lambda t} v(0) \]But wait. We had to use the initial substitution, \(u = \textbf{S}v\). Let's put this solution in terms of \(u\).
\[ u(t) = \textbf{S}v(t) = \textbf{S} e^{\Lambda t} v(0) \] \[ u(0) = \textbf{S}v(0) \Rightarrow v(0) = \textbf{S}^{-1} u(0) \] \[ u(t) = \textbf{S} e^{\Lambda t} \textbf{S}^{-1} u(0) = e^{\textbf{A}t} u(0) \] \[ u(t) = e^{\textbf{A}t} u(0) \]Here we used matrix exponentiation. That is the next topic.
Overall, linear algebra gives us a way to expresss the solution a system of linear differential equations
\[ u'(t) = \textbf{A}u(t) \]as a single compact equation
\[ u(t) = e^{\textbf{A}t} u(0) \]The last thing that I will mention here is that one method of solving higher order linear differential equations is to reduce them into a system of smaller order linear differential equations. When they are reduced, the system can be solved with matrices.
To be able to compute this, we can refer to the power series expansion for the exponential. Recall from calculus
\[ e^{x} = \sum_{n=0}^{\infty} \frac{x^{n}}{n!} = 1 + x + \frac{x^{2}}{2} + \frac{x^{3}}{6} + \cdots \]All we have to do now is plug in \(\textbf{A}t\) for \(x\).
\[ e^{\textbf{A}t} = \sum_{n=0}^{\infty} \frac{(\textbf{A}x)^{n}}{n!} = \textbf{I} + \textbf{A}x + \frac{(\textbf{A}x)^{2}}{2} + \frac{(\textbf{A}x)^{3}}{6} + \cdots \]This is where matrix diagaonalization comes into play.
Above we discussed how when given an \(n\) by \(n\) matrix with \(n\) linearly independent independent columns, we could put these columns into a matrix that we called \(\textbf{S}\) and then have
\[ \textbf{A} = \textbf{S}\Lambda\textbf{S}^{-1} \] where \(\Lambda\) is the matrix with the eigenvalues of \(\textbf{A}\) along its main diagonal.Remember that
\[ \textbf{A}^{2} = \textbf{S}\Lambda (\textbf{S}^{-1} \textbf{S}) \Lambda \textbf{S}^{-1} = \textbf{S}\Lambda^{2} \textbf{S}^{-1} \qquad \qquad \textbf{A}^{k} = \textbf{S} \Lambda^{k} \textbf{S}^{-1} \]We can utilize this here.
\[ e^{\textbf{A}t} = \sum_{n=0}^{\infty} \frac{(\textbf{A}x)^{n}}{n!} \] \[ e^{\textbf{A}t} = e^{\textbf{S}\Lambda \textbf{S}^{-1} t} = \sum_{n=0}^{\infty} \frac{(\textbf{S}\Lambda \textbf{S}^{-1} t)^{n}}{n!} = \sum_{n=0}^{\infty} \frac{\textbf{S}\Lambda^{n} \textbf{S}^{-1} t^{n}}{n!} \]We can pull out \(\textbf{S}\) and the \(\textbf{S}^{-1}\) since they are not in terms of \(n\).
\[ e^{\textbf{A}t} = \textbf{S} \left(\sum_{n=0}^{\infty} \frac{\Lambda^{n} t^{n}}{n!}\right) \textbf{S}^{-1} = \textbf{S} e^{\Lambda t} \textbf{S}^{-1} \] \[ e^{\textbf{A}t} = \textbf{S} e^{\Lambda t} \textbf{S}^{-1} \]DRAFT (NEEDS TO BE REVISED):
Example Problem:
A family of matrices \(\textbf{T}\) are tridiagonal where \(\textbf{T}_{n}\) is the \(n\) by \(n\) matrix. Given that the eigenvalues of \(\textbf{T}_{n}\) are \(\lambda\), you are told that \(\lambda^{6} = 1\). We are also given \(\textbf{T}_{4}\).
\[ \textbf{T}_{4} = \begin{bmatrix} 1 & 1 & 0 & 0 \\ 1 & 1 & 1 & 0 \\ 0 & 1 & 1 & 1 \\ 0 & 0 & 1 & 1 \end{bmatrix} \]How can we calculate the determinant of \(T_{n}\) using other matrices within its family of tridiagonal matrices? Let's go through the process.
\[ \det\textbf{T}_{n} = \det(\textbf{T}_{n-1}) - \det(\textbf{T}_{n-2}) \]Let \(D_{n} = \det \textbf{T}_{n}\)
\[ D_{n} = D_{n-1} - D_{n-2} \] \[ D_{1} = \begin{vmatrix} 1 \end{vmatrix} = 1 \] \[ D_{2} = \begin{vmatrix} 1 & 1 \\ 1 & 1 \end{vmatrix} = 0 \] \[ D_{3} = D_{2} - D_{1} = -1 \]We can turn this second order difference equation into a system of first order equations.
\[ D_{n} = D_{n-1} - D_{n-2} \] \[ D_{n-1} = D_{n-1} \]Putting these equations into a matrix, we get
\[ \begin{bmatrix} D_{n} \\ D_{n-1} \end{bmatrix} = \begin{bmatrix} D_{n-1} - D_{n-2} \\ D_{n-1} \end{bmatrix} = \begin{bmatrix} 1 & -1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} D_{n-1} \\ D_{n-2} \end{bmatrix} \]Let \(\textbf{B} = \begin{bmatrix} 1 & -1 \\ 1 & 0 \end{bmatrix}\) and \(u_{n-1} = \begin{bmatrix} D_{n} \\ D_{n-1} \end{bmatrix}\)
We have that
\[ u_{n-1} = \textbf{B}u_{n-2}\] \[ u_{n} = \textbf{B}u_{n-1}\]The starting point is based on the first two determinants that we calculating above.
\[ u_{1} = \begin{bmatrix} D_{2} \\ D_{1} \end{bmatrix} \]Let's go as far back as we can.
\[ u_{2} = \textbf{B}u_{1} \qquad u_{3} = \textbf{B}u_{2} = \textbf{B}\textbf{B}u_{1} = \textbf{B}^{2}u_{1} \qquad \cdots \qquad u_{n} = \textbf{B}^{n-1}u_{1} \]By calcuating the eigenvalues, we can determine the behavior of the system as \(n\) gets larger.
\[ \begin{bmatrix} D_{n} \\ D_{n-1} \end{bmatrix} = \begin{bmatrix} 1 & -1 \\ 1 & 0 \end{bmatrix} \begin{bmatrix} D_{n-1} \\ D_{n-2} \end{bmatrix} \] \[ \det\left( \begin{bmatrix} 1 & -1 \\ 1 & 0 \end{bmatrix} - \lambda\textbf{I} \right) = \begin{vmatrix} 1 - \lambda & -1 \\ 1 & -\lambda \end{vmatrix} = 0 \] \[ \lambda^{2} - \lambda + 1 = 0 \]Completing the square
\[ \lambda^{2} - \lambda + 1 + \frac{1}{4} = \frac{1}{4} \] \[ (\lambda - \frac{1}{2})^{2} + 1 = \frac{1}{4} \] \[ (\lambda - \frac{1}{2})^{2} = -\frac{3}{4} \] \[ \lambda = \sqrt{-\frac{3}{4}} + \frac{1}{2} \] \[ \lambda = \frac{1 \pm \sqrt{3}i}{2} \]The eigenvalues are
\[ \lambda_{1} = \frac{1 + \sqrt{3}i}{2} \qquad \lambda_{2} = \frac{1 - \sqrt{3}i}{2} \]These are complex numbers with absolute value
\[ |\lambda| = \sqrt{\frac{1}{4} + \frac{3}{4}} = 1\]With a little more work, we can show that
\[ \lambda_{1} = \frac{1 + \sqrt{3}i}{2} = e^{i\pi/3} \qquad \lambda_{2} = \frac{1 - \sqrt{3}i}{2} = e^{-i\pi/3} \]To show this, just convert from rectangular to polar form and then apply euler's formula.
The problem tells us that the 6th power of eigenvalues is 1. Since we have complex eigenvalues, the system must be periodic.
\[ \textbf{B} = \textbf{S}\Lambda\textbf{S}^{-1} \] \[ \textbf{B}^{6} = \textbf{S}\Lambda^{6}\textbf{S}^{-1} = \textbf{S}\textbf{I}\textbf{S}^{-1} = \textbf{S}\textbf{S}^{-1} = \textbf{I} \] \[ \textbf{B}^{6} = \textbf{I} \]and
\[ \textbf{B}^{7} = \textbf{B} \]and in general
\[ \textbf{B}^{n} = \textbf{B}^{n-6} \]therefore showing that the system is indeed periodic. More specifically, the system has a period of 6.
\[ u_{1} = u_{7} = u_{13} \]More generally,
\[ u_{6k+1} = u_{1} \] where \(k\) is a positive integerThe definition of the rank of a matrix is the dimension of its column space (or row space).
\[ \textrm{rank}(\textbf{A}) = \dim{(C(\textbf{A}))} \]The dimension of the column space of a matrix it equal to the dimension of its row space. Therefore
\[ \textrm{rank}(\textbf{A}) = \textrm{rank}(\textbf{A}^{T}) \]Multiplication by a full-rank matrix preserves rank. Given an \(m\) by \(n\) matrix \(\textbf{A}\) and a full-rank \(n\) by \(n\) matrix \(\textbf{B}\)
\[ \textrm{rank}(\textbf{AB}) = \textrm{rank}(\textbf{A}) \]The 2 by 2 rotation matrix
\[ \textbf{R}(\theta) = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix} \]Given a square matrix, the number of its eigenvalues equal to 0 is the dimension of its nullspace. The basis vectors of the nullspace are the corresponding eigenvectors. This is because vectors in the nullspace satisfy
\[ Ax = 0\]When we take a look at the equation for eigenvalues, they have a connection.
\[ Ax = 0 \] is the same equation as \[ Ax = \lambda x \] where \(\lambda = 0\)This essentially proves the statement that was initially mentioned. All basis vectors of the nullspace are eigenvalues.
If the matrix has all nonzero eigenvalues, then the matrix is invertible. Why? There are many ways to show this. I'll relate this idea back to the nullspace of a matrix. We know that if a matrix has an empty nullspace, then its has no zero eigenvalues. We also knwo that the product of the eigenvalues is the determinant. If all the eigenvalues are nonzero, then the determinant will also be nonzero and thus the matrix will be invertible.