Chapter 5: Eigenvectors and eigenvalues

5.1 Definitions. As we have seen, every matrix of order m×n generates a linear operator from the space Cⁿ to C^m. In this chapter, we assume that A is a square matrix of order n×n. Thus, A generates an operator from the space Cⁿ into itself. For such operators, we define the notions of eigenvectors and eigenvalues and consider their basic properties.

Definition 1  A nonzero vector x in Cⁿ is called an eigenvector of the operator A, if there exists a scalar l such that

Ax = lx.

(1)

The scalar l is called an eigenvalue of A if there exists a nonzero vector x such that (1) holds.

Examples:

(i) Every nonzero vector is an eigenvector of the identity matrix, corresponding to the eigenvalue l = 1.

(ii) The matrix

A =

é
ê
ë

0	1
1	0

ù
ú
û

has eigenvalues l₁ = 1, l₂ = -1, with the following eigenvectors, repectively:

x1 =

é ê ë

1 1

ù ú û

,  x2 =

é ê ë

-1 1

ù ú û

The identity (1) can be rewritten in the form

(A-lI)x = 0.

(2)

Thus, if we put B = (A-lI), then (2) becomes

Bx = 0.

(3)

Since the vector x in (3) is nonzero, it follows that B = (A-lI) is not invertible. Conversely, if B is not invertible, then B is not one-to-one by Theorem 3.5.   Therefore, there exists a nonzero vector in the null-space of B, i.e. there exists a nonzero vector x such that (3) holds. Since a matrix is non-invertible if and only if its determinant is zero, we have proved the following statement.

Theorem 1 In order for a scalar l to be an eigenvalue of a matrix A, it is necessary and sufficient that

|(A-lI)| = 0.

(4)

Equation (4) is called the characteristic equation, and its left hand side, denoted by D_A(l), i.e. the function D_A(l) = |(A-lI)| itself, is called the characteristic function of the matrix A. Theorem 1 says that eigenvalues of A are precisely the roots of the characteristic equation of A. Being the determinant of a matrix of the form

é
ê
ê
ê
ê
ê
ë

a11-l	a12	a13	¼	a1n
a21	a22-l	a23	¼	a2n
¼	¼	¼	¼	¼
an1	an2	an3	¼	ann-l

ù
ú
ú
ú
ú
ú
û

,

it is easy to see that the characteristic function D_A(l) of the matrix A is a polynomial of degree n whose leading coefficient, i.e. the coefficient of lⁿ, is (-1)ⁿ. Thus, D_A(l) can be written in the following form:

DA(l) = |(A-lI)| = (-1)^n(l^n+an-1l^n-1+...+a1l+a0),

(5)

where a_k, k = 1,2,..., n are some scalars defined by the elements of A.

If our scalars are assumed to be real numbers, then the characteristic function may not have any real roots, for example the polynomial

l²+1

does not have any real root. Thus, there are matrices which do not have any real eigenvalues. For instance, the matrix

é
ê
ë

0	-1
1	0

ù
ú
û

does not have any real eigenvalue since it characteristic function is l²+1.

We can, of course, allow complex numbers to enter the scene and consider matrices whose elements are complex numbers, in general, as well as multiplications by complex numbers, linear combinations with complex coefficients, etc. Then the characteristic fu

nction is a polynomial with complex coefficients. It is well known that every polynomial of degree n has n roots (among them there may be repeated roots). Hence, every complex matrix has n complex eigenvalues (among which there may be repeated). In particular, every real matrix also has complex eigenvalues.

From now on we will assume that the scalars are complex numbers. Thus, the characteristic polynomial of A has n roots l₁,...,l_n (among them there may be repeated ones). If l_i appears m_i times, then we can rearrange the roots as

l1,l1,...,l1 m1 times

,

l2,l2,...,l2 m2 times

,...,

lk,...,lk mk  times

,

where m₁+m₂+...+m_k = n. Thus, formula (5) can be rewritten as

|(A-lI)| = DA(l) = (-1)^n(l^n+

n-1 å k = 0

akl^k) = (-1)^n(l-l1)^m1...(lk-l)^mk.

(6)

The number m_i is called the multiplicity of the root l_i, or the multiplicity of the eigenvalue l_i of the matrix A. An eigenvalue l_i is callled simple eigenvalue if its multiplicity is 1.

The set {l₁,...,l_k} of all eigenvalues of the matrix A is called the spectrum of A and is denoted by s(A). The Main Theorem of Algebra implies that s(A) is always a nonempty subset of the complex plane. In general, s(A) may consist of one, two,..., or n points. The matrix A is said to have simple spectrum if every eigenvalue is simple (has multiplicity 1), i.e. A has simple spectrum if and only if s(A) consists of n points.

Example: Eigenvalues of a triangular matrix coincide with its diagonal elements. Therefore, a triangular matrix has simple spectrum if and only if all its diagonal elements are different.

5.2 Properties of eigenvectors and eigenvalues.

The following simple properties of eigenvalues follow immediately from the definitions.

1. If Ax = lx, then A^kx = l^kx,k = 1,2,..., i.e. if l is an eigenvalue of A, then l^k is an eigenvalue of A^k,, with the same eigenvectors.

2. If Ax = lx, and A is invertible, then l ¹ 0, and A^-1x = l^-1x, i.e. l^-1 is an eigenvalue of A^-1, with the same eigenvectors.

3. If l is an eigenvalue of A, then al is an eigenvalue of aA, l-c is an eigenvalue of (A-cI).

4. If l is an eigenvalue of A, then l also is an eigenvalue of A^t.

Indeed, sine l is an eigenvalue of A, we have |(A-lI)| = 0. This implies that

|(A^t-lI)| = |(A^t-lI)^t| = |((bf A^t)^t-lI)| = |(A-lI)| = 0,

hence l is an eigenvalue of A^t.

5.

|A| = l₁^m₁...l_k^m_k,

i.e. the determinant of A is equal to the product of eigenvalues of A, repeated as often as their corresponding multiplicities.

This follows from the formula (6), if we put l = 0.

Trace: The trace of a matrix A, denoted by tr(A), is the sum of the diagonal elements of A.

Thus, if A = [a_ij]_n×n, then tr(A) = a₁₁+a₂₂+...+a_nn.

Theorem 2 For an arbitrary matrix A, we have

tr(A) = -a_n-1,

where a_n-1 is the coefficient in the characteristic polynomial of A as in (5).

Proof: If we write down the determinant of (A-lI):

é
ê
ê
ê
ê
ê
ë

a11-l	a12	a13	¼	a1n
a21	a22-l	a23	¼	a2n
¼	¼	¼	¼	¼
an1	an2	an3	¼	ann-l

ù
ú
ú
ú
ú
ú
û

,

in the form

|(A-lI)| = (a₁₁-l)

é
ê
ê
ê
ê
ê
ë

a22-l	a23	a24	¼	a2n
a32	a33-l	a34	¼	a3n
¼	¼	¼	¼	¼
an2	an3	an4	¼	ann-l

ù
ú
ú
ú
ú
ú
û

-a₁₂

é
ê
ê
ê
ê
ê
ë

a21	a23	¼	a2n
a31	a33-l	a34	¼	a3n
¼	¼	¼	¼	¼
an1	an3	an4	¼	ann-l

ù
ú
ú
ú
ú
ú
û

+ ...

(7)

and note that in the formula (2) of Chapter 4, the determinant is a sum of terms, each of which is a products of n elements of the matrix such that only one element on each row and one element on each column are factors of a same product, then we can see that the coefficient of l^n- in the characteristic polynomial of A, i.e. (-1)ⁿa_n-1, is the same as the coefficient of l^n-1 of the function

(a₁₁-l)(a₂₂-l)....(a_nn-l),

which is (-1)^n-1(a₁₁+...+a_nn). Hence,

(-1)ⁿa_n-1 = (-1)^n-1(a₁₁+...+a_nn),

which implies

tr(A) = a₁₁+...+a_nn = -a_n-1.

Theorem 3 If A is an arbitrary matrix of order n×n and l₁,...,l_n are its eigenvalues (among them there may be repeated ones), then

tr(A) = l1+l2+...+ln

(8)

Proof: The characteristic function of the matrix A is

D_A(l) = (-1)ⁿ(l-l₁)(l-l₂)...(l-l_n).

If we decompose this polynomial and collect the cofficients of l^-n-1, then we can easily see that the coefficient of l^n-1 of the above polynomial is (-1)^n-1(l₁+l₂+...+l_n). Hence

(-1)^n-1(l₁+l₂+...+l_n) = (-1)ⁿa_n-1 = (-1)^n-1tr(A),

which implies tr(A) = l₁+....+l_n.

If we denote by l_i, i = 1,...,k, distinct eigenvalues of A with multiplicities m_i, then formula can be rewritten in the following form

tr(A) = m1l1+m2l2+...+mklk.

(9)

If l is an eigenvalue of a matrix A, we define a subset E_l by

E_l = {x Î Cⁿ: Ax = lx}.

Thus, E_l is the subset of all eigenvectors corresponding to the eigenvalue l. It immediately follows from the definition that if x₁, x₂ belong to E_l, and a₁, a₂ are arbitrary scalars, then a₁x₁+a₂x₂ belongs to E_l, i.e. E_l is a subspace of Cⁿ.

We have seen that there are at most n eigenvalues for a matrix of order n×n. As for eigenvectors, of course there are infinitely many eigenvectors, since any nonzero subspace contains infinitely many elements. An appropriate ''measure'' of the ''size'' of a space is its dimension. Being a subspace of Cⁿ, any maximal linearly independent subset in E_l contains not more than n elements, i.e. dim E_l £ n. The following theorem relates the dimension of E_l to the rank of the matrix A-lI.

Theorem 4 If l is an eigenvalue of a matrix A, then

dim El = n-rank(A-lI).

(10)

proof: For a vector x Î Cⁿ, we have x Î E_l if and only if x is a solution to a system of linear homogeneous equations (A-lI)x = 0. If we solve this system by the method of Chapter 2, namely by using elementary row transformations, then we see that if r = rank(A-lI), then the row reduce form of the matrix (A-lI) has precisely r nonzero rows and n-r zero rows. Thus, if x = (x₁,...,x_n) is a solution, then x_r+1,...,x_n can be chosen arbitrary and x₁,...,x_r are expressed through x_r+1,...,x_n, say x_i = f_i(x_r+1,...,x_n), i = 1,...,r, where f_i are linear functions of x_r+1,...,x_n. Thus, if we denote by e_k, k = 1,...,n, the standard basis in Cⁿ, then the general element of E_l is a linear comibinations of the elements

f_i = (f₁(e_r+1,...,f_r(e_n),0,...0)+e_i, i = r+1,...,n.

Thus E_l contains n-r linearly independent vectors, and every vector of E_l is a linear combination of those n-r vectors. Therefore, dim E_l = n-r.

Lemma 1 If l₁,l₂,...,l_k are distinct scalars, then the vectors

f_i = (1, l₁ⁱ,l₂ⁱ,...,l_nⁱ),i = 0,2,...,k-1,

are linearly independent.

Proof: Assume that

c₀f₀+c₁f₁+...+c_k-1f_k-1 = 0.

This implies that

c0+c1li+c2li^2+...+ck-1li^k-1 = 0

(11)

for all i = 1,2,...,k. Consider the polynomial of degree £ (k-1):

c0+c1 z+c2z^2+...+ck-1z^k-1.

(12)

Formula (?) implies that the polynomial (?) has k distinct roots. This is not possible unless all coefficients c_i = 0. This proves that f_i, i = 0,1,...,k-1, are linearly independent.

Corollary 1 If l₁,l₂,...,l_k are distinct scalars, then the matrix

V =

é
ê
ê
ê
ê
ê
ê
ê
ë

1	1	1	¼	1
l1	l2	l3	¼	lk
l1^2	l2^2	l3^2	¼	lk^2
¼	¼	¼	¼	¼
l1^k-1	l2^k-1	l3^k-1	¼	lk^k-1

ù
ú
ú
ú
ú
ú
ú
ú
û

(13)

is invertible.

Proof: By Theorem (?) the rank of the matrix (?) is the maximal number of linearly independent row vectors, which is n by Lemma ?. Hence, its rank is n, which implies that the matrix is invertible,

The determinant of the matrix in (?) is known as the Vandermonde determinant. It is known that the Vandermonde determinant is equal to Õ_{i < j} (l_j-l_i). From this it also follows that the Vandermonde determinant is nonzero if l_i are different, and hence the matrix (?) is invertible.

Theorem 5 Eigenvectors corresponding to different eigenvalues are linearly independent.

Proof: We need to show that if Ax_i = l_ix_i, i = 1,2,...,k, and if l_i are all different, then x₁, x₂,...,x_k are linearly independent. Assume that

a1x1+a2 x2+...+akxk = 0

(14)

Applying A to (?) we have

a1l1x1+l2a2 x2+...+lkakxk = 0
a1l1^2x1+l2 a2 x2+...+lk^2akxk = 0
a1lk-1x1+l2^k-1a2 x2+...+lk^k-1akxk = 0	(15)

Equations can be rewritten in the following form

é
ê
ê
ê
ê
ê
ê
ê
ë

1	1	1	¼	1
l1	l2	l3	¼	lk
l1^2	l2^2	l3^2	¼	lk^2
¼	¼	¼	¼	¼
l1^k-1	l2^k-1	l3^k-1	¼	lk^k-1

ù
ú
ú
ú
ú
ú
ú
ú
û

é
ê
ê
ê
ê
ê
ê
ê
ë

a₁x₁

a₂x₂

·

·

a_kx_k

ù
ú
ú
ú
ú
ú
ú
ú
û

=

é
ê
ê
ê
ê
ê
ê
ê
ë