Real Vector Spaces

Concept and calculation rules

Introduction   

Many math structures are different at first sight, but looking deeper the resemblance is astonishing.
The benefits of the study of an abstract structure is that all the properties can be applied to all the representations of that structure.
The concept of a 'real vector space' is an abstract structure in that way.

Concept

We start with a set V and the field of real numbers R. We define the concept 'vector space' by means of postulates.
We say V is a vector space if and only if

  1. There is an addition '+' in V such that V,+ is a commutative group.
  2. Any element v in V and any r in R determine a scalar product rv in V.
    This scalar product has the following properties for any r,s in R and any v,w in V.
  3. r(sv) = (rs)v
  4. r(v + w) = rv + rw
  5. (r + s)v = rv + sv
  6. 1v = v

Any element of a vector space is called a vector.
The identity element of the group V,+ is called the vector 0.
The symmetric element of v is called the opposite vector -v.
The subtraction v - v' is defined by v - v' = v + (-v') .
Examples of real vector spaces are

Go to the top of the page

Calculation rules

Deducing from the postulates of a vector space, one can prove the following calculation rules. They hold for each vector u,v in V, and for each r,s in R.

 Go to the top of the page

Subspaces

definition

M is a subspace of a vector space V
if and only if

Go to the top of the page

Criterion

Theorem:

A non-void subset M of a vector space V, is a vector space
if and only if
rx + sy is in M for any r,s in R and any x,y in M.


Part 1: first we prove that
If M is a vector space then rx + sy is in M for any r,s in R and any x,y in M.
Well, if M is a vector space then, from the postulates, rx and sy are in M and therefore rx + sy is in M .
Part 2: we prove that
If rx + sy is in M for any r,s in R and any x,y in M, then M is a vector space.

Q.E.D.

 

Go to the top of the page

Intersection of two spaces

Theorem:

The intersection of two subspaces M and N of V, is itself a subspace of V. .


Proof:
Since 0 is in M and in N, 0 is in the intersection.
For any r,s in R and any x, y in the intersection of M and N, we can state:
(rx + sy is in M) and (rx + sy is in N); so it is in the intersection.
Appealing on previous criterion the intersection of M and N is a subspace of V.

 

Go to the top of the page

Generators of a vector space

Linear combinations of vectors

Take from a vector space V, the vectors a,b,c,d ... ,l .
Take as much real numbers r,s,t, ... ,z .
Then we call ra + sb + tc + ... + zl a linear combination of the vectors a,b,c,d ... ,l.

Go to the top of the page

Generating a vector space

Let D = { a,b,c,d ... ,l } a fixed set of vectors from V.
Let M be the set of all possible linear combinations of the vectors of D.
We'll show that M is a vector space.
Indeed, appealing on previous criterion, take two vectors u,v from M.
For any r,s in R, we have that ru + sv is a linear combination of two linear combinations of a,b,c,d ... ,l. So ru + sv is itself a linear combination of a,b,c,d ... ,l and therefore ru + sv is in M.

Since each vector space containing the vectors a,b,c,d ... ,l must contain each linear combination of these vectors, M is the 'smallest' vectorspace genereted by D.

Conclusions and definitions:

All linear combinations of vectors of D = { a,b,c,d ... ,l } generate a vector space M.
The elements of D are called, generators of M.
M is called the vector space spanned by D.
The vector space spanned by the vectors a,b,c,d ... ,l is span(D).
It is the smallest vectorspace containing the set D.

Go to the top of the page

Properties of a generating set

Say D is a subset of vector space V and M = span(D). It is easy to see that:

Go to the top of the page

Linear dependent

Linear dependent vectors

A set D of vectors is called dependent, if there is at least one vector in D, that can be written as a linear combination of the other vectors of D.
A set of one vector is called dependent, if and only if it is the vector 0.

 

Linear independent vectors

A set D of vectors is called independent, if and only if that set is not a dependent set. Such set is called a free set of vectors.

 

Criterion for linear dependent vectors

Theorem:

Take a set D = {a,b,c,..,l} of (more than one) vectors from a vector space V. That set D is linear dependent
if and only if
there is a suitable set of real numbers r,s,t, ... ,z , not all zero,
such that ra + sb + tc + ... + zl = 0


Proof:
Part 1 :
If the set D is dependent, there is at least one vector in D, say b, who is a linear combination of the other vectors of D.
Then b = ra + tc + ... + zl
<=> ra + (-1)b + tc + ... + zl = 0
So, there is a suitable set of real numbers r,s = -1,t, ... ,z , not all zero,
such that ra + sb + tc + ... + zl = 0
Part 2 :
If there is a suitable set of real numbers r,s,t, ... ,z , not all zero,
such that ra + sb + tc + ... + zl = 0 , then we can choose a nonzero coefficient, say s , and then ra + tc + ... + zl = -sb . Dividing both sides by (-s), we see that b is a linear combination of the other vectors of D. So, D is an dependent set.

 

Go to the top of the page

Corollary:

Take a set D = {a,b,c,..,l} of (more than one) vectors from a vector space V.
That set D is linear independent
if and only if
ra + sb + tc + ... + zl = 0 => r = s = t = ... = z = 0

 

Second Criterion for linear dependent vectors

Take an ordered set D = {a,b,c,..,l} of (more than one) vectors from a vector space V.
That set D is linear dependent
if and only if
there is at least one vector, who is a linear combination of the PREVIOUS vectors in D.
Proof:
Part 1:
If the set D is linear dependent, we know from the first criterion that there is a suitable set of real numbers r,s,t, ... ,z , not all zero, such that ra + sb + tc + ... + vi + wj + ... + zl = 0
Say w is the last non-zero coefficient, then
ra + sb + tc + ... + vi + wj = 0 <=> -wj = ra + sb + tc + ... + vi
Dividing both sides by (-w), we see that vector j is a linear combination of the PREVIOUS vectors of D.
Part 2:
If a vector of D is a linear combination of the PREVIOUS vectors of D, then it is a linear combination of the other vectors of D (with coefficients 0 for the following vectors). Thus, D is a linear dependent set.

 

Go to the top of the page

Basis and dimension of a vector space

Minimal generating set and basis

Say M = span(D).
D is a generating set of M. We know that, if there is a vector in D, that is a linear combination of the other vectors in D, and if we remove that vector from D, then still M = span(D).
Now remove, one after another, the vectors from D who are a linear combination of the others. For the remaining part D' still holds M = span(D'), but the vectors in D' are now linear independent. D' is a free set that spans M.
Such a minimal generating set of M is called a BASIS of M.
In this introduction, we restrict the theory to vector spaces with a finite basis.

 

Go to the top of the page

Coordinates in a vector space

Say D = {a,b,c,..,l } is a ordered basis of M.
Each vector v in M can be written as a linear combination of the vectors in D. Assume v can be written in two ways as a linear combination of the vectors in D, then we have
v = ra + sb + tc + ... + zl = r'a + s'b + t'c + ... + z'l and then
(r - r')a + (s - s')b + (t - t')c + ... + (z - z')l = 0
But, appealing on the criterion of linear independent vectors, all the coefficients must be 0.
So, a = a' ; b = b' ...
Conclusion:
Each vector v of M is uniquely expressible is a linear combination of the vectors of the ordered basis D. The unique coefficients are called the coordinates of v with respect to D.
We write co(v) = (r,s,t, ... ,z) .

 

Go to the top of the page

Properties of coordinates

It is easy to verify that
co(a + b) = co(a) + co(b)
co(ra) = r.co(a)
with a,b in M and r in R.

 

Go to the top of the page

Two bases of V have exactly the same number of elements

Suppose there are two bases, B1 and B2, with a different number of elements.
Assume B1 = {a,b,c,d} and B2 = {u,v,w}
Then, span(B2) = V . Now, we have V = span{d,u,v,w} and {d,u,v,w} is a linear dependent set. It contains at least one vector, say v, who is a linear combination of the previous vectors. We can omit this vector and then V = span{d,u,w} .
Again, V = span{c,d,u,w} and {c,d,u,w} is a linear dependent set. It contains at least one vector,say w, which is a linear combination of the previous vectors. That vector can't be d, because c and d are independent (as a part of a basis). We can omit this vector and then V = span{c,d,u} .
Again, V = span{b,c,d,u} and {b,c,d,u} is a linear dependent set. It contains at least one vector, which is a linear combination of the previous vectors. That vector can't be b, c or d, because b, c and d are independent (as a part of a basis). That vector must be u! We can omit this vector and then V = span{b,c,d} .
Again, V = span{a,b,c,d} and {a,b,c,d} is a linear dependent set. This is impossible because it is a basis.
From all this we see that is is impossible that two bases, B1 and B2, have a different number of elements.

 

Go to the top of the page

Dimension of a vector space

Since a vector space has a constant number of vectors in a basis, that number n is characteristic for that space and is called the dimension of that space. We write dim(V) = n. Note that if D spans V, the linear independent vectors of D form a basis of V.

Corollary

If dim(V) = n, then

Go to the top of the page

Vector spaces and matrices

Row space of a matrix

Say A is a m x n matrix. The rows of that matrix can be viewed as a set D of vectors, of the vector space of all n-tuples of real numbers.
The space generated by D is called the row space of A. The rows of A are a generating set of the row space.
From the properties of generating parts, we have :
The row space of A does not change if we

So, such a row transformation does not change the row space of A.
The dimension of the row space, is the number of independent rows of A.

 

Go to the top of the page

Dimension of a row space

We know that it is possible to transform a matrix A, by suitable row transformations, to a row canonic matrix. Then the non-zero rows are linear independent and form a basis of the row space. But the number of non-zero rows is the rank of A. Hence, we can say that the rank of A is the dimension of the row space of A.
It can be proved that the non-zero rows of the canonic matrix form a unique basis for the row space.
Corollary : the rank of A is the number of linear independent rows.

Column space of a matrix

Say A is a m x n matrix. The columns of that matrix can be viewed as a set D of vectors of the vector space of all m-tuples of real numbers.
The space generated by D is called the column space of A. The columns of A are a generating set of the column space.
From the properties of generating parts, we have :
The column space of A does not change if we

So, such a column transformation does not change the column space of A.
The dimension of the column space, is the number of independent columns of A.

 

Go to the top of the page

Dimension of a column space

We know that it is possible to transform a matrix A, by suitable column transformations, to a column canonic matrix. Then the non-zero columns are linear independent and form a basis of the column space. But the number of non-zero columns is the rank of A. Thus, we can say that the rank of A is the dimension of the column space of A. It can be proved that the non-zero columns of the canonic matrix form a unique basis for the column space. Corollary :

Go to the top of the page

Coordinates and changing a basis

We'll show the properties in a vector space with dimension 3, but it can be extended to vector spaces with dimension n.
Take an ordered basis (u,v,w) of V. Then each vector s has coordinates (x,y,z) with respect to this basis. If we take another basis (u',v',w'), then s has other coordinates (x',y',z') with respect to that new basis.
Now, we'll investigate the link between these two ordered sets of coordinates.
We know that s = xu + yv + zw = x'u' + y'v' + z'w'. But the vectors of the new basis (u',v',w'), also have coordinates with respect to the old basis (u,v,w).
co(u') = (a,b,c) => u' = au + bv + cw
co(v') = (d,e,f) => v' = du + ev + fw
co(w') = (g,h,i) => w' = gu + hv + iw
Then 

 
s = x' (au + bv + cw) + y' (du + ev + fw) + z' (gu + hv + iw)
  = (ax' + dy' + gz')u + (bx' + ey' + hz')v + (cx' + fy' + iz')w
but from above we have also
s = xu  +  yv  + zw

Therefore, the relation between the coordinates is 

 
x = ax' + dy' + gz'
y = bx' + ey' + hz'
z = cx' + fy' + iz'

These relations can be written in matrix notation. 

 
[x]   [a  d  g] [x']
[y] = [b  e  h].[y']
[z]   [c  f  i] [z']
 
[a  d  g]
[b  e  h] is called the transformation matrix.
[c  f  i]

The columns of the transformation matrix are the coordinates of the new basis with respect to the old basis.

 

Go to the top of the page

Vector spaces and systems of linear equations

Vector spaces and homogeneous systems

Take a homogeneous system of linear equations in n unknowns. Each solution of that system can be viewed as a vector from the vector space V of all the real n-tuples.
Each real multiple of that solution is a solution too, and the sum of two solutions is a solution too. Therefore, all the solutions of the system form a subspace M of V. It is called the solution space of the system.

Basis of a solution space

By means of an example, we show how a basis of a solution space can be found. 

 
/ 2x + 3y - z + t = 0
\ x - y + 2z - t = 0

This is a system of the second kind.
x and y can be taken as main unknowns.
z and t are the side unknowns.
The solutions are 

 
x = -z + (2/5)t
y =  z - (3/5)t
The set of solutions can we written as
(-z + (2/5)t , z - (3/5)t , z , t )  with z and t in R
 
<=>
 
z(-1,1,1,0) + t(2/5,-3/5,0,1)    with z and t in R
 

Hence, all solutions are linear combinations of the linear independent vectors (-1,1,1,0) and (2/5,-3/5,0,1).
These two vectors constitute a basis of the solution space.

 

Go to the top of the page

Solutions of a non homogeneous system

We can denote such system shortly as AX = B, with coefficient matrix A, the column matrix B of the known terms and X is the column matrix of the unknowns.
Consider also the corresponding homogeneous system AX = 0 with the same A and X as above.
If X' is a fixed solution of AX = B then AX' = B .
If X" is a arbitrary solution of AX = 0 then AX" = 0 .
Then, AX' + AX" = B <=> A(X' + X") = B <=> X' + X" is a solution of AX = B.
Conclusion: If we add an arbitrary solution of AX = 0 to a fixed solution of AX = B then X' + X" is a solution of AX = B.
Furthermore:
If X' is a fixed solution of AX = B then AX' = B .
If X" is a arbitrary solution of AX = B then AX" = B .
Then, AX" - AX' = 0 <=> A(X" - X') = 0 <=> X" - X' is a solution of AX = 0.
<=> X" = X' + (a solution of AX = 0).
Conclusion:
Any arbitrary solution of AX = B is the sum of a fixed solution of AX = B and a solution of AX = 0
So, if we have a fixed solution of AX = B and we add to this solution all the solutions of the corresponding homogeneous system one after another, then we get all solutions AX = B.
Example: 

 
/ 2x + 3y - z + t = 0
\ x - y + 2z - t = 0

Above we have seen that the solutions are z(-1,1,1,0) + t(2/5,-3/5,0,1). 

 
/ 2x + 3y - z + t = 5
\ x - y + 2z - t = 0

has a solution (1,1,0,0) .
All solutions of the last system are (1,1,0,0) + z(-1,1,1,0) + t(2/5,-3/5,0,1).

 

Go to the top of the page

Sum of two vector spaces

Say A and B are subspaces of a vector space V.

We define the sum of A and B as the set

{ a + b with a in A and b in B }

We write this sum as A + B.

 

The sum as subspace

Theorem: The sum A+B, as defined above, is a subspace of V.

Proof:

For all a1 and a2 in A and all b1 and b2 in B and all r, s in R we have 

 
r(a1 + b1) + s(a2 + b2)  = (r a1 + s a2) + (r b1 + s b2)
 
   is in A + B.
 
Go to the top of the page

Direct sum of two vector spaces

The sum A+B, as defined above, is a direct sum if and only if the vector 0 is the only vector common to A and B.

Example

 
In the space R3
    A = span{ (3,2,1) }
    B = span{ (2,1,4) ; (0,1,3) }
Investigate if A+B is a direct sum

 

 
Say r,s,t are real numbers, then
each vector in space A is of the form r.(3,2,1) and
each vector in space B is of the form s.(2,1,4) + t.(0,1,3) .
 
For each common vector, there is a suitable r,s,t such that
 
    r.(3,2,1) = s.(2,1,4) + t.(0,1,3)
 
<=>
 
    r.(3,2,1) - s.(2,1,4) - t.(0,1,3) = (0,0,0)
 
<=>
    / 3r - 2s     = 0
    | 2r -  s - t = 0
    \ r  - 4s -3t = 0
 
 
Since |3  -2   0|
      |2  -1  -1| is  not 0,
      |1  -4  -3|
 
the previous system has only the solution r = s = t = 0.
 
The vector (0,0,0) is the only common vector of A and B.
Thus, A+B is a direct sum.
 
Go to the top of the page

Property

If A + B is a direct sum, then each vector v in A+B can be written, in just one way, as the sum of an element of A and an element of B.

 

Proof: 

 
Suppose v = a1 + b1 = a2 + b2  with ai in A and bi in B.
 
Then   a1 - a2 = b2 - b1 and a1 - a2 is in A and b2 - b1 is in B
 
Therefore a1 - a2 = b2 - b1 is a common element of A and B.
 
But the only common element is 0.
 
So,  a1 - a2 = 0 and b2 - b1 = 0
 
and   a1 = a2  and b2 = b1
 
Go to the top of the page

Supplementary vector spaces

Say that vector space V is the direct sum of A and B, then 

 
A is the supplementary vector space of B with respect to V.
 
B is the supplementary vector space of A with respect to V.
 
A and B are supplementary vector spaces with respect to V.
 

 

Basis and direct sum

Theorem:

Say V is the direct sum of the spaces M and N.

If {a,b,c,..,l } is a basis of M and {a',b',c',..,l' } is a basis of N,
then {a,b,c,..,l,a',b',c',..,l' } is a basis of M+N.


Proof:
Each vector v of V can be written as m + n with m in M and n in N.
Then m = ra + sb + tc + ... + zl and n = r'a' + s'b' + t'c' + ... + z'l' , with r,s,t,...z,r',s',t',...z' real coefficients.

Thus each vector v = ra + sb + tc + ... + zl + r'a' + s'b' + t'c' + ... + z'l'
Therefore the set {a,b,c,..,l,a',b',c',..,l'} generates V.

If ra + sb + tc + ... + zl + r'a' + s'b' + t'c' + ... + z'l' = 0 , then ra + sb + tc + ... + zl is a vector m of M and
r'a' + s'b' + t'c' + ... + z'l' is a vector n of N.

From a previous theorem we know that we can write the vector 0 in just one way, as the sum of an element of M and an element of N. That way is 0 = 0 + 0 with 0 in M and 0 in N.

From this we see that necessarily m = 0 and n = 0 and thus ra + sb + tc + ... + zl = 0 and
r'a' + s'b' + t'c' + ... + z'l' = 0

Since all vectors in these expressions are linear independent, it is necessarily that all coefficients are 0 and from this we know that the generating vectors {a,b,c,..,l,a',b',c',..,l'} are linear independent.

Go to the top of the page

Dimension of a direct sum

From previous theorem it follows that
dim(A+B) = dim(A) + dim(B)

Converse theorem

If {a,b,c,..,l } is a basis of M and {a',b',c',..,l' } is a basis of N, and {a,b,c,..,l,a',b',c',..,l' } are linear independent,
then M+N is a direct sum.


Proof:
Each element m of M can be written as ra + sb + tc + ... + zl .
Each element n of N can be written as r'a' + s'b' + t'c' + ... + z'l' .
For a common element we have 

 
  ra + sb + tc + ... + zl  = r'a' + s'b' + t'c' + ... + z'l'
 
<=>
 ra + sb + tc + ... + zl - r'a' - s'b' - t'c' - ... - z'l' = 0

Since all vectors are linear independent, all coefficients must be 0. The only common vector is 0.

 

Go to the top of the page

Direct sum criterion

From the two previous theorems we deduce that:

If {a,b,c,..,l } is a basis of M and {a',b',c',..,l' } is a basis of N, then 

 
M+N is a direct sum   <=> {a,b,c,..,l,a',b',c',..,l' } are linear independent
 

 

Projection in a vector space

Choose two supplementary subspaces M and N with respect to the space V. Each vector v of V can be written in exactly one way as the sum of an element m of M and an element n of N.

Then v = m + n .

Now we can define the transformation 

 
p: V --> V : v --> m
 
We define this transformation as
 
        the projection of V on M with respect to N
 
Go to the top of the page

Projection, example

V is the space of all polynomials with a degree not greater than 3.
We define two supplementary subspaces
M = span { 1, x }
N = span { x2, x3 }
Each vector of V is the sum of exactly one vector of M and of N.
e.g. 2x3 - x2 + 4x - 7 = (2x3 - x2) + (4x - 7)

Say p is the projection of V on M with respect to N, then 

 
    p(2x3 - x2 + 4x - 7 ) = 4x - 7

Say q is the projection of V on N with respect to M, then 

 
    q(2x3 - x2 + 4x - 7 ) = 2x3 - x2
 
Go to the top of the page

Similarity transformation of a vector space

Let r = any constant real number.
In a vector space V we define the transformation 

 
   h : V --> V : v --> r.v

We say that h is a similarity transformation of V with factor r.

Important special values of r are 0, 1 and -1.

Reflection in a vector space

Choose two supplementary subspaces M and N with respect to the space V.
Each vector v of V is the sum of exactly one vector m of M and n of N.

Now we define the transformation 

 
    s : V --> V : v --> m - n

We say that s is the reflection of V in M with respect to the N.

This definition is a generalisation of the ordinary reflection in a plane. Indeed, if you take the ordinary vectors in a plane and if M and N are one dimensional supplementary subspaces, then you'll see that with the previous definition, s becomes the ordinary reflection in M with respect to the direction given by N.

Go to the top of the page

Example of a reflection

 
Take V = R4.
 
M = span{(0,1,3,1);(1,0,-1,0)}
 
N = span{(0,0,0,1);(3,2,1,0)}

It is easy to show that M and N have only the vector 0 in common. (This is left as an exercise.) So, M and N are supplementary subspaces.

Now we'll calculate the image of the reflection of vector v = (4,3,3,1) in M with respect to N.

First we write v as the sum of exactly one vector m of M and n of N. 

 
 (4,3,3,1) = x.(0,1,3,1) + y.(1,0,-1,0) + z.(0,0,0,1) + t.(3,2,1,0)

The solution of this system gives x = 1; y = 1; z = 0; t = 1. The unique representation of v is 

 
(4,3,3,1) = (1,1,2,1) + (3,2,1,0)

The image of the reflection of vector v = (4,3,3,1) in M with respect to N is vector v' = 

 
   (1,1,2,1) - (3,2,1,0) = (-2,-1,1,1)
 
Go to the top of the page

You can find solved problems about vector spaces here