# LINEAR ALGEBRA NOTES

8. Vector spaces

**vector space: **set V of vectors with verctor addition and scalar
multiplication satisfying

for all and

**examples:** ,**P**
polynomials, polynomials with degree less
than n, sequences,

sequences converging to 0, functions on **R**, **C**(**R**) continuous
functions on **R**, solutions of

homogeneous systems

**subspace of V** : subset W of V that is a vector space with same operations

**
proper subspace of V :** subspace but not
and not

**V**

**examples:**

W = and W = V , subspaces of V

W = lines through origin, subspace of

W =planes through origin, subspace of

W =diagonal n × n matrices, subspace of

W = , subspace of V where

W =convergent sequences, subspace of V =sequences

W =continuous functions on

**R**, subspace of V =functions on

**R**

**fact:**subset W of V is a subspace of V iff

**nonempty:**

**closed under addition:**

**closed under scalar multiplication :**

9. Linear independence

** linearly independent: **
implies

**linearly dependent:** not independent

**parallel vectors:** one is scalar multiple of the other

notation

** properties :**

u, v linearly independent

vectors are dependent i one of them is linear combination of the others

subset of lineraly independent set is linearly independent

columns of matrix A are independent i AX = 0 has only trivial solution

columns of square matrix A are independent i A invertible iff detA ≠ 0

independent,
implies
independent

independent,
implies

rows of row echelon matrix are independent

leading columns of echelon matrix are independent

10. Bases

**S spans W: **spanS = W

S is a spanning set of W

**basis of V :** linearly independent spanning set of V

maximal independent set in V

minimal spanning set of V

**standard bases for V :
**

**properties:
**

implies T dependent

all bases of V has same number of vectors

**dimension of V :**dimV =number of vectors in a basis of V

**examples:**

**properties:**

W proper subspace of V implies dimW < dimV

independent subset of V can be extended to a basis of V

spanning set of V contains a basis of V

11. row, column and null spaces

**notation:**sizeA = m× n

**row space of A:**RowA =subspace of spanned by rows of A

**row rank of A:**dim RowA

**column space of A:**ColA =subspace of spanned by columns of A

**column rank of A:**dim ColA

**algorithm for basis of RowA:**

(i) reduce A to echelon form B

(ii) take nonzero row vectors of B

**algorithm for basis of ColA:**

(i) reduce A to echelon form B

(ii) take columns of A corresponding to leading columns of B

**fact:**row rank A equals column rank A

**rank A:**this common value

**null space of A:**NullA = = solution set of homogeneous system, subspace of

**properties:**

A, B row equivalent implies RowA = RowB

A, B row equivalent implies colums of A and columns of B have the same dependence relations

Ax = b consistent iff b ∈ ColA

rankA + dim NullA = n

12. Coordinates

**notation:** bases for
eng standard basis for V

**fact:** each v ∈ V can be written uniquily as

** coordinates of v in basis B:**

**huge fact:** is an isomorphism (
are the 'only' finite dimensional vector spaces)

**transition matrix from basis B to basis** **D:**
square matrix

**properties:**

**algorithm for finding a basis for**
**
in V :**

(i) find a bases B for V (use standard if possible)

(ii) put the coordinates of the v_{i}'s as rows (columns) for a matrix A

(iii) find a basis for the rowspace (columnspace) of A

(iv) use this basis as coordinates to build the basis of W

13. Linear transformations

**notation:** basis for
basis for W, E standard basis for V

**linear transformation:** L : V → W such that for all

L(u + v) = L(u) + L(v) additive

multiplicative

**kernel:**

**range:**

**L is one-to-one (1-1):** L(u) = L(v) implies u = v

**L is onto W:** ranL = W

**properties:
**

kerL subspace of V

ranL subspace of W

L is 1-1 i

**matrix of L:**

properties:

properties:

**R, S are similar matrices:**S = P

^{-1}RP for some P

**fact:**R, S are similar iff for some L : V → V and bases B, D for V

(P is the transition matrix)

**rank of L:**rankL = dim ranL

**properties:**

[ranL]D = ColM

[kerL]B = NullM

rankL = rankM

dim kerL = dim nullM

rankL + dim kerL = dimV

14. Eigenvalues and eigenvectors

**notation:** L : V → V linear transformation,
coordinates of u

**eigenvalue problem:
**

transformation version

**eigenvalue:**λ

**eigenvector of L associated to λ:**u

**eigenspace associated to λ:**

matrix version

**eigenvalue:**λ

**eigenvector of A associated to λ:**x

**eigenspace associated to λ :**

**characteristic polynomial:**

if A~ B then charpoly(A) = charpoly(B)

**characteristic equation :**λ

**eigenvalue of A iff**

15. Diagonalization

**A diagonalizable:**A similar to diagonal matrix

**fact:**implies

is a basis of eigenvectors with associated eigenvalues in the diagonal od D

**properties:**

if eigenvectors associated to distinct eigenvalues then they are independent

if sizeA = n × n and A has n distinct eigenvalues then A diagonalizable

distinct eigenvalues, bases for eigenspaces implies is independent

**algorithm for diagonalization:**

(i) solve charachteristic equation to find eigenvalues

(ii) for each eigenvalue nd basis of associated eigenspace

(iii) if the union of the bases is not a basis for the vectorspace than not diagonalizable

(iv) build P from the eigenvectors as columns

(v) build D from the corresponding eigenvalues

16. Bilinear functional

**product of U and V :**

**bilinear functional on V :**such that for all and

**fact:**Every bilinear functional f on is for some

where

The bilinear functional f can be

**symmetric:**f(u, v) = f(v, u) for all

**positive semi definite:**f(v, v) ≥0 for all v ∈ V

**positive definite:**f(v, v) > 0 for all

**negative semi definite:**f(v, v) ≤0 for all v∈ V

**negative definite:**f(v, v) < 0 for all

**indefinite:**neither positive nor negative semidefinite

17. Inner product

**inner product: **symmetric, positive definite, bilinear functional

**examples of inner products:
**

**dot product (standard inner product) on :**

**standard inner product on C**[0, 1]: (continuous functions on [0,1]),

**fact:**every inner product on is where A is a symmetric (therefore diagonalizable)

matrix with positive eigenvalues and

**length (norm):**

**unit vector:**

**unit vector in the direction of v:**

**distance:**

**angle:**

**orthogonal:**iff iff

**orthogonal:**for all i, j

**fact:**nonzero orthogonal vectors are independent

**orthonormal:**S is orthogonal and for all i

**Cauchy- Schwartz inequality :**

**Triangle inequality:**

**Pythagorean theorem:**implies

**orthogonal complement:**, W is subspace of V

**properties:**W is subspace of

is a subspace

W = span(S), for all i implies

(basis of W) [ (basis of ) is basis of

18. Orthogonal bases and Gram-Schmidt algorithm

**fact:**orthogonal basis for a subspace W of V , y ∈ V

if such that and then and

**orthogonal projection:**= the unique p ∈ W such that

**Gram-Schmidt algorithm:**for finding an orthogonal basis for

(i) make independent if necessary

(ii) let

(ii) inductively let

**fact:**

19. Least square solution and linear regression

**fact:**if W subspace of V , w ∈ W, y ∈ V then is minimum when

**fact:**is minimum

least square regression line ax + b: data

, β makes minimum, that is,

Prev | Next |