# Matrix Algebra Review

**Definition.** An m*n matrix, A_{m*n}, is a rectangular array of real
numbers with m rows and n

columns. Element in the i^{th} row and the j^{th} column is denoted by a_{ij} .

**Definition.** A vector a of length n is an n*1 matrix with each element
denoted by a_{i}. The i^{th}

element is called the i^{th} component of the vector and n is the
dimensionality.

1. Two matrices A and B of the same dimensions can be added . The sum **A + B**
has (i, j) entry

a_{ij} + b_{ij} . So

2. A matrix may also be multiplied by a constant c. The product cA is the
matrix that results

from multiplying each element of A by c. Thus

3. The transpose operation A ^{T} or A' of a matrix changes the columns into rows
so that the first

column of A becomes the first row of A^{T} , the second column becomes second row,
and etc.

So the (i,j)^{th} element in A_{m*n} becomes the (j,i)^{th}
in the transpose

4. We can define matrix multiplication A B if the number of elements in a row
of A is the same

as the number of elements in the columns of B. E.g. when A is (p*k) and B is
(k*n). An

element of the new matrix AB is formed by taking the inner product of each row
of A with

each column of B. The matrix product AB is

**Special Square Matrix **

1. A square matrix A is said to be symmetric if A = A^{T} or a_{ij}=
a_{ji} for all i and j.

is symmetric,

is not symmetric

2. Diagonal Matrix.

3. Identity Matrix.

The identity matrix is a square matrix with ones on the diagonal and zeros
elsewhere . It

follows from the definition of matrix multiplication that the (i, j) enrty of AI
is

.
So **AI = A** . Similarly, **IA = A** .

Therefore matrix I acts like 1 in ordinary multiplication.

The fundamental scalar relation about the existence of an inverse number a^{-1}
such that

, has the following matrix algebra extension .

then B is called the inverse of A and is denoted by A^{-1}.

**Other Matrix Properties **

1. **Trace. **The sum of the diagonal elements,

2. A square matrix that does not have a matrix inverse is called a singular
matrix

The inverse of a 2*2 matrix is given by

3. A matrix is singular if and only if its determinant is 0. The determinant
of a matrix A is

denoted as |A| The determinant of a 2*2 matrix is given by

**Examples 1. Simultaneous equations **

We can rewrite the above three equations as a single matrix equation.

**Example 2. Variance/Covariance Matrix**

For a vector of random variables,
,
we can write a matrix containing their variances

and their covariances. Let
be the variance of Y_{i} and let cov_{ij} be the covariance between Y_{i} and

Y_{j} , i < j. Then the variance/covariance matrix for
is

Also note that the above can be written as

where is the
correlation of Y_{i} and Y_{j} . Note that all of these matrices
are symmetric. Furthermore,

the terms on the diagonal of the variance/covariance matrix must be positive and
terms off

the diagonal of the correlation matrix are bounded by -1 and 1.

**Example 3. Multiple Linear Regression **

We have a response Y and a set of p independnet variables
X _{1},X_{2}, . . . ,X_{p}. Assume we have n

observations and for each i = 1, . . . , n observation, we assume

where Normal (0,).

We can represent n observations simultaneously in matrix form as

The residual can be written as

The residual sum of squares (RSS) is

Minimizing the above RSS gives the usual least -squared estimate for β

Prev | Next |