Try our Free Online Math Solver!

Matrix Algebra Review
Definition. An m*n matrix, A_{m*n}, is a rectangular array of real
numbers with m rows and n
columns. Element in the i^{th} row and the j^{th} column is denoted by a_{ij} .
Definition. A vector a of length n is an n*1 matrix with each element
denoted by a_{i}. The i^{th}
element is called the i^{th} component of the vector and n is the
dimensionality.
Matrix Operations
1. Two matrices A and B of the same dimensions can be added. The sum A + B
has (i, j) entry
a_{ij} + b_{ij} . So
2. A matrix may also be multiplied by a constant c. The product cA is the
matrix that results
from multiplying each element of A by c. Thus
3. The transpose operation A ^{T} or A' of a matrix changes the columns into rows
so that the first
column of A becomes the first row of A^{T} , the second column becomes second row,
and etc.
So the (i,j)^{th} element in A_{m*n} becomes the (j,i)^{th}
in the transpose
4. We can define matrix multiplication A B if the number of elements in a row
of A is the same
as the number of elements in the columns of B. E.g. when A is (p*k) and B is
(k*n). An
element of the new matrix AB is formed by taking the inner product of each row
of A with
each column of B. The matrix product AB is
Special Square Matrix
1. A square matrix A is said to be symmetric if A = A^{T} or a_{ij}= a_{ji} for all i and j.
is symmetric,
is not symmetric
2. Diagonal Matrix.
3. Identity Matrix.
The identity matrix is a square matrix with ones on the diagonal and zeros
elsewhere . It
follows from the definition of matrix multiplication that the (i, j) enrty of AI
is
.
So AI = A . Similarly, IA = A .
Therefore matrix I acts like 1 in ordinary multiplication.
The fundamental scalar relation about the existence of an inverse number a^{1}
such that
, has the following matrix algebra extension .
then B is called the inverse of A and is denoted by A^{1}.
Other Matrix Properties
1. Trace. The sum of the diagonal elements,
2. A square matrix that does not have a matrix inverse is called a singular
matrix
The inverse of a 2*2 matrix is given by
3. A matrix is singular if and only if its determinant is 0. The determinant
of a matrix A is
denoted as A The determinant of a 2*2 matrix is given by
Examples 1. Simultaneous equations
We can rewrite the above three equations as a single matrix equation.
Example 2. Variance/Covariance Matrix
For a vector of random variables ,
,
we can write a matrix containing their variances
and their covariances. Let
be the variance of Y_{i} and let cov_{ij} be the covariance between Y_{i} and
Y_{j} , i < j. Then the variance/covariance matrix for
is
Also note that the above can be written as
where is the
correlation of Y_{i} and Y_{j} . Note that all of these matrices
are symmetric. Furthermore,
the terms on the diagonal of the variance/covariance matrix must be positive and
terms off
the diagonal of the correlation matrix are bounded by 1 and 1.
Example 3. Multiple Linear Regression
We have a response Y and a set of p independnet variables
X _{1},X_{2}, . . . ,X_{p}. Assume we have n
observations and for each i = 1, . . . , n observation, we assume
where Normal (0,).
We can represent n observations simultaneously in matrix form as
The residual can be written as
The residual sum of squares (RSS) is
Minimizing the above RSS gives the usual least squared estimate for β
Prev  Next 