Matrix Approach to Simple Linear Regression
Use of Inverse
• Inverse similar to using reciprocal of a scalar
• Pertains to a set of equations
• Assuming A has an inverse:
Random Vectors and Matrices
• Contain elements that are random variables
• Can compute expectation and (co)variance
• In regression set up,
both ε and Y are random
vectors
• Expectation vector:
• Covariance matrix: symmetric
Basic Theorems
• Consider random vector Y
• Consider constant matrix A
• Suppose W = AY
– W is also a random vector
– E(W) = AE(Y)
– 
Regression Matrices
• Can express observations
Y = Xβ +ε
• Both Y and ε are random vectors
E(Y)
= Xβ+E(ε)
= Xβ
• Express quantity Q
• Taking derivative →
• This means 
• The fitted values 
• Matrix 
is called the hat matrix
– H is symmetric, i.e., H' = H
– H is idempotent, i.e., HH = H
• Matrix H used in diagnostics (chapter 9)
Residuals
• Residual matrix
• e a random vector
ANOVA
• Quadratic form defined as
where A is symmetric n × n matrix
• Sums of squares can be shown to be quadratic forms (page
207)
• Quadratic forms play significant role in the theory of linear
models when errors are normally distributed
Inference
• Vector 
• The mean and variance are
• Thus, b is multivariate Normal
• Consider 
• Mean response 
• Prediction of new observation
Chapter Review
• Review of Matrices
• Regression Model in Matrix Form
| Prev | Next |