Introduction to numerical methods

INTRODUCTION, APPROXIMATION & ERRORS

Chapter 01.01 Introduction to numerical methods 1
Multiple-choice test 7
Problem set 9

Chapter 01.02 Measuring errors 11
True error 11
Relative true error 12
Approximate error 13
Relative approximate error 14
Significant digits 15
Multiple -choice test 17
Problem set 19

Chapter 01.03 Sources of error 21
What is round off error? 21
What problems can be created by round off errors? 21
What is truncation error? 22
Can you give me other examples of truncation error? 23
Multiple-choice test 27
Problem set 29

Chapter 01.04 Binary representation of numbers 33
Multiple-choice test 40
Problem set 42

Chapter 01.05 Floating point representation 43
Multiple-choice test 51
Problem set 53

Chapter 01.06 Propagation of errors 54
Multiple-choice test 57

Chapter 01.07 Taylor theorem revisited 59
Multiple-choice test 67

DIFFERENTIATION

Physical problems
Chapter 02.00A Physical problem - general engineering 69
Chapter 02.00B Physical problem - chemical engineering 71
Chapter 02.00D Physical problem - computer engineering 73
Chapter 02.00E Physical problem - electrical engineering 77
Chapter 02.00F Physical problem - industrial engineering 81
Chapter 02.00G Physical problem - mechanical engineering 85

Chapter 02.01 Primer on differential calculus (View it on the web)
Multiple-choice test 89
Problem set 91

Chapter 02.02 Differentiation of continuous functions 93
Forward difference approximation of the first derivative 93
Backward difference approximation of the first derivative 96
Forward difference approximation from the Taylor series 97
Finite difference approximation of higher derivatives 100
Multiple-choice test 105
Problem set 107

Chapter 02.03 Differentiation of discrete functions 109
Forward difference approximation of the first derivative 109
Direct fit polynomials 111
Lagrange polynomial 113
Multiple-choice test 115
Problem set 118

NONLINEAR EQUATIONS

Physical problems
Chapter 03.00A Physical problem - general engineering 120
Chapter 03.00B Physical problem - chemical engineering 124
Chapter 03.00C Physical problem - civil engineering 127
Chapter 0.3.00D Physical problem - computer engineering 133
Chapter 03.00E Physical problem - electrical engineering 136
Chapter 03.00F Physical problem – industrial engineering 139
Chapter 03.00G Physical problem - mechanical engineering 145

Chapter 03.01 Solution of quadratic equations 149
Multiple-choice test 152
Problem set 154

Chapter 03.03 Bisection method of solving a nonlinear equation 156
Bisection method 156
Algorithm for the bisection method 159
Advantages of bisection method 162
Drawbacks of bisection method 162
Multiple-choice test 165
Problem set 167

Chapter 03.04 Newton-Raphson method of solving a nonlinear equation 169
Introduction 169
Derivation 169
Algorithm 170
Drawbacks of the Newton-Raphson method 173
What is an inflection point? 174
Derivation of Newton Raphson method from Taylor series 177
Multiple-choice test 178
Problem set 180

Chapter 03.05 Secant method of solving nonlinear equations 182
What is the secant method and why would I want to use it instead of the
Newton-Raphson method? 182
Multiple-choice test 187
Problem set 189

SIMULTANEOUS LINEAR EQUATIONS

Physical problems
Chapter 04.00A Physical problem - general engineering 191
Chapter 04.00B Physical problem - chemical engineering 194
Chapter 04.00C Physical problem - civil engineering 196
Chapter 04.00D Physical problem - computer engineering 201
Chapter 04.00E Physical problem - electrical engineering 206
Chapter 04.00F Physical problem – industrial engineering 212
Chapter 04.00G Physical problem - mechanical engineering 215

Chapter 4.1 Introduction to matrix algebra 221
What is a matrix? 221
What are the special types of matrices? 222
Square matrix 223
Upper triangular matrix 223
Lower triangular matrix 223
Diagonal matrix 224
Identity matrix 224
Zero matrix 224
Tridiagonal matrices 225
When are two matrices considered to be equal? 225
How do you add two matrices? 226
How do you subtract two matrices? 227
How do I multiply two matrices? 228
What is a scalar product of a constant and a matrix? 230
what is a linear combination of matrices ? 231
What are some of the rules of binary matrix operations ? 231
Transpose of a matrix 234
Symmetric matrix 234
Matrix algebra is used for solving system of equations. Can you illustrate
this concept? 235
Can you divide two matrices? 237
Can I use the concept of the inverse of a matrix to find the solution of a set
of equations [A] [X] = [C]? 238
How do I find the inverse of a matrix? 238
If the inverse of a square matrix [A] exists, is it unique? 241
Multiple-choice test 242
Problem set 245

Chapter 04.06 Gaussian elimination 249
How are a set of equations solved numerically? 249
Forward elimination of unknowns 250
Back substitution 251
Are there any pitfalls of Naïve Gauss elimination method? 252
Round-off error 256
What are the techniques for improving Naïve Gauss elimination method?
258
How does Gaussian elimination with partial pivoting differ from Naïve
Gauss elimination? 258
Can we use Naïve Gauss elimination methods to find the determinant of a
square matrix? 261
What if I cannot find the determinant of the matrix using Naive Gauss
elimination method, for example, if I get division by zero problems during
Naïve Gauss elimination method? 262
Multiple-choice test 264
Problem set 267

Chapter 04.07 LU decomposition 269
I hear about LU decomposition used as a method to solve a set of
simultaneous linear equations? What is it and why do we need to learn
different methods of solving a set of simultaneous linear equations ? 269
How do I decompose a non-singular matrix [A], that is, how do I find
[A] = [L] [U]? 271
How do I find the inverse of a square matrix using LU decomposition?
275
Multiple-choice test 279
Problem set 283

Chapter 04.08 Gauss-Seidel method 285
Why do we need another method to solve a set of simultaneous linear
equations? 285
The above system of equations does not seem to converge. Why? 290
Multiple-choice test 295
Problem set 299

INTERPOLATION

Physical problems
Chapter 05.00A Physical problem - general engineering 300
Chapter 05.00B Physical problem - chemical engineering 302
Chapter 05.00C Physical problem - civil engineering 306
Chapter 05.00D Physical problem - computer engineering 309
Chapter 05.00E Physical problem - electrical engineering 312
Chapter 05.00F Physical problem - industrial engineering 315
Chapter 05.00G Physical problem - mechanical engineering 317

Chapter 05.01 Background of interpolation
Multiple-choice test 321

Chapter 05.02 Direct method of interpolation 323
What is interpolation? 323
Direct method 324
Multiple-choice test 331
Problem set 333

Chapter 05.03 Newton’s divided difference interpolation 335
What is interpolation? 335
Newton’s divided difference polynomial method 335
Linear interpolation 336
Quadratic interpolation 338
General form of Newton’s divided difference polynomial 341
Multiple-choice test 346
Problem set 348

Chapter 05.05 Spline method of interpolation 350
What is interpolation? 350
Linear spline interpolation 353
Quadratic splines 355
Multiple-choice test 360
Problem set 363

Chapter 05.06 Extrapolation is a bad idea 365
Chapter 05.07 Higher order interpolation is a bad idea 369
Chapter 05.08 Why do we need splines? 372
Chapter 05.10 Shortest path of a robot 375

REGRESSION

Physical problems
Chapter 06.00A Physical problem - general engineering 380
Chapter 06.00B Physical problem - chemical engineering 384
Chapter 06.00C Physical problem - civil engineering 387
Chapter 06.00D Physical problem - computer engineering 390
Chapter 06.00E Physical problem - electrical engineering 393
Chapter 06.00F Physical problem - industrial engineering 397
Chapter 06.00G Physical problem - mechanical engineering 399

Chapter 06.01 Statistics background of regression analysis 404
Review of statistical terminologies 404
Elementary statistics 404
A brief history of regression 408

Chapter 06.02 Introduction of regression analysis 410
What is regression analysis? 410
Comparison of regression and correlation 411
Uses of regression analysis 411
Abuses of regression analysis 411
Extrapolation 411
Least squares methods 414
Why minimize the sum of the square of the residuals? 414
Multiple-choice test 416
Problem set 418

Chapter 06.03 Linear regression 419
Why minimize the sum of the square of the residuals? 419
Multiple-choice test 432
Problem set 434

Chapter 06.04 Nonlinear models for regression 436
Nonlinear models using least squares 436
Exponential model 436
Growth model 440
Polynomial models 442
Linearization of data 446
Exponential model 446
Logarithmic functions 449
Power functions 452
Multiple-choice test 457
Problem set 459

Chapter 06.05 Adequacy of models for regression 463
Quality of fitted model 463
Caution in the use of r2 467
What else should I check for the adequacy of the model in example 1?
467
Adequacy of coefficient of regression 469
Problem set 470

INTEGRATION

Physical problems
Chapter 07.00A Physical problem - general engineering 473
Chapter 07.00B Physical problem - chemical engineering 476
Chapter 07.00C Physical problem - civil engineering 479
Chapter 07.00D Physical problem - computer engineering 485
Chapter 07.00E Physical problem - electrical engineering 496
Chapter 07.00F Physical problem – industrial engineering 501
Chapter 07.00G Physical problem - mechanical engineering 505

Chapter 07.01 Primer on integration (View it on the web)
Multiple-choice test 509
Problem set 511

Chapter 07.02 Trapezoidal rule of integration 514
What is integration? 514
What is the trapezoidal rule? 514
Derivation of the trapezoidal rule 515
Multiple-segment trapezoidal rule 521
Error in multiple-segment trapezoidal rule 527
Multiple-choice test 530
Problem set 532

Chapter 07.03 Simpson’s 1/3 rule of integration 535
What is integration? 535
Simpson’s 1/3 rule 535
Multiple-segment Simpson’s 1/3 rule 542
Error in multiple-segment Simpson’s 1/3 rule 545
Multiple-choice test 547
Problem set 549

Chapter 07.05 Gauss quadrature 551
What is integration? 551
Gauss quadrature rule 552
Derivation of two-point Gaussian quadrature rule 553
Higher point Gaussian quadrature formulas 555
Arguments and weighing factors for n-point Gauss quadrature rules 556
Multiple-choice test 565
Problem set 568

Chapter 07.06 Integrating discrete functions 570
What is integration? 570
Integrating discrete functions 571
Trapezoidal rule for discrete functions with unequal segments 575
Problem set 578

Chapter 07.07 Integrating improper functions 581
What is integration? 581
What is an improper integral? 582
Problem set 592

ORDINARY DIFFERENTIAL EQUATIONS

Physical problems
Chapter 08.00A Physical problem - general engineering 593
Chapter 08.00B Physical problem - chemical engineering 597
Chapter 08.00C Physical problem - civil engineering 599
Chapter 08.00D Physical problem - computer engineering 601
Chapter 08.00E Physical problem - electrical engineering 605
Chapter 08.00F Physical problem – industrial engineering 610
Chapter 08.00G Physical problem - mechanical engineering 616

Chapter 08.01 Primer for ordinary differential equations (View it on web)
Multiple-choice test 622
Problem set 624

Chapter 08.02 Euler’s method for ordinary differential equations 626
What is Euler’s method? 626
Derivation of Euler’s method 627
Multiple-choice test 635
Problem set 638

Chapter 08.03 Runge-Kutta 2nd order method 642
What is the Runge-Kutta 2nd order method? 643
Heun’s method 645
Midpoint method 645
Ralston’s method 646
How do these three methods compare with results obtained if we found
f'(x,y) directly? 649
How do we get the 2nd order Runge-Kutta method equations? 650
Multiple-choice test 653
Problem set 656

Chapter 08.04 Runge-Kutta 4th order method 660
What is the Runge-Kutta 4th order method? 660
How does one write a first order differential equation in the above form?
660
Multiple-choice test 667
Problem set 671

Chapter 08.05 On Solving higher order equations 675
Problem set 684

Chapter 08.07 Finite difference method 686
What is the finite difference method? 686
Multiple-choice test 694
Problem set 699