Transcription

Introduction to Numerical AnalysisS. Baskar

2General InstructionsCourse Number : SI 507Course Title: Numerical AnalysisCourse Syllabus1. Mathematical Preliminaries: Continuity of a Function and Intermediate Value Theorem; Mean ValueTheorem for Differentiation and Integration; Taylor’s Theorem (1 and 2 dimensions).2. Error Analysis: Floating-Point Approximation of a Number; Loss of Significance and Error Propagation;Stability in Numerical Computation.3. Linear Systems: Gaussian Elimination; Pivoting Strategy; LU factorization; Residual Corrector Method;Solution by Iteration; Conjugate Gradient Method; Ill-Conditioned Matrices, Matrix Norms; Eigenvalue problem - Power Method; Gershgorin’s Theorem.4. Nonlinear Equations: Bisection Method; Fixed-Point Iteration Method; Secant Method; Newton Method;Rate of Convergences; Solution of a System of Nonlinear Equations; Unconstrained Optimization.5. Interpolation by Polynomials: Lagrange Interpolation; Newton Interpolation and Divided Differences;Hermite Interpolation; Error of the Interpolating Polynomials; Piecewise Linear and Cubic Spline Interpolation; Trigonometric Interpolation; Data Fitting and Least-Squares Approximation Problem.6. Differentiation and Integration: Difference formulae; Some Basic Rules of Integration; Adaptive Quadratures; Gaussian Rules; Composite Rules; Error Formulae.7. Differential Equations: Euler Method; Runge-Kutta Methods; Multi-Step Formulae; Predictor-CorrectorMethods; Stability and Convergence; Two Point Boundary Value Problems.Texts/References1. K. E. Atkinson, An Introduction to Numerical Analysis (2nd edition), Wiley-India, 1989.2. S. D. Conte and Carl de Boor, Elementary Numerical Analysis - An Algorithmic Approach (3rd edition),McGraw-Hill, 1981.General Rules1. Attendance in lectures as well as tutorials is compulsory. Students not fulfilling the 80% attendance requirement may be awarded the XX grade.2. Attendance will be recorded through an attendance sheet that will be circulated in the class. Each studentis expected to sign against his/her name only. Students who are found indulging in proxy attendance or anyform of cheating will be severely punished.Evaluation Plan1.2.3.4.There will be two quizzes (dates will be announced later), each of weightage 10% and one hour duration.The Mid-Semester Examination scheduled during 11-18 September 2010 will be of 30% weightage.The End-Semester Examination scheduled during 16-28 November will be of 40% weightage.Lab assignments will be given through out the semester and the students are expected to complete theassignment and produce all the outputs asked at the end of the semester. A oral viva will be conducted toeach student. The weightage will be of 10%.5. To pass the course (DD), one needs to score minimum of 40% of the maximum mark scored in the class. Forinstance, if the maximum mark scored is 80% at the end of the semester, then the passing mark will be 32%.Higher grades will be based on the over all performance of the class.Web Page: Course related materials will be uploaded inhttp://www.math.iitb.ac.in/ baskar/baskar t.htm

3PrefaceIn addition to the references provided above, class notes will be distributed in the class as a typedmaterial. These notes are meant only for SI 507 in Autumn 2010 as a supplementary material and cannotbe considered as a text book. Students are requested to refer the text books listed under course syllabusfor more details. These notes may have errors of all kind and the author request the readers to take careof such error while going through the material. The author will be grateful to those who brings to hisnotice any kind of error.

ContentsIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91.1 Continuity of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91.2 Differentiation of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.3 Integration of a Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.4 Taylor’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142Error Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.1 Floating-Point Form of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.2 Chopping and Rounding a Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3 Different Type of Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.4 Loss of Significant Digits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.5 Propagation of Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.1 Gaussian Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 LU Factorization Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.3 Error in Solving Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.4 Matrix Norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.5 Iterative Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.6 Eigenvalue Problem: The Power Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393.7 Gerschgorin’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.1 Fixed-Point Iteration Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.2 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.3 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 554.4 Newton-Raphson Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574.5 System of Nonlinear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.6 Unconstrained Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

6ContentsExercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 645Interpolation by Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.1 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 675.2 Newton Interpolation and Divide Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 695.3 Error in Polynomial Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.4 Piecewise Linear and Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Exercise 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 776Numerical Differentiation and Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.1 Numerical Differentiation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 796.2 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Exercise 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 917Numerical Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 937.1 Review on Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 937.2 Discretization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.3 Euler’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 957.4 Runge-Kutta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 997.5 An Implicit Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1007.6 Multistep Methods: Predictor and Corrector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101Exercise 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

IntroductionNumerical analysis is a branch of Mathematics that deals with devising efficient methods for obtainingnumerical solutions to difficult Mathematical problems.Most of the Mathematical problems that arise in science and engineering are very hard and sometimeimpossible to solve exactly. Thus, an approximation to a difficult Mathematical problem is very important to make it more easy to solve. Due to the immense development in the computational technology,numerical approximation has become more popular and a modern tool for scientists and engineers. As aresult many scientific softwares are developed (for instance, Matlab, Mathematica, Maple etc.) to handlemore difficult problems in an efficient and easy way. These softwares contain functions that uses standardnumerical methods, where a user can pass the required parameters and get the results just by a singlecommand without knowing the details of the numerical method. Thus, one may ask why we need tounderstand numerical methods when such softwares are at our hands?In fact, there is no need of a deeper knowledge of numerical methods and their analysis in most of thecases in order to use some standard softwares as an end user. However, there are at least three reasonsto gain a basic understanding of the theoretical background of numerical methods.1. Learning different numerical methods and their analysis will make a person more familiar with thetechnique of developing new numerical methods. This is important when the available methods arenot enough or not efficient for a specific problem to be solved.2. In many circumstances, one has more methods for a given problem. Hence, choosing an appropriatemethod is important for producing an accurate result in lesser time.3. With a sound background, one can use methods properly (especially when a method has its ownlimitations and/or disadvantages in some specific cases) and, most importantly, one can understandwhat is going wrong when results are not as expected.Numerical analysis include three parts. The first part of the subject is about the development of amethod to a problem. The second part deals with the analysis of the method, which includes the erroranalysis and the efficiency analysis. Error analysis gives us the understanding of how accurate the resultwill be if we use the method and the efficiency analysis tells us how fast we can compute the result.The third part of the subject is the development of an efficient algorithm to implement the method asa computer code. A complete knowledge of the subject includes familiarity in all these three parts. Thiscourse is designed to meet this goal.A first course in Calculus is indispensable for numerical analysis. The first chapter of these lecturenotes quickly reviews all the e