MTH104 Handouts

Preface

These class notes complement the class textbook Numerical Linear Algebra by Trefethen and Bau. They do not suffice in themselves as study material; it is only in combination with designated portions of the textbook that you have a complete picture of the material covered. The notes also show the order in which the subjects are taught in class.

Chapter 1

Preliminaries

This chapter contains an expanded discussion of selected concepts introduced in Lecture 1 of the textbook. A vector x of dimension n is an n-tuple of numbers xj with 1 ≤ j ≤ n. The xj are called the components of x. When the xj are real, one can think of the vector x as either a point in the abstract space Rn, or equivalently the arrow pointing from the origin to that point, and the geometrical intuition you have about the case n = 3 is perfectly adequate. Note that sometimes people (e.g. physicists) prefer to be more careful about the definition of vector, and say that it should be defined independently of a basis, but in this course we will simply identify a vector with its components in a basis. (The notion of basis will be explained later.) Am m-by-n matrix A is an array of numbers Aij , where the subscript i indexes the rows (1 ≤ i ≤ m) and j indexes the columns (1 ≤ j ≤ n). The result of the matrix-vector product A times x is the vector b whose components bi are given by the sum

bi = n X j=1 Aij xj .

In short, b = Ax. In matrix notation, this also reads b1

bm A11 A1n

Am1 Amn x1

xn Notice that the components of x and b have been put in a column: when matrices are around we will almost always view a vector as being a column vector, or in other words as a n-by-1 matrix. We need to transpose it in order to obtain a row vector: xT is defined as the 1-by-n matrix

(x1, . . . , xn).

You should become familiar with switching between the index notations and the matrix notations. For instance, notice the role of the free index i and the dummy (or summation) index j. Dummy indices can be renamed. Any free index in the left-hand side of an expression also needs to appear as a free index in the right-hand side. Many problems in science engineering can be cast as linear systems of equations, or perhaps simplified in order to be cast as linear systems. In turn, one can view such systems in matrix notation as Ax = b and the problem is to find x given the matrix A and the vector b. The components xj of a vector, or the entries Aij of a matrix are usually real-valued, but things are not much different when they are complex-valued, so the Trefethen-Bau textbook takes the general viewpoint of considering complex numbers. Even if the entries of a matrix are real-valued, complex numbers often come back through the back door when we calculate its eigenvalues (discussed later), so it’s a good idea to be familiar with complex numbers. We reserve the word “scalar” for either a real number or a complex number. If you are confident about complex numbers, skip the next section.

1.1

Square roots of negative numbers are forbidden in the usual calculus of real numbers, so they have long puzzled and posed a problem to mathematicians. It was only in the mid 1800s that it was recognized that such things could no longer be avoided, for instance in computing the roots of polynomials. The consensus that emerged was to denote by i the imaginary unit, work with the abstract law i2 = −1,

and define the two square roots of −1 as ±i. Like for negative numbers, familiarity would do the rest to make this concept concrete. The choice of letter i is commonplace in math and physics, and I apologize in advance to the engineers who use the letter j instead. Complex numbers are then defined as linear combinations of real and imaginary numbers, like a + ib where a and b are real. The number a is