Convergence and Acceleration

We investigate the convergence of iterative processes defined by fixed-point iterations.

Note that the formula in the middle of page 58 of the text book should have the derivative of g, as g'(x) determines the rate of convergence (or divergence).

A general method to accelerate iterative processes is Aitken's acceleration. The book provides no explanation for the derivation of the formula, therefore, see below:

Aitken acceleration of convergence

We want to accelerate the convergence of a sequence of points x(k), where k=0,1,..,infinity, defined by x(k+1) = g(x(k)). We assume the sequence converges to the fixed point x(infinity) = g(x(infinity)).

Denote e(k) = x(k+1) - x(k) = g(x(k)) - x(k). In practice, e(k) is used to measure the error of the iteration process, if |e(k)| is small enough we terminate the sequence. By the assumption of convergence we have that e(k) goes to 0 as k goes to infinity, and e(infinity) = 0.

Consider now e(k) as a function of x: e(k) = e(x(k)). To accelerate the convergence, we wish to find the value for x for which e(x) = 0. Suppose we know e(x(k)) and e(x(k+1)). To find a root of e(x) = 0, we apply the idea of the secant method: construct a line through the points (x(k),e(x(k))) and (x(k+1),e(x(k+1))). An approximation for the root of e(x) = 0 is where the line intersects the x-axis.

The formula for the line through the points (x(k),e(x(k)) and (x(k+1),e(x(k+1))) is

                  e(x(k+1)) - e(x(k))
   y - e(x(k)) = --------------------- (x - x(k))
                    x(k+1)  -   x(k)

To compute the intersection with the x-axis, we set y = 0 in the equation above and solve for x:

                  x(k+1)  -   x(k)
   0 - e(x(k)) --------------------- + x(k) = x 
                e(x(k+1)) - e(x(k))

To simplify, we compute the following:

       e(x(k+1)) = g(x(k+1)) - x(k+1) = x(k+2) - x(k+1)
   - ( e(x(k))   = g(x(k))   - x(k)   = x(k+1) - x(k)   )
  --------------------------------------------------------
       e(x(k+1)) - e(x(k)) = x(k+2) - 2*x(k+1) + x(k)

Thus, we obtain

                 ( x(k+1) - x(k) )^2
   x = x(k) - -------------------------
               x(k+2) - 2*x(k+1) + x(k)