Convergence and Acceleration

A general method to accelerate iterative processes is Aitken's acceleration. The book provides no explanation for the derivation of the formula, therefore, see below:

Aitken acceleration of convergence

We want to accelerate the convergence of a sequence of points x(k), where k=0,1,..,infinity, defined by x(k+1) = g(x(k)). We assume the sequence converges to the fixed point x(infinity) = g(x(infinity)).

Denote e(k) = x(k+1) - x(k) = g(x(k)) - x(k). In practice, e(k) is used to measure the error of the iteration process, if |e(k)| is small enough we terminate the sequence. By the assumption of convergence we have that e(k) goes to 0 as k goes to infinity, and e(infinity) = 0.

Consider now e(k) as a function of x: e(k) = e(x(k)). To accelerate the convergence, we wish to find the value for x for which e(x) = 0. Suppose we know e(x(k)) and e(x(k+1)). To find a root of e(x) = 0, we apply the idea of the secant method: construct a line through the points (x(k),e(x(k))) and (x(k+1),e(x(k+1))). An approximation for the root of e(x) = 0 is where the line intersects the x-axis.

The formula for the line through the points (x(k),e(x(k)) and (x(k+1),e(x(k+1))) is

                  e(x(k+1)) - e(x(k))
   y - e(x(k)) = --------------------- (x - x(k))
                    x(k+1)  -   x(k)

To compute the intersection with the x-axis, we set y = 0 in the equation above and solve for x:

                  x(k+1)  -   x(k)
   0 - e(x(k)) --------------------- + x(k) = x 
                e(x(k+1)) - e(x(k))

To simplify, we compute the following:

       e(x(k+1)) = g(x(k+1)) - x(k+1) = x(k+2) - x(k+1)
   - ( e(x(k))   = g(x(k))   - x(k)   = x(k+1) - x(k)   )
  --------------------------------------------------------
       e(x(k+1)) - e(x(k)) = x(k+2) - 2*x(k+1) + x(k)

Thus, we obtain

                 ( x(k+1) - x(k) )^2
   x = x(k) - -------------------------
               x(k+2) - 2*x(k+1) + x(k)

a numerical example of Aitken acceleration

function a = aitken(x)
%
%  Applies Aitken acceleration to the sequence in x,
%  returning a sequence of n-2 new approximations,
%  where n is the length of the vector x.
%
n = length(x);
for k=1:n-2
  a(k) = x(k) - (x(k+1) - x(k))^2/(x(k+2) - 2*x(k+1) + x(k));
end;

function [x,a] = show_aitken(m,n,x0)
%
%  DESCRIPTION :
%    Applies first Newton's method to (x-1)^m,
%    using n steps starting at x0, followed by
%    Aitken acceleration method on the sequence. 
%
%  ON ENTRY :
%    m        multiplicity of 1 in (x-1)^m;
%    n        number of new approximations;
%    x0       starting value for the iteration.
%
%  ON RETURN :
%    x        sequence of approximation to 1,
%             as vector of range 1..n+1;
%    a        result of Aitken acceleration,
%             as vector of range 1..n-1.
%
x = zeros(1,n+1);
x(1) = x0;
for k=1:n 
  deltax = (x(k)-1)^m/(m*(x(k)-1)^(m-1));
  x(k+1) = x(k) - deltax;
end;
a = aitken(x);

diary output_aitken;
format long e
[x,a] = show_aitken(3,5,1.001);
fprintf('original sequence ');
x = x'
fprintf('accelerated sequence ');
a = a'
fprintf('original errors ');
e = x - ones(size(x))
fprintf('errors after acceleration ');
e = a - ones(size(a))
diary off;

original sequence x =

   1.00100000000000e+00
   1.00066666666667e+00
   1.00044444444444e+00
   1.00029629629630e+00
   1.00019753086420e+00
   1.00013168724280e+00

accelerated sequence a =

   1.00000000000000e+00
   9.99999999999999e-01
   1.00000000000000e+00
   1.00000000000000e+00

original errors e =

   9.99999999999890e-04
   6.66666666666593e-04
   4.44444444444470e-04
   2.96296296296239e-04
   1.97530864197493e-04
   1.31687242798328e-04

errors after acceleration e =

    6.66133814775094e-16
   -1.11022302462516e-15
    4.44089209850063e-16
    0.00000000000000e+00