embarrassingly parallel computations

We defined ideal or naturally parallel computations as parallel computations which needed no interprocessor communication during the computations. In the extreme case, we just execute the same program on multiple processors, although that leads to problems of termination detection and collection of results. To solve these problems, we use a manager-worker model. Even as our computations are embarrassingly parallel, we must care about the computation/communication ration and the balance of the load distribution.

We considered two related but different application areas: image processing and computer graphics. The low-level pixel-by-pixel processing of images can happen on screen (where the manager writes to a display), or on file (where every processor transforms a piece of a picture, stored in a postscript file). Because of communication overhead, the on screen image processing is recommended only for shared memory multicomputers, while in a message passing environment we better distribute the files among the processors.

The lecture ended with a sketch of a dynamic load distribution algorithm to compute the Mandelbrot set. A good speedup will be realized as long as the average number of iterations per pixel exceeds the number of processors.