Barriers to synchronize computations
Pipelined computations are partially synchronized computations,
because one processor has to wait for the previous one to start
its calculations. To avoid deadlock in this synchronization,
MPI provides the MPI_Sendrecv() instruction.
Synchronizing all processors is achieved by the MPI_Barrier()
command. We considered three possible implementations:
a linear barrier requires 2*p stages, a tree barrier operates
in 2*log(p) stages, and we ended by outlining the algorithm
for a butterfly barrier which requires only log(p) stages to
synchronize p processors.
Bibliography
- Marc Snir, Steve Otto, Steven Huss-Lederman, David Walker,
and Jack Dongarra:
"MPI - The Complete Reference Volume 1, The MPI Core"
second edition, Massachusetts Institute of Technology, 1998.
Online documentation:
The Message Passing Interface (MPI) standard