data partitioning and divide-and-conquer strategies

In this lecture we introduced "Data Partitioning", perhaps better known as Domain Decomposition, as one of the techniques to introduce concurrency and create parallel programs. A complementary strategy to create parallel programs is "functional decomposition": instead of distributing data, we focus on distributing the functions.

We used the parallel summation as a running example. A first algorithm for this program was given in lecture 5. We analyzed its computation cost -- which is very promising -- and communication cost, which led to a disappointing computation/communication ratio.

Applying the principle of divide and conquer, we described a second algorithm for the summation problem, illustrating the complementary strategy of functional decomposition, distributing the "+" operations in a recursive fashion. To reduce the communication time, we recalled the fanning out mechanism to scatter (in lecture 6 we saw the fan out broadcast). The lecture ended by showing how the communications between the nodes could be used to reduce the communication time of the gathering of the results from p to log(p) steps, for p processors.