MCS 572 Spring 2006 Notes

The links below provide a summary of what was covered in class.

Part I Basic Techniques

1. Parallel Computers

  1. introduction -- speedup and efficiency
  2. scalability -- message passing computing
  3. types of parallel computers, clusters
2. Message-Passing Computing
  1. basics of MPI
  2. using MPI to write parallel programs
  3. evaluating parallel programs
  4. debugging and visualization
3. Embarrassingly Parallel Computations
  1. embarrassingly parallel computations
  2. static and dynamic task assignment
  3. Monte Carlo methods
4. Partitioning and Divide-and-Conquer Strategies
  1. data partitioning and divide-and-conquer strategies
  2. parallel sorting methods: bucket sort
  3. numerical integration on parallel computers
  4. the N-body problem
5. Pipelined Computations
  1. pipelining to create parallel algorithms
  2. adding and sorting using pipelines
  3. solving triangular linear systems
6. Synchronous Computations
  1. barriers to synchronize computations
  2. data parallel computations and synchronous iteration
  3. parallel iterative methods for linear systems
  4. solving the heat distribution problem
7. Load Balancing and Termination Detection
  1. static and dynamic load balancing
  2. dynamic load balancing
  3. the parallel knapsack problem
  4. polynomial root finding
8. Programming with Shared Memory
  1. shared memory multiprocessor machines
  2. concurrent programming using threads
  3. parallel programming languages
  4. introduction to OpenMP - threads
  5. parallel numerical integration with OpenMP
  6. performance issues
9. Distributed Shared Memory Systems and Programming
  1. Hybrid MPI/OpenMP programming
  2. Distributed Shared Memory Systems and Programming
  1. guest lecture by Anton Leykin

Part II: Algorithms and Applications

  1. parallel sorting algorithms
  2. parallel numerical linear algebra
  3. the conjugate gradient method
  4. parallel FFT
Lectures 39 to 44 were reserved for project presentations.