Pipelining and Synchronized Computations

• Pipelined Computations
  • Functional Decomposition
  • Loop Unrolling
  • Pipeline Implementations
  • Using MPI to implement a pipeline
  • Exercises
• Pipelined Sorting, Sieving, Substitution
  • Sorting Numbers
  • Prime Number Generation
  • Solving Triangular Systems
  • Bibliography
  • Exercises
• Solving Triangular Systems
  • Ill Conditioned Matrices and Quad Doubles
  • On a Parallel Shared Memory Computer with OpenMP
  • Accelerated Back Substitution
  • Bibliography
  • Exercises
• Barriers for Synchronizations
  • Synchronizing Computations
  • The Prefix Sum Algorithm
  • Brent’s Theorem
  • Bibliography
  • Exercises
• Parallel Iterative Methods for Linear Systems
  • Jacobi Iterations
  • A Parallel Implementation with MPI
  • Gather-to-All with MPI_Allgather
  • Strip Partitioning and Reduce Barriers in Julia
  • Exercises
• Domain Decomposition Methods
  • Gauss-Seidel Relaxation
  • Parallel Gauss-Seidel with OpenMP
  • Solving the Heat Equation
  • Solving the Heat Equation with PETSc
  • Bibliography
  • Exercises
• Memory Coalescing Techniques
  • Accessing Global and Shared Memory
  • Memory Coalescing Techniques
  • Avoiding Bank Conflicts
  • Exercises
• Introduction to Tensor Cores
  • High Throughput Computing
  • Volta, Ampere, Hopper Architectures
  • Simple Matrix Multiplication
  • Bibliography
  • Exercises
• Performance Considerations
  • Dynamic Partitioning of Resources
  • The Compute Visual Profiler
  • Data Prefetching and Instruction Mix
  • Thread Coarsening
  • Exercises