Index

Symbols

2-by-2 switches

A

Ada array and ring topology

arrival phase atomic variable attainable performance

B

bandwidth bank conflict barrier Bernstein's conditions, [1] bidirectional data transfer bisection bandwidth

bisection width blocking communication, [1] blocking receive blocking send Brent's theorem broadcast butterfly barrier

C

circuit switching cluster coalesced access to global memory coarse granularity collective communication, [1], [2], [3] communication granularity communication latency communication-to-computation ratio

compiler directive Cooperative Thread Array critical path critical section, [1], [2] cross over crossbar switch CTA CUDA

D

data parallel computation data parallelism data staging deadlock, [1] departure phase

diameter of network distributed memory multicomputer divide and conquer domain decomposition dynamic network topology dynamic work load balancing

E

efficiency, [1] ethernet

exclusive scan execution time

F

fan out broadcast fine granularity

flops, [1] fragment, [1] functional decomposition, [1]

G

gather

grain granularity

H

heap High Performance Computing High Throughput Computing

HPC HTC hypercube network

I

immediate receive

immediate send isoefficiency function

J

job queue

job scheduling

K

kernel

L

latency

lifetime livelock

M

manager/worker model master construct matrix and torus topology MCA memory access-to-computation ratio memory allocation memory coalescing message latency MIMD MISD Monte Carlo MPI_Bcast MPI_Comm_rank MPI_Comm_size MPI_COMM_WORLD MPI_Finalize MPI_GATHER

MPI_Init MPI_Iprobe MPI_IRECV MPI_ISEND MPI_RECV MPI_REDUCE MPI_SCATTER MPI_SEND MPI_Sendrecv MPI_SUM MPI_TEST MPI_WAIT MPI_Wtime Multiple Instruction Multiple Data stream Multiple Instruction Single Data stream multiprocessing module multistage network mutex

N

network latency

nonblocking communication, [1] numerical integration

O

Omega interconnection network omp_get_thread_num()

omp_set_num_threads() overhead overhead time

P

packet switching parallel construct parallel region, [1] pass through performance bottleneck performance cliff pipeline pipeline cycle pipeline latency

pleasingly parallel point-to-point communication, [1] POSIX PRAM prefix sum prefix sum algorithm private clause process, [1] pseudorandom numbers Python

Q

quality up, [1]

R

race condition reduction algorithm release phase

ridge point roofline model routing algorithm

S

scalability scaled speedup scatter scope of a variable shared memory multicomputer SIMD, [1], [2] Simpson rule SIMT single construct Single Instruction Multiple Data Single Instruction Multiple Data stream single instruction multiple thread Single Instruction Single Data stream

Single Program Multiple Data stream Single Program, Multiple Data SISD space-time diagram speedup, [1] SPMD, [1] square partition stack static work load assignment strip partition supercomputer superlinear speedup switches

T

task tasking thread, [1] thread coarsening thread divergence, [1] thread safe

throughput trapping phase tree barrier tree network type 1 pipeline, [1] type 2 pipeline type 3 pipeline

W

warp, [1] Warp Matrix Multipy and Accumulate

WMMA work crew