Using copper (cu.nsca.uiuc.edu):

To illustrate the use of copper, we took our cost_scatter.c program of lecture 6 to scatter a 2520-by-2520 matrix over p-1 workers, using p processors.

First, we can make interactive use of copper, to compile the program and run it with 4 processors (or less):

cu12:~/Notes/L6106% mpcc -o cost_scatter cost_scatter.c
cu12:~/Notes/L6107% poe cost_scatter -procs 4

Total wall time = 2.654583 seconds on 4 processors
Wall time on processor 1 = 2.655105 seconds.
Wall time on processor 2 = 2.653180 seconds.
Wall time on processor 3 = 2.654620 seconds.
cu12:~/Notes/L6108%
Notice the improved network connections, compared to our cluster computer:
[phcpack@idefix L6]$ mpirun -np 4 cost_scatter

Total wall time = 3.652681 seconds on 4 processors
Wall time on processor 1 = 1.517176 seconds.
Wall time on processor 2 = 2.602139 seconds.
Wall time on processor 3 = 3.685254 seconds.
[phcpack@idefix L6]$

To use more than 4 processors, we write a job script: job.ll and submit it

cu12:~/Notes/L6181% llsubmit job.ll
llsubmit: Processed command file through Submit Filter: "/u/loadl/etc/ll.submit.filter".
llsubmit: The job "cu11.ncsa.uiuc.edu.22479" has been submitted.
cu12:~/Notes/L6182% llq | grep "janv"
cu11.22479.0             janv        3/15 10:41 I  50  batch
cu12:~/Notes/L6183% llhist cu11.22479.0
--------------------------------------------------------
        IBM pSeries 690 Batch Job Summary
--------------------------------------------------------
  Job Id           :     22479
  Job Name         :     scatter_cost
  User             :     janv

--------------------------------------------------------
                STEP  0
--------------------------------------------------------
  Job Status       :     Idle
  Submitted        :     Wed Mar 15 10:41:00 CST 2006
  Project          :     lmm
  Class            :     batch

Limits:

  Wall Clock Limit :     00:10:00  [hh:mm:ss]
  Number of CPU's  :     5
  Memory           :     2.44 GB

cu12:~/Notes/L6184%
After completion, an email is sent and we can confirm the termination again via llhist:
cu12:~/Notes/L6186% !llhis
llhist cu11.22479.0
--------------------------------------------------------
        IBM pSeries 690 Batch Job Summary
--------------------------------------------------------
  Job Id           :     22479
  Job Name         :     scatter_cost
  User             :     janv

--------------------------------------------------------
                STEP  0
--------------------------------------------------------
  Job Status       :     Completed
  Submitted        :     Wed Mar 15 10:41:00 CST 2006
  Started          :     Wed Mar 15 11:11:54 CST 2006
  Finished         :     Wed Mar 15 11:12:18 CST 2006
  Host             :     cu04
  Project          :     lmm
  Class            :     batch

Usage:

  Cpu time         :     00:00:28  [hh:mm:ss]
  Run Time         :     00:00:24
  Peak Task Memory :     64.16 MB
  Service Units    :     0.01

Limits:

  Wall Clock Limit :     00:10:00  [hh:mm:ss]
  Number of CPU's  :     5
  Memory           :     2.44 GB

cu12:~/Notes/L6187%
The output file is scatter_cost.22479.out.

For more information, see the web pages at NSCA on copper.