MPI

Message Passing Interface (MPI) is a communication protocol for parallel programming. MPI is specifically used to allow applications to run in parallel across a number of separate computers connected by a network.

Basic Features of MPI

Message passing programs run the same (usually serial) code on multiple processors, which communicate with one another via library calls which fall into a few general categories:

• Calls to initialize, manage, and terminate communications.
• Calls to communicate between two individual processes (point-to-point).
• Calls to communicate among a group of processes (collective).
• Calls to create custom datatypes.

Implementations of MPI

There are several different implementations of MPI available on the U2 cluster.

• MPICH - A portable implementation of standard Message Passing Interface (MPI) created by the Argonne National Laboratory.
  • MPICH is built specifically for a network interface and compiler. The U2 cluster has two internal networks, Gigabit ethernet and Myrinet. The compilers are GNU, Intel, and PGI.
  • The versions of MPICH available on the U2 cluster are usually updated once every 4 to 6 months.
  • Show all the current versions of MPICH: module avail mpich
• OPENMPI - An open source implementation of MPI that is developed and maintained by a consortium made up of researchers from academia and industry.
  • This implementation is network aware, so it will automatically select the network interface.
  • OPENMPI is built specifically for a particular compiler.
  • Show all the current versions of OPENMPI: module avail openmpi
• HP-MPI - The HP message passing interface library. This is a propriety implementation of MPI created by HP.
  • This implementation is network aware, so it will automatically select the network interface.
  • Some pre-compiled applications use HP-MPI, however it is NOT available for code development.

Using MPI with GNU compilers

• Use the default MPICH module to set the paths for the GNU MPICH mpi compilers and commands.
• See Basic Compilation for examples of compiling C and Fortran codes without MPI.

  [cdcatbono samples]$ module load mpich

• Create a nodefile:
  • In these examples the nodefile contains entries for the front-end machine (bono).

    [cdcatbono samples]$ cat nodefile
     bono
     bono
     bono
     bono
     [cdcatbono samples]$

Compiling with MPI C

• Code: cpi from MPICH test suite.
  • view cpi code
• Compilation:

  [cdcatbono samples]$ mpicc -o cpi-gnu cpi.c

• Running the code:

  [cdcatbono samples]$ mpirun -machinefile nodefile -np 4 ./cpi-gnu
   Process 0 of 4 on bono.ccr.buffalo.edu
   pi is approximately 3.1415926544231239, Error is 0.0000000008333307
   wall clock time = 0.000000
   Process 1 of 4 on bono.ccr.buffalo.edu
   Process 2 of 4 on bono.ccr.buffalo.edu
   Process 3 of 4 on bono.ccr.buffalo.edu
   [cdcatbono samples]$

Compiling with MPI Fortran

• Code: fpi from MPICH test suite, modified to not run interactively.
  • view fpi code
• Compilation:

  [cdcatbono samples]$ mpif77 -o fpi-gnu fpi.f

• Running the code:

  [cdcatbono samples]$ mpirun -machinefile nodefile -np 4 ./fpi-gnu
    Process  0 of  4 is alive
     pi is approximately: 3.1415926744231277  Error is: 0.0000000208333346
    Process  2 of  4 is alive
    Process  3 of  4 is alive
    Process  1 of  4 is alive
   [cdcatbono samples]$

Tutorials on MPI

Parallel Computing

Cluster Resources

The U2 cluster provides extensive resources for parallel computing.

• Schematic Design of the U2 Cluster:
• Two networks:
  • Gigabit Ethernet
  • Myrinet
• 2112 processors
• 45TB of global disk storage

Running Interactively

Using the Login Machine

• If you wish to run an MPI code interactively on the front-end machine (only for a few processes and short duration, please - otherwise use the batch system), you can simply launch using mpirun:
  • mpirun -np 2 ./mine.x which in this case is for two processes.

Using a Compute Node

• Machines within the cluster are available for interactive use through the batch scheduler.

Running Programs on the Cluster

Tutorials on Parallel Computing