MPI
Message Passing Interface (MPI) is a communication protocol for parallel programming. MPI is specifically used to allow applications to run in parallel across a number of separate computers connected by a network.
Basic Features of MPI
Message passing programs run the same (usually serial) code on multiple processors, which communicate with one another via library calls which fall into a few general categories:
- Calls to initialize, manage, and terminate communications.
- Calls to communicate between two individual processes (point-to-point).
- Calls to communicate among a group of processes (collective).
- Calls to create custom datatypes.
Implementations of MPI
There are several different implementations of MPI available on the U2 cluster.
- MPICH - A portable implementation of standard Message Passing Interface (MPI) created by the Argonne National Laboratory.
- MPICH is built specifically for a network interface and compiler. The U2 cluster has two internal networks, Gigabit ethernet and Myrinet. The compilers are GNU, Intel, and PGI.
- The versions of MPICH available on the U2 cluster are usually updated once every 4 to 6 months.
- Show all the current versions of MPICH: module avail mpich
- OPENMPI - An open source implementation of MPI that is developed and maintained by a consortium made up of researchers from academia and industry.
- This implementation is network aware, so it will automatically select the network interface.
- OPENMPI is built specifically for a particular compiler.
- Show all the current versions of OPENMPI: module avail openmpi
- HP-MPI - The HP message passing interface library. This is a propriety implementation of MPI created by HP.
- This implementation is network aware, so it will automatically select the network interface.
- Some pre-compiled applications use HP-MPI, however it is NOT available for code development.
Using MPI with GNU compilers
- Use the default MPICH module to set the paths for the GNU MPICH mpi compilers and commands.
- See Basic Compilation for examples of compiling C and Fortran codes without MPI.
[cdcatbono samples]$ module load mpich
- Create a nodefile:
- In these examples the nodefile contains entries for the front-end machine (bono).
[cdcatbono samples]$ cat nodefile bono bono bono bono [cdcatbono samples]$
- In these examples the nodefile contains entries for the front-end machine (bono).
Compiling with MPI C
- Code: cpi from MPICH test suite.
- Compilation:
[cdcatbono samples]$ mpicc -o cpi-gnu cpi.c
- Running the code:
[cdcatbono samples]$ mpirun -machinefile nodefile -np 4 ./cpi-gnu Process 0 of 4 on bono.ccr.buffalo.edu pi is approximately 3.1415926544231239, Error is 0.0000000008333307 wall clock time = 0.000000 Process 1 of 4 on bono.ccr.buffalo.edu Process 2 of 4 on bono.ccr.buffalo.edu Process 3 of 4 on bono.ccr.buffalo.edu [cdcatbono samples]$
Compiling with MPI Fortran
- Code: fpi from MPICH test suite, modified to not run interactively.
- Compilation:
[cdcatbono samples]$ mpif77 -o fpi-gnu fpi.f
- Running the code:
[cdcatbono samples]$ mpirun -machinefile nodefile -np 4 ./fpi-gnu Process 0 of 4 is alive pi is approximately: 3.1415926744231277 Error is: 0.0000000208333346 Process 2 of 4 is alive Process 3 of 4 is alive Process 1 of 4 is alive [cdcatbono samples]$
Tutorials on MPI
Parallel Computing
Cluster Resources
The U2 cluster provides extensive resources for parallel computing.
- Schematic Design of the U2 Cluster:

- Two networks:
- Gigabit Ethernet
- Myrinet
- 2112 processors
- 45TB of global disk storage
Running Interactively
Using the Login Machine
- If you wish to run an MPI code interactively on the front-end machine (only for a few processes and short duration, please - otherwise use the batch system), you can simply launch using mpirun:
- mpirun -np 2 ./mine.x which in this case is for two processes.
Using a Compute Node
- Machines within the cluster are available for interactive use through the batch scheduler.
