CCR x86_64 Linux Cluster Hardware Page

Schematic Design of the Cluster

Front-End (interactive use, debugging, cpu time limit of 30m)

• Hostname = u2.ccr.buffalo.edu (front end for logins, job submission)
• Vendor = Dell
• Architecture = 2850 Server
• Number of Processors = 2
• Processor Description:
  • 2 3.2GHz Intel Xeon "Irwindale" Processors
  • Main memory size: 4096 Mbytes
  • Instruction cache size: 16 Kbytes
  • Data cache size: 16 Kbytes
  • Secondary unified instruction/data cache size: 2048 Kbytes
• Operating System: Linux (RedHat Enterprise Linux 4, 2.6 Kernel)

Production Machine (accessible only through the batch system )

• Primary SC1425 2-Way Compute Nodes
• Vendor = Dell
• Architecture = PowerEdge SC1425 Server
• Number of Processors = 2
• Processor Description:
  • 2x3.2GHz (800 nodes) or 2x3.0GHz (256 nodes) Intel Xeon "Irwindale" Processors
  • Main memory size: 2048 Mbytes (960 nodes), 4096MBytes (64 nodes), 8192 (32 nodes)
  • Instruction cache size: 16 Kbytes
  • Data cache size: 16 Kbytes
  • Secondary unified instruction/data cache size: 2048 Kbytes
• Myrinet 2G M3F-PCIX-D Network Card (2Gb/s) (768 nodes, including all large memory nodes, specified using the GM property in PBS)
• Available node properties in PBS/Torque batch system:
  • GM, (e.g. "-lnodes=2:GM:ppn=2") for Myrinet connected nodes
  • MEM4GB, (e.g. "-lnodes=2:MEM4GB:ppn=2") for nodes with 4GB of RAM
  • MEM8GB, (e.g. "-lnodes=2:MEM8GB:ppn=2") for nodes with 8GB of RAM
  • Note that the 4GB and 8GB nodes all automatically have the GM property, so it is not necessary to combine them.
• Operating System: Linux (RedHat Enterprise Linux 4, 2.6 Kernel)
• Disk: 1 80GB 7200 rpm SATA hard drive (local operating system, /scratch)

Disk Layout

• /home, User $HOME directories, NFS mounted from the CCR SAN to the compute nodes and front-ends.
• /scratch, Primary high-performance scratch space, located on each compute node (about 65GB is available on each /scratch).
  • Accessible through PBS, which will automatically create a unique scratch directory in /scratch for each new batch job (the directory name will be
  • /scratch/$PBS_JOBID). Inside your PBS script, you can easily access the appropriate scratch directory using the $PBSTMPDIR environmental variable.
  • All scratch space will be scrubbed automatically at the end of each batch job. Files that need to be stored long term should be kept elsewhere.
• /san/scratch, globally accessible NFS mounted scratch space for staging/preserving data between runs (~2TB in size).
• /ibrix/scratch, globally accessible NFS mounted scratch space for staging/preserving data between runs (~25TB in size).