High Performance Computing
High End Visualization
About CCR Contact Us Events Facilities Faculty Project Highlights History Job Opportunities News Partners Staff
Bioinformatics Consulting Services Grid Computing Visualization
Accounts Application Software Consulting Getting Started Hardware Resources Frequently Asked Questions Machine Status MyStats Overview Request Help Storage Resources Training/Courses Web Portals
Access Grid Training Outreach
Events Logos Media News Photo Album Videos
Contact Information Directions
  Dashboard > CCR Web > Getting Started > Basic Analysis and Monitoring
Log In   View a printable version of the current page.
Basic Analysis and Monitoring

Command Line Tools

top

  • The top command actively monitors processes on a machine.
    • Displays CPU and memory utilization.
    • Processes consuming CPU and memory resources.
    • PID is the process ID of a task.
    • Example of running top on the U2 front-end machine.
top - 11:43:01 up 8 days,  3:29, 66 users,  load average: 0.44, 0.38, 0.87
Tasks: 407 total,   1 running, 401 sleeping,   3 stopped,   2 zombie
Cpu(s):  5.0% us,  1.1% sy,  0.0% ni, 92.9% id,  0.9% wa,  0.0% hi,  0.2% si
Mem:   4045144k total,  2302052k used,  1743092k free,     5996k buffers
Swap:  8193140k total,   861336k used,  7331804k free,   488456k cached

  PID USER      PR  NI %CPU    TIME+  %MEM  VIRT  RES  SHR S COMMAND
 9514 root      16   0    9 206:47.32  0.7 35496  25m 1384 S pbs_server
19770 user1     15   0    4   3:13.07  2.4  335m  96m  20m S firefox-bin
24126 user2     16   0    1  62:49.60  0.0 57600 1572 1000 S sshd
16524 user3     16   0    1   0:00.04  0.0  5560 1344  816 R top
 2138 user4     16   0    0   0:15.33  0.1 57576 2328 1280 S sshd
 4495 root      16   0    0 223:20.70  0.4 66384  15m  816 S fm_server
10493 user4     16   0    0   3:30.97  0.0 58008 1928 1200 S sshd
15988 user5     15   0    0   0:00.04  0.0 53244 1516 1104 S bash
    1 root      15   0    0   0:02.87  0.0  4756  464  432 S init
    2 root      RT   0    0   0:03.22  0.0     0    0    0 S migration/0
    3 root      34  19    0   0:02.78  0.0     0    0    0 S ksoftirqd/0
    4 root      RT   0    0   0:02.61  0.0     0    0    0 S migration/1
    5 root      34  19    0   0:00.47  0.0     0    0    0 S ksoftirqd/1
    6 root      RT   0    0   0:03.18  0.0     0    0    0 S migration/2
    7 root      34  19    0   0:00.82  0.0     0    0    0 S ksoftirqd/2
    8 root      RT   0    0   0:02.87  0.0     0    0    0 S migration/3
    9 root      34  19    0   0:00.58  0.0     0    0    0 S ksoftirqd/3

Using the Unix Pipe

  • A pipe is used to allow the output of one command to be sent as input to another comand.
    • The pipe is represented with | (shift backslash key)

grep

  • The grep command searches for a pattern.
    • Example that counts occurrences of a text in a file:
[bono:~/pbs-examples]$ grep c06 nodelist
c06n19
c06n16
c06n02
[bono:~/pbs-examples]$ grep c06 nodelist | wc -l
3
[bono:~/pbs-examples]$

file

  • The file command determines the file type.
[bono:~/pbs-examples]$ file cpi.c
cpi.c: ASCII C program text
[bono:~/pbs-examples]$ file cpi-intel-9.1
cpi-intel-9.1: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for GNU/Linux 2.4.0, dynamically linked (uses shared libs), not stripped
[bono:~/pbs-examples]$

ldd

  • The ldd command will show the shared library dependencies.
[bono:~/pbs-examples]$ ldd cpi-intel-9.1
        libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x000000317dd00000)
        librt.so.1 => /lib64/tls/librt.so.1 (0x000000317eb00000)
        libm.so.6 => /lib64/tls/libm.so.6 (0x000000317d700000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x000000317e900000)
        libc.so.6 => /lib64/tls/libc.so.6 (0x000000317d400000)
        libdl.so.2 => /lib64/libdl.so.2 (0x000000317d900000)
        /lib64/ld-linux-x86-64.so.2 (0x000000317d200000)
[bono:~/pbs-examples]$

nm

  • The nm command lists symbols from an object file
    • piped through more
[bono:~/pbs-examples]$ nm cpi-intel-9.1  | more
0000000000440c70 r _2il0floatpacket.1
0000000000440eb0 r _2il0floatpacket.1
0000000000444470 r _2il0floatpacket.1
00000000004445d8 r _2il0floatpacket.1
0000000000447ce8 r _2il0floatpacket.1
0000000000448498 r _2il0floatpacket.1
000000000044a3f0 r _2il0floatpacket.1
0000000000440c78 r _2il0floatpacket.2
0000000000444480 r _2il0floatpacket.2
00000000004445e0 r _2il0floatpacket.2
000000000044a400 r _2il0floatpacket.2
0000000000440c80 r _2il0floatpacket.3
0000000000444490 r _2il0floatpacket.3
00000000004445e8 r _2il0floatpacket.3
000000000044a410 r _2il0floatpacket.3
0000000000440c88 r _2il0floatpacket.4
000000000044a420 r _2il0floatpacket.4
0000000000440c60 r _2il0floatpacket.5
0000000000440c90 r _2il0floatpacket.6
0000000000440c98 r _2il0floatpacket.7
000000000043f7ca t A0Q0
000000000043f7c6 t A0Q1
--More--
  • nm used to display references to MPI
[bono:~/pbs-examples]$ nm cpi-intel-9.1  | grep -i mpi | more
0000000000409d88 T MPI_Attr_get
000000000040a40c T MPI_Attr_put
000000000040b724 T MPI_Barrier
000000000040b800 T MPI_Bcast
00000000004290c8 T MPI_Cancel
0000000000564a98 B MPICHX_QOS_BANDWIDTH
0000000000564a9c B MPICHX_QOS_PARAMETERS
000000000040ab78 T MPI_Comm_free
000000000040adc0 T MPI_Comm_rank
000000000040ae64 T MPI_Comm_set_name
000000000040b034 T MPI_Comm_size
0000000000422a8e T MPID_Abort
0000000000426b02 T MPID_ArgSqueeze
0000000000433a2a T MPID_BsendContig
00000000004237c8 T MPID_BSwap_N_copy
00000000004238ce T MPID_BSwap_N_inplace
0000000000571088 B MPID_byte_order
0000000000425416 T MPID_ByteSwapInt
000000000042495c T MPID_Cancel_print_pkt
0000000000425012 T MPID_CH_Abort
0000000000438174 T MPID_CH_Check_incoming
00000000004252e4 T MPID_CH_Comm_msgrep
--More--

ps

  • The ps command shows a snapshot of the current processes on the system
    • The -ef flags will show all processes in full display.
    • The use of a pipe and grep can be used to refine the output.
[bono:~/pbs-examples]$ ps -ef | grep user1
root     10651  4272  0 16:09 ?        00:00:00 sshd: user1 [priv]
user1    10723 10651  0 16:09 ?        00:00:00 sshd: user1@pts/34
user1    10785 10723  0 16:09 pts/34   00:00:00 -bash
user1    11334     1  0 16:09 pts/34   00:00:00 /bin/sh /usr/bin/clustervis -h c23n20,c23n19,c23n18,c23n17,c23n16,c22n24,c22n06,c21n25
user1    11870 11334  0 16:09 pts/34   00:00:00 pmview -h c23n20 -title SGI PCP : Cluster Node Activity -xrm *iconName: clustervis
user1    11877 11870  0 16:09 ?        00:00:00 /usr/share/pcp/bin/pmtime -h -p /tmp/pmview.agWVsT -D 0
user1    12083 10785  0 16:10 pts/34   00:00:00 ps -ef
user1    12084 10785  0 16:10 pts/34   00:00:00 grep user1
[bono:~/pbs-examples]$
  • The -f and -u username will display all processes belong to that user
[bono:~/pbs-examples]$ ps -f -u user1
UID        PID  PPID  C STIME TTY          TIME CMD
user1    10723 10651  0 16:09 ?        00:00:00 sshd: user1@pts/34
user1    10785 10723  0 16:09 pts/34   00:00:00 -bash
user1    20863     1  0 16:43 pts/34   00:00:00 /bin/sh /usr/bin/clustervis -h c
user1    21382 20863  0 16:43 pts/34   00:00:00 pmview -h c23n20 -title SGI PCP
user1    21387 21382  0 16:43 ?        00:00:00 /usr/share/pcp/bin/pmtime -h -p
user1    21897 10785  0 16:44 pts/34   00:00:00 ps -f -u user1
[bono:~/pbs-examples]$

Graphical Tools

Debugging

Profiling

Center for Computational Research - University at Buffalo - State University of New York