High Performance Computing
High End Visualization
About CCR Contact Us Events Facilities Faculty Project Highlights History Job Opportunities News Partners Staff
Bioinformatics Consulting Services Grid Computing Visualization
Accounts Application Software Consulting Getting Started Hardware Resources Frequently Asked Questions Machine Status MyCCR MyStats Overview Request Help Storage Resources Training/Courses Web Portals
Access Grid Training Outreach
Events Logos Media News Photo Album Videos
Contact Information Directions
  Dashboard > CCR Web > ... > PVFS version 2 scratch filesystem on U2 > PVFS version 2 scratch filesystem on U2 - Test Results
Log In   View a printable version of the current page.
PVFS version 2 scratch filesystem on U2 - Test Results

Test results for the PVFS filesystem on U2


Tests against the /pvfsv2/scratch filesystem

Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "simple_stripe"
distribution over a 256K stripe with iozone writing 4GB of data from
each client in 256K blocks:

Initial write
216,321.93 KB/sec
Rewrite
111,199.63 KB/sec
Read
143,716.59 KB/sec
Re-read
152,099.49 KB/sec
Initial write
427,684.05 KB/sec
Rewrite
290,176.45 KB/sec
Read
184,586.92 KB/sec
Re-read
167,355.25 KB/sec
Initial write
319,170.51 KB/sec
Rewrite
236,503.51 KB/sec
Read
156,532.33 KB/sec
Re-read
144,944.00 KB/sec
Mean Average:
Initial write
321,058.83 KB/sec
Rewrite
212,626.53 KB/sec
Read
161,611.95 KB/sec
Re-read
154,799.58 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "twod_stripe"
distribution using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe
over each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:

Initial write
386815.16 KB/sec
Rewrite
233697.07 KB/sec
Read
187679.12 KB/sec
Re-read
180028.95 KB/sec
Initial write
361,327.47 KB/sec
Rewrite
231,244.02 KB/sec
Read
167,334.02 KB/sec
Re-read
165,654.88 KB/sec
Initial write
347,032.47 KB/sec
Rewrite
266,103.62 KB/sec
Read
171,985.70 KB/sec
Re-read
167,333.48 KB/sec
Mean Average:
Initial write
365,058.37 KB/sec
Rewrite
243,681.57 KB/sec
Read
175,666.28 KB/sec
Re-read
171,005.77 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "varstrip_dist"
distribution with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 server nodes:

Initial write
263,788.30 KB/sec
Rewrite
152,860.62 KB/sec
Read
207,850.97 KB/sec
Re-read
200,526.21 KB/sec
Initial write
323,117.79 KB/sec
Rewrite
207,106.18 KB/sec
Read
158,681.16 KB/sec
Re-read
149,900.88 KB/sec
Initial write
309,979.92 KB/sec
Rewrite
231,989.58 KB/sec
Read
243,098.39 KB/sec
Re-read
210,171.22 KB/sec
Mean Average:
Initial write
298,962.00 KB/sec
Rewrite
197,318.79 KB/sec
Read
203,210.17 KB/sec
Re-read
186,866.10 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "simple_stripe"
distribution over a 256K stripe with iozone writing 4GB of data from
the client in 256K blocks:

Initial write
77,267.67 KB/sec
Rewrite
77,154.38 KB/sec
Read
89,688.97 KB/sec
Re-read
89,621.87 KB/sec
Initial write
77,111.09 KB/sec
Rewrite
76,897.03 KB/sec
Read
89,865.34 KB/sec
Re-read
89,489.88 KB/sec
Initial write
76,186.16 KB/sec
Rewrite
76,823.60 KB/sec
Read
88,708.39 KB/sec
Re-read
89,031.81 KB/sec
Mean Average:
Initial write
76,854.97 KB/sec
Rewrite
76,958.34 KB/sec
Read
89,420.90 KB/sec
Re-read
89,381.19 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "twod_stripe"
distribution using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe
over each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:

Initial write
81,857.34 KB/sec
Rewrite
82,699.58 KB/sec
Read
89,442.70 KB/sec
Re-read
89,721.23 KB/sec
Initial write
81,624.40 KB/sec
Rewrite
82,899.76 KB/sec
Read
89,825.02 KB/sec
Re-read
89,953.81 KB/sec
Initial write
81,749.70 KB/sec
Rewrite
82,835.59 KB/sec
Read
89,962.66 KB/sec
Re-read
89,653.48 KB/sec
Mean Average:
Initial write
81,743.81 KB/sec
Rewrite
82,811.64 KB/sec
Read
89,743.46 KB/sec
Re-read
89,776.17 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "varstrip_dist"
distribution with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 server nodes:

Initial write
58,210.20 KB/sec
Rewrite
58,007.46 KB/sec
Read
65,090.93 KB/sec
Re-read
65,098.22 KB/sec
Initial write
58,187.00 KB/sec
Rewrite
58,072.45 KB/sec
Read
65,097.72 KB/sec
Re-read
65,043.86 KB/sec
Initial write
57,718.01 KB/sec
Rewrite
57,775.35 KB/sec
Read
65,090.68 KB/sec
Re-read
64,994.39 KB/sec
Mean Average:
Initial write
58,038.40 KB/sec
Rewrite
57,951.75 KB/sec
Read
65,093.11 KB/sec
Re-read
65,045.49 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Tests against PVFSv2 on the fly filesystems

Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 256K stripe with iozone writing 4GB of data from each client
in 256K blocks:

Initial write
217,327.22 KB/sec
Rewrite
120,611.10 KB/sec
Read
202,984.49 KB/sec
Re-read
211,821.49 KB/sec
Initial write
201,766.35 KB/sec
Rewrite
141,896.95 KB/sec
Read
181,411.11 KB/sec
Re-read
197,476.14 KB/sec
Initial write
238,825.01 KB/sec
Rewrite
129,889.35 KB/sec
Read
165,353.37 KB/sec
Re-read
165,397.42 KB/sec
Mean Average:
Initial write
219,306.19 KB/sec
Rewrite
130,799.13 KB/sec
Read
183,249.66 KB/sec
Re-read
191,565.02 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe over
each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:

Initial write
296,768.42 KB/sec
Rewrite
150,572.37 KB/sec
Read
172,171.22 KB/sec
Re-read
176,810.98 KB/sec
Initial write
333,846.28 KB/sec
Rewrite
251,851.72 KB/sec
Read
146,690.60 KB/sec
Re-read
136,346.34 KB/sec
Initial write
401,015.35 KB/sec
Rewrite
286,796.02 KB/sec
Read
182,275.91 KB/sec
Re-read
186,093.76 KB/sec
Mean Average:
Initial write
343,876.68 KB/sec
Rewrite
229,740.04 KB/sec
Read
167,045.91 KB/sec
Re-read
166,417.03 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 nodes:

Initial write
69,492.11 KB/sec
Rewrite
69,492.11 KB/sec
Read
69,492.11 KB/sec
Re-read
69,492.11 KB/sec
Initial write
69,327.24 KB/sec
Rewrite
69,608.65 KB/sec
Read
65,117.41 KB/sec
Re-read
65,245.79 KB/sec
Initial write
69,325.20 KB/sec
Rewrite
69,581.06 KB/sec
Read
65,010.98 KB/sec
Re-read
65,188.54 KB/sec
Mean Average:
Initial write
69,381.52 KB/sec
Rewrite
69,611.84 KB/sec
Read
64,910.00 KB/sec
Re-read
65,248.22 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 256K stripe with iozone writing 4GB of data from "one client"
in 256K blocks:

Initial write
80,655.78 KB/sec
Rewrite
80,072.49 KB/sec
Read
94,615.61 KB/sec
Re-read
94,553.12 KB/sec
Initial write
79,990.79 KB/sec
Rewrite
80,284.61 KB/sec
Read
94,515.17 KB/sec
Re-read
94,237.48 KB/sec
Initial write
79,515.85 KB/sec
Rewrite
79,831.05 KB/sec
Read
94,458.15 KB/sec
Re-read
94,448.64 KB/sec
Mean Average:
Initial write
80,054.14 KB/sec
Rewrite
80,062.72 KB/sec
Read
94,529.64 KB/sec
Re-read
94,413.08 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe over
each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from "one client" in 256K blocks:

Initial write
84,378.85 KB/sec
Rewrite
85,881.42 KB/sec
Read
94,208.76 KB/sec
Re-read
94,470.95 KB/sec
Initial write
84,331.27 KB/sec
Rewrite
85,084.39 KB/sec
Read
94,591.73 KB/sec
Re-read
94,590.86 KB/sec
Initial write
84,542.70 KB/sec
Rewrite
86,194.13 KB/sec
Read
94,106.49 KB/sec
Re-read
94,902.87 KB/sec
Mean Average:
Initial write
84,417.61 KB/sec
Rewrite
85,719.98 KB/sec
Read
94,302.33 KB/sec
Re-read
94,654.89 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from "one client" in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 nodes:

Initial write
59,513.06 KB/sec
Rewrite
59,647.35 KB/sec
Read
66,560.80 KB/sec
Re-read
66,358.22 KB/sec
Initial write
58,402.26 KB/sec
Rewrite
58,361.07 KB/sec
Read
64,443.17 KB/sec
Re-read
65,097.45 KB/sec
Initial write
59,994.73 KB/sec
Rewrite
60,027.61 KB/sec
Read
65,241.22 KB/sec
Re-read
65,820.34 KB/sec
Mean Average:
Initial write
59,303.35 KB/sec
Rewrite
59,345.34 KB/sec
Read
65,415.06 KB/sec
Re-read
65,758.67 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 1 node,
where the node runs data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 16K "stripe" (writing 16K blocks) with iozone writing 4GB of
data from the client in 256K blocks:

Initial write
46,952.98 KB/sec
Rewrite
37,435.67 KB/sec
Read
48,961.50 KB/sec
Re-read
51,697.74 KB/sec
Initial write
38,263.25 KB/sec
Rewrite
26,562.21 KB/sec
Read
34,992.07 KB/sec
Re-read
32,740.65 KB/sec
Initial write
54,252.75 KB/sec
Rewrite
41,115.54 KB/sec
Read
53,793.60 KB/sec
Re-read
57,763.04 KB/sec
Mean Average:
Initial write
46,489.66 KB/sec
Rewrite
35,037.81 KB/sec
Read
45,915.72 KB/sec
Re-read
47,400.48 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 1 node,
where the node run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using a 1 group with group strip factor 1, writing 64KB "stripe"
with iozone writing 4GB of data from the client in 256K blocks:

Initial write
53,920.40 KB/sec
Rewrite
40,583.86 KB/sec
Read
57,601.16 KB/sec
Re-read
55,430.86 KB/sec
Initial write
56,067.90 KB/sec
Rewrite
39,230.10 KB/sec
Read
49,900.64 KB/sec
Re-read
55,006.43 KB/sec
Initial write
56,487.15 KB/sec
Rewrite
39,892.87 KB/sec
Read
52,636.87 KB/sec
Re-read
57,601.55 KB/sec
Mean Average:
Initial write
55,491.82 KB/sec
Rewrite
39,902.28 KB/sec
Read
53,379.56 KB/sec
Re-read
56,012.95 KB/sec

[PBS script for this test] - [Full output from the iozone tests]


Test results using PVFSv2 on the fly filesystem over 1 node,
where the node run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from the client in 256K blocks.
The distribution was configured to write 16K blocks to the node.

Initial write
50,170.40 KB/sec
Rewrite
39,576.95 KB/sec
Read
53,018.77 KB/sec
Re-read
53,581.30 KB/sec
Initial write
52,681.21 KB/sec
Rewrite
36,710.65 KB/sec
Read
53,072.68 KB/sec
Re-read
53,128.93 KB/sec
Initial write
45,512.93 KB/sec
Rewrite
38,960.90 KB/sec
Read
49,923.59 KB/sec
Re-read
52,805.11 KB/sec
Mean Average:
Initial write
49,454.85 KB/sec
Rewrite
38,416.17 KB/sec
Read
52,005.01 KB/sec
Re-read
53,171.78 KB/sec

[PBS script for this test] - [Full output from the iozone tests]

Center for Computational Research - University at Buffalo - State University of New York