Test results for the PVFS filesystem on U2
Tests against the /pvfsv2/scratch filesystem
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "simple_stripe"
distribution over a 256K stripe with iozone writing 4GB of data from
each client in 256K blocks:
| Initial write
|
216,321.93 KB/sec
|
| Rewrite
|
111,199.63 KB/sec
|
| Read
|
143,716.59 KB/sec
|
| Re-read
|
152,099.49 KB/sec
|
| Initial write
|
427,684.05 KB/sec
|
| Rewrite
|
290,176.45 KB/sec
|
| Read
|
184,586.92 KB/sec
|
| Re-read
|
167,355.25 KB/sec
|
| Initial write
|
319,170.51 KB/sec
|
| Rewrite
|
236,503.51 KB/sec
|
| Read
|
156,532.33 KB/sec
|
| Re-read
|
144,944.00 KB/sec
|
Mean Average:
| Initial write
|
321,058.83 KB/sec
|
| Rewrite
|
212,626.53 KB/sec
|
| Read
|
161,611.95 KB/sec
|
| Re-read
|
154,799.58 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "twod_stripe"
distribution using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe
over each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:
| Initial write
|
386815.16 KB/sec
|
| Rewrite
|
233697.07 KB/sec
|
| Read
|
187679.12 KB/sec
|
| Re-read
|
180028.95 KB/sec
|
| Initial write
|
361,327.47 KB/sec
|
| Rewrite
|
231,244.02 KB/sec
|
| Read
|
167,334.02 KB/sec
|
| Re-read
|
165,654.88 KB/sec
|
| Initial write
|
347,032.47 KB/sec
|
| Rewrite
|
266,103.62 KB/sec
|
| Read
|
171,985.70 KB/sec
|
| Re-read
|
167,333.48 KB/sec
|
Mean Average:
| Initial write
|
365,058.37 KB/sec
|
| Rewrite
|
243,681.57 KB/sec
|
| Read
|
175,666.28 KB/sec
|
| Re-read
|
171,005.77 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 16 client (only) nodes on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "varstrip_dist"
distribution with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 server nodes:
| Initial write
|
263,788.30 KB/sec
|
| Rewrite
|
152,860.62 KB/sec
|
| Read
|
207,850.97 KB/sec
|
| Re-read
|
200,526.21 KB/sec
|
| Initial write
|
323,117.79 KB/sec
|
| Rewrite
|
207,106.18 KB/sec
|
| Read
|
158,681.16 KB/sec
|
| Re-read
|
149,900.88 KB/sec
|
| Initial write
|
309,979.92 KB/sec
|
| Rewrite
|
231,989.58 KB/sec
|
| Read
|
243,098.39 KB/sec
|
| Re-read
|
210,171.22 KB/sec
|
Mean Average:
| Initial write
|
298,962.00 KB/sec
|
| Rewrite
|
197,318.79 KB/sec
|
| Read
|
203,210.17 KB/sec
|
| Re-read
|
186,866.10 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "simple_stripe"
distribution over a 256K stripe with iozone writing 4GB of data from
the client in 256K blocks:
| Initial write
|
77,267.67 KB/sec
|
| Rewrite
|
77,154.38 KB/sec
|
| Read
|
89,688.97 KB/sec
|
| Re-read
|
89,621.87 KB/sec
|
| Initial write
|
77,111.09 KB/sec
|
| Rewrite
|
76,897.03 KB/sec
|
| Read
|
89,865.34 KB/sec
|
| Re-read
|
89,489.88 KB/sec
|
| Initial write
|
76,186.16 KB/sec
|
| Rewrite
|
76,823.60 KB/sec
|
| Read
|
88,708.39 KB/sec
|
| Re-read
|
89,031.81 KB/sec
|
Mean Average:
| Initial write
|
76,854.97 KB/sec
|
| Rewrite
|
76,958.34 KB/sec
|
| Read
|
89,420.90 KB/sec
|
| Re-read
|
89,381.19 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "twod_stripe"
distribution using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe
over each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:
| Initial write
|
81,857.34 KB/sec
|
| Rewrite
|
82,699.58 KB/sec
|
| Read
|
89,442.70 KB/sec
|
| Re-read
|
89,721.23 KB/sec
|
| Initial write
|
81,624.40 KB/sec
|
| Rewrite
|
82,899.76 KB/sec
|
| Read
|
89,825.02 KB/sec
|
| Re-read
|
89,953.81 KB/sec
|
| Initial write
|
81,749.70 KB/sec
|
| Rewrite
|
82,835.59 KB/sec
|
| Read
|
89,962.66 KB/sec
|
| Re-read
|
89,653.48 KB/sec
|
Mean Average:
| Initial write
|
81,743.81 KB/sec
|
| Rewrite
|
82,811.64 KB/sec
|
| Read
|
89,743.46 KB/sec
|
| Re-read
|
89,776.17 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using the PVFSv2 filesystem served from 16 nodes,
each node running as a data & metadata server.
Filesystem mounted on 1 client (only) node on the /pvfsv2/scratch
directory. Test data written to a subdirectory using the "varstrip_dist"
distribution with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 server nodes:
| Initial write
|
58,210.20 KB/sec
|
| Rewrite
|
58,007.46 KB/sec
|
| Read
|
65,090.93 KB/sec
|
| Re-read
|
65,098.22 KB/sec
|
| Initial write
|
58,187.00 KB/sec
|
| Rewrite
|
58,072.45 KB/sec
|
| Read
|
65,097.72 KB/sec
|
| Re-read
|
65,043.86 KB/sec
|
| Initial write
|
57,718.01 KB/sec
|
| Rewrite
|
57,775.35 KB/sec
|
| Read
|
65,090.68 KB/sec
|
| Re-read
|
64,994.39 KB/sec
|
Mean Average:
| Initial write
|
58,038.40 KB/sec
|
| Rewrite
|
57,951.75 KB/sec
|
| Read
|
65,093.11 KB/sec
|
| Re-read
|
65,045.49 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Tests against PVFSv2 on the fly filesystems
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 256K stripe with iozone writing 4GB of data from each client
in 256K blocks:
| Initial write
|
217,327.22 KB/sec
|
| Rewrite
|
120,611.10 KB/sec
|
| Read
|
202,984.49 KB/sec
|
| Re-read
|
211,821.49 KB/sec
|
| Initial write
|
201,766.35 KB/sec
|
| Rewrite
|
141,896.95 KB/sec
|
| Read
|
181,411.11 KB/sec
|
| Re-read
|
197,476.14 KB/sec
|
| Initial write
|
238,825.01 KB/sec
|
| Rewrite
|
129,889.35 KB/sec
|
| Read
|
165,353.37 KB/sec
|
| Re-read
|
165,397.42 KB/sec
|
Mean Average:
| Initial write
|
219,306.19 KB/sec
|
| Rewrite
|
130,799.13 KB/sec
|
| Read
|
183,249.66 KB/sec
|
| Re-read
|
191,565.02 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe over
each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from the client in 256K blocks:
| Initial write
|
296,768.42 KB/sec
|
| Rewrite
|
150,572.37 KB/sec
|
| Read
|
172,171.22 KB/sec
|
| Re-read
|
176,810.98 KB/sec
|
| Initial write
|
333,846.28 KB/sec
|
| Rewrite
|
251,851.72 KB/sec
|
| Read
|
146,690.60 KB/sec
|
| Re-read
|
136,346.34 KB/sec
|
| Initial write
|
401,015.35 KB/sec
|
| Rewrite
|
286,796.02 KB/sec
|
| Read
|
182,275.91 KB/sec
|
| Re-read
|
186,093.76 KB/sec
|
Mean Average:
| Initial write
|
343,876.68 KB/sec
|
| Rewrite
|
229,740.04 KB/sec
|
| Read
|
167,045.91 KB/sec
|
| Re-read
|
166,417.03 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from each client in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 nodes:
| Initial write
|
69,492.11 KB/sec
|
| Rewrite
|
69,492.11 KB/sec
|
| Read
|
69,492.11 KB/sec
|
| Re-read
|
69,492.11 KB/sec
|
| Initial write
|
69,327.24 KB/sec
|
| Rewrite
|
69,608.65 KB/sec
|
| Read
|
65,117.41 KB/sec
|
| Re-read
|
65,245.79 KB/sec
|
| Initial write
|
69,325.20 KB/sec
|
| Rewrite
|
69,581.06 KB/sec
|
| Read
|
65,010.98 KB/sec
|
| Re-read
|
65,188.54 KB/sec
|
Mean Average:
| Initial write
|
69,381.52 KB/sec
|
| Rewrite
|
69,611.84 KB/sec
|
| Read
|
64,910.00 KB/sec
|
| Re-read
|
65,248.22 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 256K stripe with iozone writing 4GB of data from "one client"
in 256K blocks:
| Initial write
|
80,655.78 KB/sec
|
| Rewrite
|
80,072.49 KB/sec
|
| Read
|
94,615.61 KB/sec
|
| Re-read
|
94,553.12 KB/sec
|
| Initial write
|
79,990.79 KB/sec
|
| Rewrite
|
80,284.61 KB/sec
|
| Read
|
94,515.17 KB/sec
|
| Re-read
|
94,237.48 KB/sec
|
| Initial write
|
79,515.85 KB/sec
|
| Rewrite
|
79,831.05 KB/sec
|
| Read
|
94,458.15 KB/sec
|
| Re-read
|
94,448.64 KB/sec
|
Mean Average:
| Initial write
|
80,054.14 KB/sec
|
| Rewrite
|
80,062.72 KB/sec
|
| Read
|
94,529.64 KB/sec
|
| Re-read
|
94,413.08 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using 4 groups of 4 servers, striping 4 x 64KB = 256KB stripe over
each group, writing 8 * 256KB = 2MB to each group in turn.
iozone writing 4GB of data from "one client" in 256K blocks:
| Initial write
|
84,378.85 KB/sec
|
| Rewrite
|
85,881.42 KB/sec
|
| Read
|
94,208.76 KB/sec
|
| Re-read
|
94,470.95 KB/sec
|
| Initial write
|
84,331.27 KB/sec
|
| Rewrite
|
85,084.39 KB/sec
|
| Read
|
94,591.73 KB/sec
|
| Re-read
|
94,590.86 KB/sec
|
| Initial write
|
84,542.70 KB/sec
|
| Rewrite
|
86,194.13 KB/sec
|
| Read
|
94,106.49 KB/sec
|
| Re-read
|
94,902.87 KB/sec
|
Mean Average:
| Initial write
|
84,417.61 KB/sec
|
| Rewrite
|
85,719.98 KB/sec
|
| Read
|
94,302.33 KB/sec
|
| Re-read
|
94,654.89 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 16 nodes,
where all nodes run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from "one client" in 256K blocks.
The distribution was configured to write 16K to each node, thereby creating
a 256K stripe over the 16 nodes:
| Initial write
|
59,513.06 KB/sec
|
| Rewrite
|
59,647.35 KB/sec
|
| Read
|
66,560.80 KB/sec
|
| Re-read
|
66,358.22 KB/sec
|
| Initial write
|
58,402.26 KB/sec
|
| Rewrite
|
58,361.07 KB/sec
|
| Read
|
64,443.17 KB/sec
|
| Re-read
|
65,097.45 KB/sec
|
| Initial write
|
59,994.73 KB/sec
|
| Rewrite
|
60,027.61 KB/sec
|
| Read
|
65,241.22 KB/sec
|
| Re-read
|
65,820.34 KB/sec
|
Mean Average:
| Initial write
|
59,303.35 KB/sec
|
| Rewrite
|
59,345.34 KB/sec
|
| Read
|
65,415.06 KB/sec
|
| Re-read
|
65,758.67 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 1 node,
where the node runs data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "simple_stripe" distribution
over a 16K "stripe" (writing 16K blocks) with iozone writing 4GB of
data from the client in 256K blocks:
| Initial write
|
46,952.98 KB/sec
|
| Rewrite
|
37,435.67 KB/sec
|
| Read
|
48,961.50 KB/sec
|
| Re-read
|
51,697.74 KB/sec
|
| Initial write
|
38,263.25 KB/sec
|
| Rewrite
|
26,562.21 KB/sec
|
| Read
|
34,992.07 KB/sec
|
| Re-read
|
32,740.65 KB/sec
|
| Initial write
|
54,252.75 KB/sec
|
| Rewrite
|
41,115.54 KB/sec
|
| Read
|
53,793.60 KB/sec
|
| Re-read
|
57,763.04 KB/sec
|
Mean Average:
| Initial write
|
46,489.66 KB/sec
|
| Rewrite
|
35,037.81 KB/sec
|
| Read
|
45,915.72 KB/sec
|
| Re-read
|
47,400.48 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 1 node,
where the node run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "twod_stripe" distribution
using a 1 group with group strip factor 1, writing 64KB "stripe"
with iozone writing 4GB of data from the client in 256K blocks:
| Initial write
|
53,920.40 KB/sec
|
| Rewrite
|
40,583.86 KB/sec
|
| Read
|
57,601.16 KB/sec
|
| Re-read
|
55,430.86 KB/sec
|
| Initial write
|
56,067.90 KB/sec
|
| Rewrite
|
39,230.10 KB/sec
|
| Read
|
49,900.64 KB/sec
|
| Re-read
|
55,006.43 KB/sec
|
| Initial write
|
56,487.15 KB/sec
|
| Rewrite
|
39,892.87 KB/sec
|
| Read
|
52,636.87 KB/sec
|
| Re-read
|
57,601.55 KB/sec
|
Mean Average:
| Initial write
|
55,491.82 KB/sec
|
| Rewrite
|
39,902.28 KB/sec
|
| Read
|
53,379.56 KB/sec
|
| Re-read
|
56,012.95 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]
Test results using PVFSv2 on the fly filesystem over 1 node,
where the node run data server, metadata server & client.
Filesystem mounted on the ${PBSTMPDIR}/mnt directory. Test data
written to a subdirectory using the "varstrip_dist" distribution
with iozone writing 4GB of data from the client in 256K blocks.
The distribution was configured to write 16K blocks to the node.
| Initial write
|
50,170.40 KB/sec
|
| Rewrite
|
39,576.95 KB/sec
|
| Read
|
53,018.77 KB/sec
|
| Re-read
|
53,581.30 KB/sec
|
| Initial write
|
52,681.21 KB/sec
|
| Rewrite
|
36,710.65 KB/sec
|
| Read
|
53,072.68 KB/sec
|
| Re-read
|
53,128.93 KB/sec
|
| Initial write
|
45,512.93 KB/sec
|
| Rewrite
|
38,960.90 KB/sec
|
| Read
|
49,923.59 KB/sec
|
| Re-read
|
52,805.11 KB/sec
|
Mean Average:
| Initial write
|
49,454.85 KB/sec
|
| Rewrite
|
38,416.17 KB/sec
|
| Read
|
52,005.01 KB/sec
|
| Re-read
|
53,171.78 KB/sec
|
[PBS script for this test] - [Full output from the iozone tests]