HP ProLiant DL380 G5 Tuning Guide

This hardware was generously donated by HP.
== Upcoming tests ==
Linux Tunables: 
* SLUB vs. SLAB (memory management)
** http://lwn.net/Articles/229984/
* stats_targets (dbt3/tpc-h dependent)
* I/O Elevators
* SCSI controller tuning (might not be possible)
* physical disk partitioning
Postgres GUCs: 
* number of connections
Memory
* shared_buffers
WAL
* checkpoint_segments
* checkpoint_completion_target
* turning archive_mode on and just a vanilla copy
Planner
* effective cache size
* auto vacuum
* random_page_cost
* should we look at these? 
 #cpu_tuple_cost = 0.01 # same scale as above
 #cpu_index_tuple_cost = 0.005 # same scale as above
 #cpu_operator_cost = 0.0025 # same scale as above
* default_statistics_target = 10 # range 1-1000
== Hardware Details ==
* 2 x Quad Core Intel(R) Xeon(R) E5405 (2.00GHz, 1333MHz FSB, 80W) Processor
* HP 32GB Fully Buffered DIMM PC2-5300 8x4GB DR LP Memory
* HP Smart Array P800 Controller
* 8 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive
* MSA70 with 25 x HP 72GB Hot Plug 2.5 SAS 15,000 rpm Hard Drive
== Software Details ==
* 2.6.28-gentoo (Gentoo)
= Hardware Baseline =
== fio ==
We will use fio v1.23 to get some simple system characteristics using the internal disks of the system on various file systems, software and hardware raid configurations, and volume management configurations. 5 simple tests will be run: a sequential read, sequential write, random read, random write, and read-write test.
=== sequential read ===
 [seq-read]
 rw=read
 size=56g
 directory=/test
 fadvise_hint=0
 blocksize=8k
 direct=0
 numjobs=1
 nrfiles=1
 runtime=1h
 time_based
 exec_prerun=echo 3 > /proc/sys/vm/drop_caches
=== sequential write ===
 [seq-write]
 rw=write
 size=56g
 directory=/test
 fadvise_hint=0
 blocksize=8k
 direct=0
 numjobs=1
 nrfiles=1
 time_based
 runtime=1h
 exec_prerun=echo 3 > /proc/sys/vm/drop_caches
=== random read ===
 [random-read]
 rw=randread
 size=7g
 directory=/test
 fadvise_hint=0
 blocksize=8k
 direct=0
 numjobs=8
 nrfiles=1
 runtime=1h
 time_based
 exec_prerun=echo 3 > /proc/sys/vm/drop_caches
=== random write ===
 [random-write]
 rw=randwrite
 size=7g
 directory=/test
 fadvise_hint=0
 blocksize=8k
 direct=0
 numjobs=8
 ; this is number of files total per process
 nrfiles=1
 runtime=1h
 time_based
 exec_prerun=echo 3 > /proc/sys/vm/drop_caches
=== read-write===
 [read-write]
 rw=rw
 rwmixread=50
 size=7g
 directory=/test
 fadvise_hint=0
 blocksize=8k
 direct=0
 numjobs=8
 nrfiles=1
 runtime=1h
 time_based
 exec_prerun=echo 3 > /proc/sys/vm/drop_caches
== Results ==
=== One Disk ===
 Aggregate Bandwidth (MB/s) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	57.9 	89.0 	2.3 	6.6 	8.0/5.6
ext2-noatime 	59.4 	92.0 	2.3 	6.5 	8.0/6.8
ext3 	59.4 	81.8 	2.3 	22.8 	8.2/7.0
ext3 (noatime) 	 	 	 	 	
ext3 (data=journal) 	61.4 	72.4 	2.3 	11.0 	7.5/7.5
ext3 (data=journal,noatime}	60.8 	71.2 	2.3 	11.0 	7.5/7.5
jfs 	65.4 	97.0 	2.3 	4.8 	8.2/7.0
jfs (noatime) 	65.5 	95.0 	2.3 	4.8 	6.8/8.0
xfs 	71.9 	96.2 	* 	* 	*
xfs (noatime) 	68.8 	96.3 	2.3 	* 	*
* - Test did not complete successfully.
=== Two Disks ===
==== Software Raid ====
 Aggregate Bandwidth (MB/s) - RAID 0 (64KB chunk) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	87.0 	177.0 	4.2 	11.9 	13.6/13.6
ext2-noatime 	80.2 	180.0 	4.3 	12.0 	13.4/13.4
ext3 	86.2 	158 	4.2 	38.1 	17.8/9.0
ext3 (noatime) 	88.2 	161.0 	4.3 	39.6 	13.4/13.4
ext3 (data=journal) 	85.8 	83.0 	4.2 	17.1 	7.9/13.9
ext3 (data=journal,noatime)	87.6 	83.4 	4.2 	17.1 	10.3/14.4
jfs 	88.0 	181 	4.1 	7.9 	14.1/18.8
jfs (noatime) 	88.7 	175 	4.3 	7.8 	16.4/16.8
reiserfs 	96.2 	116 	4.3 	64.5 	13.6/18.2
reiserfs (noatime) 	92.6 	114 	4.3 	64.5 	16.0/11.4
==== Hardware Raid ====
 Aggregate Bandwidth (MB/s) - RAID 0 (128KB stripe) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	78.8 	166.0 	4.3 	12.0 	17.8/13.4
ext2-noatime 	78.9 	170.0 	4.3 	12.0 	13.4/15.6
ext3 	73.9 	82.5 	4.3 	41.0 	13.0/13.0
ext3 (noatime) 	74.5 	77.5 	4.3 	48.0 	15.2/11.0
ext3 (data=journal) 	78.9 	75.9 	4.3 	16.6 	15.8/15.8
ext3 (data=journal,noatime)	78.0 	75.9 	4.3 	16.4 	13.8/9.9
jfs 	78.8 	128.0 	4.4 	8.0 	16.2/15.9
jfs (noatime) 	79.3 	133.0 	4.4 	8.0 	18.2/11.5
reiserfs 	75 	115 	4.32 	61.2 	13.1/10.1
reiserfs (noatime) 	ND 	ND 	ND 	ND 	ND
==== LVM ====
 Aggregate Bandwidth (MB/s) - 4MB Stripesize - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	136.0 	186.0 	4.5 	11.9 	26.4/26.4
ext2 (noatime) 	137.0 	190.0 	4.5 	12.1 	23.2/23.2
ext3 	129.0 	161.0 	4.5 	38.9 	12.8/22.8
ext3 (noatime) 	136.0 	169.0 	4.5 	38.4 	26.2/19.9
ext3 (data=journal) 	135.0 	90.4 	4.4 	16.6 	15.2/21.2
ext3 (data=journal,noatime)	137.0 	89.6 	4.5 	16.9 	21.2/15.2
jfs 	148.0 	180.0 	4.5 	7.9 	17.9/25.0
jfs (noatime) 	147.0 	180.0 	4.5 	8.1 	24.9/21.4
=== Three Disks ===
==== Software RAID ====
 Aggregate Bandwidth (MB/s) - RAID 5 (4MB chunk) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	200.0 	43.1 	5.8 	4.2 	8.2/9.9
ext2 (noatime) 	200.0 	43.3 	5.8 	4.2 	11.5/9.9
ext3 	199.0 	45.6 	5.7 	9.6 	24.0/24.0
ext3 (noatime) 	201.0 	46.3 	5.7 	9.5 	4.7/12.6
ext3 (data=journal) 	199.0 	24.3 	5.8 	4.7 	4.7/7.3
ext3 (data=journal,noatime)	200.0 	24.4 	5.8 	4.6 	6.1/8.5
jfs 	218.0 	43.2 	5.8 	2.7 	12.0/12.0
jfs (noatime) 	218.0 	43.3 	5.9 	2.7 	6.9/12.1
 Aggregate Bandwidth (MB/s) - RAID 5 (4MB chunk) - 8192KB readahead buffer - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2	199.0 	42.6 	5.2 	4.2 	9.9/9.9
==== Hardware RAID ====
 Aggregate Bandwidth (MB/s) - RAID 5 - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	 	164.0 	5.6 	6.2 	7.0/8.4
ext2 (noatime) 	108.0 	169.0 	5.6 	6.2 	9.3/9.3
ext3 	104.0 	129.0 	5.5 	23.5 	7.9/9.2
ext3 (noatime) 	94.6 	131.0 	5.5 	23.6 	8.1/9.4
ext3 (data=journal) 	107.0 	79.2 	5.5 	9.7 	8.0/5.4
ext3 (data=journal,noatime)	108.0 	79.0 	5.5 	9.4 	5.3/8.0
jfs 	105.0 	179.0 	5.3 	4.6 	10.0/8.6
jfs (noatime) 	104.0 	179.0 	5.6 	4.5 	8.6/8.6
reiserfs 	108.0 	76.5 	5.6 	58.8 	8.3/9/7
reiserfs (noatime) 	109.0 	75.3 	5.6 	59.0 	9.7/8.5
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	113.0 	170.0 	5.8 	6.4 	10.9/7.9
ext2 (noatime) 	103.0 	169.0 	5.8 	6.3 	12.4/9.4
ext3 	106.0 	131.0 	5.7 	21.0 	11.7/10.0
ext3 (noatime) 	112.0 	135.0 	5.7 	21.4 	11.9/10.4
ext3 (data=journal) 	112.0 	74.8 	5.7 	8.0 	11.7/10.3
ext3 (data=journal,noatime)	112.0 	74.2 	5.8 	8.1 	11.7/5.8
ext3 (data=writeback) 	113.0 	135.0 	5.7 	5.2 	11.0/3.1
jfs 	110.0 	178.0 	5.5 	4.4 	9.5/11.1
jfs (noatime) 	110.0 	172.0 	5.8 	4.5 	9.8/9.5
reiserfs 	114.0 	92.6 	5.8 	58.3 	10.8/12.3
reiserfs (noatime) 	112.0 	78.7 	5.8 	58.6 	6.5/12.3
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 1 whole disk partition
 	sequential read	sequential write	random read	random write	read-write
ext2 	113.0 	171.0 	5.8 	6.3 	7.8/10.9
ext2 (noatime) 	108.0 	169.0 	5.8 	6.3 	7.8/10.9
ext3 	109.0 	132.0 	5.7 	21.0 	10.5/9.0
ext3 (noatime) 	112.0 	135.0 	5.7 	20.9 	10.5/9.0
ext3 (data=journal) 	113.0 	75.4 	5.7 	8.3 	4.4/11.4
ext3 (data=journal,noatime)	113.0 	75.4 	5.8 	8.3 	8.9/11.7
jfs 	110.0 	172.0 	5.8 	4.5 	8.0/11.1
jfs (noatime) 	111.0 	178.0 	5.8 	4.6 	6.4/9.6
reiserfs 	113.0 	113.0 	5.8 	58.4 	7.7/10.8
reiserfs (noatime) 	114.0 	98.0 	5.8 	59.0 	10.8/6.3
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 256KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	124.0 	172.0 	5.8 	6.3 	12.8/8.0
ext2 (noatime) 	125.0 	174.0 	5.8 	6.3 	8.0/11.2
ext3 	118.0 	131.0 	5.7 	21.0 	10.7/5.1
ext3 (noatime) 	119.0 	136.0 	5.7 	21.1 	10.7/9.3
ext3 (data=journal) 	116.0 	73.8 	5.7 	8.4 	10.5/4.6
ext3 (data=journal,noatime) 	125.0 	74.2 	5.2 	8.2 	10.5/12.0
ext3 (data=writeback) 	121.0 	134.0 	5.7 	5.2 	6.6/7.9
ext3 (data=writeback,noatime)	118.0 	133.0 	5.7 	5.3 	11.1/12.7
jfs 	121.0 	178.0 	5.8 	4.6 	8.4/9.8
jfs (noatime) 	120.0 	178.0 	5.8 	4.5 	9.9/9.8
reiserfs 	125.0 	93.3 	5.8 	58.4 	9.5/12.7
reiserfs (noatime) 	131.0 	74.0 	5.8 	58.5 	11.1/12.7
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 512KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	156.0 	167.0 	5.8 	6.3 	6.6/8.3
ext2 (noatime) 	141.0 	171.0 	5.8 	6.4 	11.6/9.9
ext3 	154.0 	133.0 	5.7 	21.2 	6.3/11.0
ext3 (noatime) 	155.0 	132.0 	5.2 	20.7 	9.5/12.6
ext3 (data=journal) 	156.0 	76.1 	5.7 	8.3 	12.4/9.3
ext3 (data=journal,noatime) 	155.0 	74.8 	5.8 	8.4 	12.4/9.4
ext3 (data=writeback) 	163.0 	132.0 	5.7 	5.2 	10.9/9.3
ext3 (data=writeback,noatime)	154.0 	132.0 	5.2 	5.2 	9.8/8.2
jfs 	169.0 	179.0 	5.5 	4.6 	10.1/10.1
jfs (noatime) 	170.0 	178.0 	5.8 	4.5 	11.8/11.8
reiserfs 	161.0 	95.7 	5.8 	58.5 	9.8/9.8
reiserfs (noatime) 	163.0 	69.3 	5.8 	58.5 	5.1/11.5
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 1024KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	171.0 	172.0 	5.8 	6.3 	12.0/8.5
ext2 (noatime) 	173.0 	171.0 	5.8 	6.3 	10.2/13.6
ext3 	172.0 	136.0 	5.7 	21.3 	11.6/9.8
ext3 (noatime) 	175.0 	133.0 	5.8 	20.8 	8.2/13.1
ext3 (data=journal) 	172.0 	75.5 	5.8 	8.2 	4.8/11.3
ext3 (data=journal,noatime) 	171.0 	75.3 	5.8 	8.3 	12.4/9.4
ext3 (data=writeback) 	178.0 	135.0 	5.7 	5.2 	11.4/9.7
ext3 (data=writeback,noatime)	181.0 	133.0 	5.7 	5.2 	6.7/13.5
jfs 	188.0 	172.0 	5.8 	4.4 	14.0/10.5
jfs (noatime) 	187.0 	178.0 	5.5 	4.3 	14.0/12.3
reiserfs 	185.0 	104.0 	5.8 	58.4 	13.6/3.4
reiserfs (noatime) 	187.0 	81.0 	5.8 	58.6 	10.2/8.5
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 2048KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	174.0 	172.0 	5.8 	6.3 	9.9/6.6
ext2 (noatime) 	171.0 	165.0 	5.8 	6.3 	8.6/10.4
ext3 	172.0 	135.0 	5.7 	20.7 	13.3/8.3
ext3 (noatime) 	172.0 	137.0 	5.7 	21.2 	11.7/11.7
ext3 (data=journal) 	173.0 	76.1 	5.8 	8.1 	8.2/8.3
ext3 (data=journal,noatime) 	173.0 	76.7 	5.8 	8.2 	12.7/9.5
ext3 (data=writeback) 	173.0 	133.0 	5.7 	5.2 	10.3/10.3
ext3 (data=writeback,noatime)	173.0 	134.0 	5.8 	5.2 	13.7/8.6
jfs 	174.0 	173.0 	5.8 	4.4 	12.5/5.4
jfs (noatime) 	173.0 	178.0 	5.8 	4.5 	12.5/12.5
reiserfs 	174.0 	113.0 	5.8 	58.4 	12.2/12.2
reiserfs (noatime) 	174.0 	75.0 	5.8 	58.6 	13.9/12.2
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 4096KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	176.0 	170.0 	5.8 	6.3 	13.9/13.9
ext2 (noatime) 	187.0 	173.0 	5.4 	6.3 	5.4/14.0
ext3 	179.0 	135.0 	5.7 	21.1 	8.4/13.5
ext3 (noatime) 	178.0 	133.0 	5.7 	21.0 	8.4/13.5
ext3 (data=journal) 	178.0 	74.3 	5.8 	7.9 	11.3/9.7
ext3 (data=journal,noatime) 	177.0 	76.0 	5.8 	8.2 	8.4/11.7
ext3 (data=writeback) 	175.0 	134.0 	5.7 	5.2 	8.7/12.1
ext3 (data=writeback,noatime)	186.0 	133.0 	5.2 	5.2 	7.0/12.1
jfs 	185.0 	172.0 	5.8 	4.5 	12.7/12.8
jfs (noatime) 	185.0 	172.0 	5.8 	4.5 	9.1/14.5
reiserfs 	188.0 	82.6 	5.8 	58.6 	14.0/12.3
reiserfs (noatime) 	191.0 	88.5 	5.8 	58.3 	8.9/12.3
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 8192KB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	183.0 	170.0 	5.8 	6.3 	7.0/12.3
ext2 (noatime) 	183.0 	171.0 	5.8 	6.3 	8.8/8.8
ext3 	177.0 	136.0 	5.7 	21.2 	10.2/11.9
ext3 (noatime) 	174.0 	134.0 	5.7 	21.1 	8.6/11.9
ext3 (data=journal) 	171.0 	75.7 	5.7 	8.2 	5.1/11.8
ext3 (data=journal,noatime) 	185.0 	76.1 	5.8 	7.9 	11.8/11.8
ext3 (data=writeback) 	185.0 	131.0 	5.7 	5.2 	8.7/12.2
ext3 (data=writeback,noatime)	186.0 	131.0 	5.7 	5.2 	13.9/10.4
jfs 	193.0 	172.0 	5.8 	4.5 	9.2/12.8
jfs (noatime) 	194.0 	172.0 	5.8 	4.5 	11.0/9.2
reiserfs 	194.0 	98.0 	5.8 	58.6 	12.8/12.7
reiserfs (noatime) 	195.0 	92.3 	5.8 	58.7 	14.3/12.5
 Aggregate Bandwidth (MB/s) - RAID 5 (256KB stripe) - 128MB readahead
 	sequential read	sequential write	random read	random write	read-write
ext2 	183.0 	168.0 	5.8 	6.3 	13.4/10.9
ext2 (noatime) 	187.0 	162.0 	5.8 	6.3 	12.9/9.2
ext3 	173.0 	132.0 	5.7 	21.1 	12.0/8.7
ext3 (noatime) 	168.0 	135.0 	5.8 	20.8 	7.0/12.0
ext3 (data=journal) 	184.0 	73.9 	5.7 	8.1 	8.4/10.1
ext3 (data=journal,noatime) 	182.0 	76.0 	5.8 	8.3 	11.8/8.4
ext3 (data=writeback) 	180.0 	133.0 	5.7 	5.2 	11.3/7.3
ext3 (data=writeback,noatime)	181.0 	135.0 	5.7 	5.2 	11.0/7.6
jfs 	204.0 	172.0 	5.8 	4.5 	8.0/15.6
jfs (noatime) 	204.0 	173.0 	5.8 	4.4 	15.6/9.7
reiserfs 	199.0 	83.6 	5.8 	58.7 	13.3/13.3
reiserfs (noatime) 	199.0 	113.0 	5.8 	58.4 	11.4/13.3
=== Four Disks ===
==== Software RAID ====
 Aggregate Bandwidth (MB/s) - RAID 10 (4MB chunk) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	262.0 	178.1 	7.8 	54.9 	20.0/28.0
ext2 (noatime) 	262.0 	183.0 	6.9 	56.9 	26.5/19.8
ext3 	168.0 	140.0 	7.6 	28.2 	22.8/27.4
ext3 (noatime) 	269.0 	135.0 	7.8 	28.2 	23.3/24.1
ext3 (data=journal) 	262.0 	61.7 	7.6 	11.1 	19.1/19.1
ext3 (data=journal,noatime)	263.0 	61.5 	7.8 	10.0 	19.0/15.8
jfs 	329.0 	162.0 	7.8 	7.2 	25.3/25.3
jfs (noatime) 	328.0 	163.0 	7.8 	7.3 	33.7/21.0
reiserfs 	316.0 	88.0 	7.6 	32.4 	24.3/28.5
reiserfs (noatime) 	318.0 	84.6 	7.8 	32.3 	20.1/28.2
 Aggregate Bandwidth (MB/s) - RAID 10 (4MB chunk) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2	239.0 	179.0 	7.8 	55.7 	27.2/23.8
==== Hardware RAID ====
 Aggregate Bandwidth (MB/s) - RAID 10 (128KB stripe) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	132.0 	170.0 	7.5 	11.5 	16.5/18.7
ext2 (noatime) 	131.0 	172.0 	7.6 	11.9 	16.5/18.7
ext3 	127.0 	115.0 	7.4 	40.5 	6.7/15.7
ext3 (noatime) 	127.0 	115.0 	7.5 	40.7 	14.0/14.0
ext3 (data=journal) 	132.0 	73.2 	7.5 	17.3 	10.7/14.9
ext3 (data=journal,noatime)	127.0 	115.5 	7.5 	40.7 	14.0/14.0
jfs 	135.0 	145.0 	7.6 	32.4 	24.3/28.5
jfs (noatime) 	137.0 	148.0 	7.6 	8.1 	12.8/12.6
reiserfs 	127.0 	62.1 	7.5 	59.3 	17.0/17.4
reiserfs (noatime) 	130.0 	63.9 	7.6 	59.5 	17.0/15.2
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - No partition table
 	sequential read	sequential write	random read	random write	read-write
ext2 	123.0 	170.0 	7.6 	11.7 	20.6/18.1
ext2 (noatime) 	122.0 	172.0 	7.6 	11.7 	15.5/15.5
ext3 	117.0 	122.0 	7.5 	38.5 	17.0/12.9
ext3 (noatime) 	112.0 	104.0 	7.6 	38.9 	19.5/14.7
ext3 (data=journal) 	122.0 	71.4 	7.6 	16.8 	13.3/11.1
ext3 (data=journal,noatime)	122.0 	69.8 	7.6 	16.9 	13.3/15.5
jfs 	122.0 	146.0 	7.7 	8.0 	18.2/13.0
jfs (noatime) 	123.0 	146.0 	7.7 	8.0 	18.3/15.7
reiserfs 	120.0 	67.2 	7.6 	59.7 	13.0/20.3
reiserfs (noatime) 	120.0 	75.1 	7.7 	59.4 	17.8/15.3
 Aggregate Bandwidth (MB/s) - RAID 10 (128KB stripe) - Using 1 whole disk partition
 	sequential read	sequential write	random read	random write	read-write
ext2 	131.0 	172.0 	7.6 	11.5 	15.1/12.3
ext2 (noatime) 	131.0 	172.0 	7.6 	11.6 	9.9/19.9
ext3 	127.0 	115.0 	7.4 	40.1 	16.3/9.3
ext3 (noatime) 	129.0 	114.0 	7.4 	40.0 	16.3/16.3
ext3 (data=journal) 	131.0 	73.0 	7.5 	17.2 	12.8/13.1
ext3 (data=journal,noatime)	131.0 	74.1 	7.5 	17.0 	12.8/14.9
jfs 	133.0 	148.0 	7.6 	8.2 	20.2/12.6
jfs (noatime) 	134.0 	149.0 	7.6 	7.8 	20.3/15.2
reiserfs 	126.0 	58.7 	7.5 	59.7 	17.0/17.4
reiserfs (noatime) 	129.0 	61.6 	7.6 	59.0 	19.5/12.2
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 512KB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
ext2 	159.0 	173.0 	7.6 	11.6 	19.8/15.1
ext2-noatime 	158.0 	170.0 	7.6 	11.7 	17.0/19.8
ext3 	155.0 	120.0 	7.5 	38.7 	12.1/12.1
ext3-journal 	157.0 	71.9 	7.6 	16.9 	11.4/16.0
ext3-journal-noatime 	146.0 	70.2 	7.6 	16.7 	11.4/15.9
ext3-noatime 	157.0 	122.0 	7.5 	38.6 	17.6/20.1
ext3-writeback 	157.0 	117.0 	7.6 	7.7 	17.3/10.2
ext3-writeback-noatime	158.0 	118.0 	7.6 	7.2 	17.2/12.3
jfs 	161.0 	150.0 	7.7 	8.2 	14.4/17.3
jfs-noatime 	166.0 	150.0 	7.3 	8.3 	14.4/17.3
reiserfs 	158.0 	65.8 	7.6 	59.6 	17.9/17.4
reiserfs-noatime 	162.0 	76.2 	7.7 	60.3 	14.9/12.4
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 1024KB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
ext2 	163.0 	172.0 	7.6 	11.6 	21.0/15.0
ext2-noatime 	164.0 	169.0 	7.6 	11.7 	15.5/15.0
ext3 	163.0 	121.0 	7.6 	39.1 	15.5/15.5
ext3-journal 	161.0 	72.1 	7.6 	17.1 	17.5/8.8
ext3-journal-noatime 	162.0 	71.2 	7.6 	16.6 	15.4/11.3
ext3-noatime 	163.0 	127.0 	7.6 	38.1 	13.0/18.2
ext3-writeback 	161.0 	118.0 	7.6 	7.7 	20.6/15.5
ext3-writeback-noatime	161.0 	118.0 	7.6 	7.3 	18.1/18.1
jfs 	171.0 	146.0 	7.7 	8.0 	22.2/25.4
jfs-noatime 	169.0 	150.0 	7.3 	8.2 	25.4/15.9
reiserfs 	167.0 	63.8 	7.6 	59.3 	18.2/18.2
reiserfs-noatime 	170.0 	68.9 	7.7 	59.8 	18.2/18.2
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 2048KB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
ext2 	169.0 	171.0 	7.6 	11.5 	12.5/24.9
ext2-noatime 	168.0 	171.0 	7.6 	11.6 	15.9/22.5
ext3 	169.0 	117.0 	7.5 	38.9 	15.7/15.7
ext3-journal 	168.0 	69.4 	6.9 	17.1 	11.1/15.6
ext3-journal-noatime 	168.0 	71.6 	7.6 	17.4 	13.5/11.1
ext3-noatime 	168.0 	125.0 	7.6 	38.9 	15.7/15.8
ext3-writeback 	165.0 	118.0 	6.9 	7.6 	16.0/18.6
ext3-writeback-noatime	168.0 	120.0 	7.6 	7.7 	15.9/13.2
jfs 	174.0 	147.0 	7.7 	8.3 	23.5/23.5
jfs-noatime 	170.0 	150.0 	7.7 	7.9 	20.2/20.2
reiserfs 	172.0 	62.5 	7.6 	60.0 	16.9/13.5
reiserfs-noatime 	173.0 	64.6 	7.7 	59.5 	16.7/16.1
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 4096KB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
ext2 	174.0 	172.0 	7.6 	11.6 	22.9/19.6
ext2-noatime 	173.0 	171.0 	7.6 	11.6 	18.4/18.4
ext3 	174.0 	122.0 	7.6 	38.7 	16.7/17.1
ext3-journal 	174.0 	71.3 	7.6 	16.7 	11.3/18.1
ext3-journal-noatime 	172.0 	70.9 	7.6 	16.8 	15.4/17.6
ext3-noatime 	173.0 	123.0 	7.6 	38.3 	19.1/13.6
ext3-writeback 	174.0 	117.0 	7.6 	7.4 	13.8/19.3
ext3-writeback-noatime	173.0 	117.0 	7.6 	7.6 	16.6/13.8
jfs 	178.0 	146.0 	7.7 	8.1 	27.5/17.2
jfs-noatime 	174.0 	150.0 	7.7 	8.2 	20.6/24.1
reiserfs 	178.0 	82.9 	7.6 	59.6 	19.7/19.7
reiserfs-noatime 	178.0 	67.1 	7.7 	59.4 	17.0/22.6
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 8192KB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
ext2 	172.0 	172.0 	6.9 	11.6 	19.9/23.2
ext2-noatime 	170.0 	172.0 	7.6 	11.6 	23.2/20.0
ext3 	173.0 	118.0 	7.6 	39.1 	11.3/16.8
ext3-journal 	171.0 	71.5 	7.6 	16.8 	13.6/11.4
ext3-journal-noatime 	171.0 	70.7 	7.6 	16.8 	13.6/11.3
ext3-noatime 	174.0 	119.0 	7.6 	38.8 	19.8/17.0
ext3-writeback 	171.0 	119.0 	7.6 	7.3 	16.6/19.5
ext3-writeback-noatime	171.0 	117.0 	7.6 	7.3 	14.1/22.3
jfs 	188.0 	149.0 	7.3 	8.3 	17.5/24.4
jfs-noatime 	187.0 	150.0 	7.7 	8.2 	17.4/20.9
reiserfs 	185.0 	62.6 	7.6 	60.1 	23.0/11.5
reiserfs-noatime 	186.0 	73.1 	7.7 	59.9 	11.6/23.0
 Aggregate Bandwidth (MB/s) - RAID 10 (256KB stripe) - 128MB readahead buffer.
 	sequential read	sequential write	random read	random write	read-write
reiserfs 	189.0 	67.5 	7.6 	60.0 	14.4/17.2
reiserfs-noatime	188.0 	73.3 	7.7 	59.6 	20.1/17.2
= Analysis =
== Hardware vs. Software Raid ==
=== Sequential Writes ===
 Total Processor Utilization (%) x Throughput (MB/s)
 	One Disk 	Two Disk Software RAID-0 (64KB stripe)	Two Disk Hardware RAID-0 (128KB stripe)	LVM (4MB stripe)	Three Disk Software RAID-5 (4MB stripe)	Three Disk Hardware RAID-5 (128KB stripe)	Three Disk Hardware RAID-5 (256KB stripe)	Three Disk Hardware RAID-5 (256KB stripe) w/ partition table	Four Disk Software RAID-10 (4MB stripe)	Four Disk Hardware RAID-10 (128KB stripe)	Four Disk Hardware RAID-10 (256KB stripe)	Four Disk Hardware RAID-10 (128KB stripe) w/ partition table
ext2	50-100 | 89.0	70-95 | 177.0 	30 | 166.0 	40 | 186.0 	30 | 43.1 	50 | 164.0 	40 | 170.0 	40 | 171.0 	60-100 | 178.1 	30 | 170.0 	30 | 170.0 	30 | 172.0
== Readahead Buffer Size ==
The readahead buffer size can be set per device by using the 'blockdev' tool as part of the util-linux package. For example, the command is 'blockdev --setra <blocks>'. Note that a block is 512 bytes. 'man blockdev' for more info.
=== Hardware RAID ===
 Sequential Read (MB/s) - 256KB stripe - ext2
readahead buffer size	128KB	256KB	512KB	1024KB	2048KB	4096KB	8192KB	128MB
RAID-5 	113 	124 	156 	171 	174 	176 	183 	183
=Presentations=
* Linux Plumbers Conference 2008 - Filesystem I/O From a Database Perspective
* PostgreSQL West 2008 - Linux Filesystems, RAID, and more
* SCaLE Feb 2009 - What Assuptions Make: Filesystem I/O from a database perspective
Data used for these presentation are publicly available in a git repository: http://git.postgresql.org/gitweb?p=performance-tuning.git
Category:Benchmarking