RAID - Optimising & Management

Permanently increase RAID sync speed limits in /etc/sysctl.conf

dev.raid.speed_limit_min = 2000000
dev.raid.speed_limit_max = 2000000

Increase the RAID stripe cache size (number of entries in the stripe cache) using the following formulae;
((4096*stripe_cache_size)*num_drives)= RAM consumed for stripe cache
((4096*16384)*4) = 268435456/1024/1024 = 256MBs (268435456 / 16384 cache entries = 16384 16KB entries)

echo 16384 | tee /sys/block/md0/md/stripe_cache_size

It seems that NCQ is ignorant of the software RAID 5 under Linux and so whilst enabled on a per-disk basis it has only mnior positive effects with seemingly no negative effects, on a RAID basis it minor positive effects with major negative effects (even on sequential reads).

echo 1 > /sys/block/sdb/device/queue_depth
echo 1 > /sys/block/sdc/device/queue_depth
echo 1 > /sys/block/sdd/device/queue_depth
echo 1 > /sys/block/sde/device/queue_depth

Incrase the read-ahead value for the array as a whole, this can have a negative effect if raised to high though. Check the current read-ahead value (N * 512 byte sectors);

blockdev --getra /dev/md0
256

(256*512) / 1024 = 128kBs. Since this is mostly streaming media, increasing the value to 2048*512 bytes reads 1MB of data at a time.

blockdev --setra 2048 /dev/md0

Enable an interal write-intent bitmap log (as opposed to externally stored on another off-RAID device) for faster re-syncing of drives that are temporarily out of the array (such as disconnected or not detected during start up), which are only slightly out of sync;

mdadm --grow --bitmap=internal /dev/md0

---