Date created: 03/03/14 13:39:24. Last modified: 03/03/14 14:53:04

RAID - Set up & creation

Check disk tables,
fdisk -l

Disk /dev/sda: 120.0 GB, 120034123776 bytes
255 heads, 63 sectors/track, 14593 cylinders, total 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xcf66116f

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048      206847      102400    7  HPFS/NTFS/exFAT
/dev/sda2          206848   112639999    56216576    7  HPFS/NTFS/exFAT
/dev/sda3       112642046   234440703    60899329    5  Extended
/dev/sda5       112642048   120453119     3905536   82  Linux swap / Solaris
/dev/sda6       120455168   234440703    56992768   83  Linux

Disk /dev/sdb: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/sdb doesn't contain a valid partition table

Disk /dev/sdc: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/sdc doesn't contain a valid partition table

Disk /dev/sdd: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/sdd doesn't contain a valid partition table

Disk /dev/sde: 3000.6 GB, 3000592982016 bytes
255 heads, 63 sectors/track, 364801 cylinders, total 5860533168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x00000000

Disk /dev/sde doesn't contain a valid partition table

Create GTP tables on each raw disk and align them to a cylinder divisible by 4096 (repeat for /dev/sd[cde]). The default unit for mkpart is megabytes so start 1MB in and finish at 100% of disk size (-1). ;
parted -a optimal /dev/sdb

GNU Parted 2.3
Using /dev/sdd
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mklabel gpt                                                      
(parted) mkpart primary 1 -1                                              
(parted) align-check                                                      
alignment type(min/opt)  [optimal]/minimal? optimal                       
Partition number? 1                                                       
1 aligned
(parted) quit

fdisk can no longer show the table alignments of these drive because it doesn't support GPT to disks larger than 2TB. Use gdisk instead;
gdisk -l /dev/sdb

GPT fdisk (gdisk) version 0.8.1

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdb: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): BC49445F-2559-4B6B-8387-09ACA853DB25
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      5860532223   2.7 TiB     0700  primary

gdisk -l /dev/sdc

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdc: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 7D92F6A3-B46C-474C-8210-A86F5A8212F7
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      5860532223   2.7 TiB     0700  primary

gdisk -l /dev/sdd

Found valid GPT with protective MBR; using GPT.
Disk /dev/sdd: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): B83091EF-B6EF-4F44-89A5-18C31E307701
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      5860532223   2.7 TiB     0700  primary

gdisk -l /dev/sde

Found valid GPT with protective MBR; using GPT.
Disk /dev/sde: 5860533168 sectors, 2.7 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 25EC6A05-8839-4F9D-801C-4860905FADCB
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 5860533134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2925 sectors (1.4 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      5860532223   2.7 TiB     0700  primary

A quick performance test to each RAID partition rather than strait to risk shows similar results as before the partitioning so the alignment is correct;
dd if=/dev/zero of=/dev/sd[bcde] bs=1G count=1 oflag=direct

1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 7.37314 s, 146 MB/s

1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 7.77681 s, 138 MB/s

1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 7.82861 s, 137 MB/s

1+0 records in
1+0 records out
1073741824 bytes (1.1 GB) copied, 7.71615 s, 139 MB/s

madam version;

mdadm - v3.2.5 - 18th May 2012

Create a RAID 5 array, use the superblock version 1 format on RAID disks specifically sub-version 1.2 to place the superblock 4KB from the start of the disk for member each disk. Also use a chunk size of 4KB to mach the physical sector size of 4KB;
mdadm --create --verbose /dev/md0 --metadata 1.2 --level=5 --chunk=4 --raid-devices=4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1

mdadm: layout defaults to left-symmetric
mdadm: size set to 2930133880K
mdadm: array /dev/md0 started

Speed up the RAID sync by lifting the min and max sync rates

echo 500000 > /proc/sys/dev/raid/speed_limit_min
echo 500000 > /proc/sys/dev/raid/speed_limit_max

Also lift the stripe_cache_size

echo 16384 > /sys/block/md0/md/stripe_cache_size

Sync rate went from 71065K/sec to 143690K/sec.