Date created: 10/07/21 11:09:31. Last modified: 10/14/21 11:54:55

HTPCv3/Pi-NAS

Move Software RAID from Old NAS

Hard disk order in existing / legacy NAS:

gdisk -l /dev/sdb
Disk identifier (GUID): BC49445F-2559-4B6B-8387-09ACA853DB25

gdisk -l /dev/sdc
Disk identifier (GUID): 7D92F6A3-B46C-474C-8210-A86F5A8212F7

gdisk -l /dev/sdd
Disk identifier (GUID): B83091EF-B6EF-4F44-89A5-18C31E307701

gdisk -l /dev/sde
Disk identifier (GUID): 25EC6A05-8839-4F9D-801C-4860905FADCB

Match UUIDs /dev/sdX names with disk serial numbers:

$ udevadm info /dev/sdb | grep -E "ID_SERIAL|UUID"
E: ID_SERIAL=WDC_WD30EFRX-68EUZN0_WD-WMC4N1475240
E: ID_SERIAL_SHORT=WD-WMC4N1475240
E: ID_PART_TABLE_UUID=bc49445f-2559-4b6b-8387-09aca853db25

$ udevadm info /dev/sdc | grep -E "ID_SERIAL|UUID"
E: ID_SERIAL=WDC_WD30EFRX-68EUZN0_WD-WMC4N1911391
E: ID_SERIAL_SHORT=WD-WMC4N1911391
E: ID_PART_TABLE_UUID=7d92f6a3-b46c-474c-8210-a86f5a8212f7

$ udevadm info /dev/sdd | grep -E "ID_SERIAL|UUID"
E: ID_SERIAL=WDC_WD30EFRX-68EUZN0_WD-WMC4N1954715
E: ID_SERIAL_SHORT=WD-WMC4N1954715
E: ID_PART_TABLE_UUID=b83091ef-b6ef-4f44-89a5-18c31e307701

$ udevadm info /dev/sde | grep -E "ID_SERIAL|UUID"
E: ID_SERIAL=WDC_WD30EFRX-68EUZN0_WD-WCC4N0663527
E: ID_SERIAL_SHORT=WD-WCC4N0663527
E: ID_PART_TABLE_UUID=25ec6a05-8839-4f9d-801c-4860905fadcb

 

On the new machine /dev/sd[b,c,d,e] are /dev/sd[g,h,i,j] respectively.

With the disks in the new machine, and the mdadm.conf file copied over, the following command failed:

$ sudo mdadm --assemble --scan --verbose
mdadm: looking for devices for /dev/md/0
mdadm: No super block found on /dev/sdg (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdg
mdadm: No super block found on /dev/sdj (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdj
mdadm: No super block found on /dev/sdi (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdi
mdadm: No super block found on /dev/sdh (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdh
...
mdadm: Merging with already-assembled /dev/md/htpc-ubuntu:0
mdadm: /dev/sdg1 is identified as a member of /dev/md/htpc-ubuntu:0, slot 0.
mdadm: /dev/sdj1 is identified as a member of /dev/md/htpc-ubuntu:0, slot 3.
mdadm: /dev/sdi1 is identified as a member of /dev/md/htpc-ubuntu:0, slot 2.
mdadm: /dev/sdh1 is identified as a member of /dev/md/htpc-ubuntu:0, slot 1.
mdadm: failed to add /dev/sdh1 to /dev/md/htpc-ubuntu:0: Device or resource busy
mdadm: failed to add /dev/sdi1 to /dev/md/htpc-ubuntu:0: Device or resource busy
mdadm: failed to add /dev/sdj1 to /dev/md/htpc-ubuntu:0: Device or resource busy
mdadm: failed to add /dev/sdg1 to /dev/md/htpc-ubuntu:0: Device or resource busy
mdadm: failed to RUN_ARRAY /dev/md/htpc-ubuntu:0: Input/output error

The OS had tried to build device md127 automatically. Just needed to stop that device and then the RAID was detected and started just fine:

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md127 : inactive sde1[2] sdc1[1] sdd1[0] sdb1[4]
11720536064 blocks super 1.2

$ sudo mdadm --stop /dev/md127
mdadm: stopped /dev/md127

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
unused devices: <none>

$ sudo mdadm --assemble --scan --verbose
mdadm: looking for devices for /dev/md/0
mdadm: No super block found on /dev/sdg (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdg
mdadm: No super block found on /dev/sdj (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdj
mdadm: No super block found on /dev/sdi (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdi
mdadm: No super block found on /dev/sdh (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdh
....
mdadm: /dev/sdg1 is identified as a member of /dev/md/0, slot 0.
mdadm: /dev/sdj1 is identified as a member of /dev/md/0, slot 3.
mdadm: /dev/sdi1 is identified as a member of /dev/md/0, slot 2.
mdadm: /dev/sdh1 is identified as a member of /dev/md/0, slot 1.
mdadm: added /dev/sdh1 to /dev/md/0 as 1
mdadm: added /dev/sdi1 to /dev/md/0 as 2
mdadm: added /dev/sdj1 to /dev/md/0 as 3
mdadm: added /dev/sdg1 to /dev/md/0 as 0
mdadm: /dev/md/0 has been started with 4 drives.

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active (auto-read-only) raid5 sdg1[0] sdj1[4] sdi1[2] sdh1[1]
8790401640 blocks super 1.2 level 5, 4k chunk, algorithm 2 [4/4] [UUUU]
bitmap: 0/22 pages [0KB], 65536KB chunk

unused devices: <none>

$ /usr/share/mdadm/checkarray --status /dev/md0
md0: idle

 

If the old mdadm.conf file is not available we can create a new one. Add the following to /etc/mdadm/mdadm.conf:

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays

Then run the following (as root) to append the disk array definition to the file:

mdadm --detail --scan >> /etc/mdadm/mdadm.conf

 

Add to fstab and mount, this remounts the RAID as read-write (note it was auo-read-only above and one could have used "mdadm --readwrite md0" but that would be temporary). Note that nofail is after auto, so that if the drive isn't present at boot time the system won't stall:

$ echo "/dev/md0 /media/9TB-Array ext4 rw,auto,nofail,errors=remount-ro,nosuid,nodev,uhelper=udisks2 0 0" >> /etc/fstab

$ mount -a

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdg1[0] sdj1[4] sdi1[2] sdh1[1]
8790401640 blocks super 1.2 level 5, 4k chunk, algorithm 2 [4/4] [UUUU]
bitmap: 0/22 pages [0KB], 65536KB chunk

unused devices: <none>


$ sudo mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Mon Mar 3 14:14:29 2014
Raid Level : raid5
Array Size : 8790401640 (8383.18 GiB 9001.37 GB)
Used Dev Size : 2930133880 (2794.39 GiB 3000.46 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent

Intent Bitmap : Internal

Update Time : Thu Oct 14 12:51:45 2021
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 4K

Consistency Policy : bitmap

Name : htpc-ubuntu:0
UUID : 8ea128cd:47130c7a:9fb4a5a5:66750b6b
Events : 16225

Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 49 1 active sync /dev/sdd1
2 8 65 2 active sync /dev/sde1
4 8 33 3 active sync /dev/sdc1

 

Speed Test

USB is not a great protocol for disk access. A 4x3TB RAID-5 software array over a single USB3 port on a RasPi4 is not as slow as suspected:

Write:
$ dd status=progress if=/dev/zero of=/media/9TB-Array/zeros bs=1M count=1000
972029952 bytes (972 MB, 927 MiB) copied, 9 s, 108 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 9.61399 s, 109 MB/s

$ dd status=progress if=/dev/urandom of=/media/9TB-Array/urand bs=1M count=1000
1048576000 bytes (1.0 GB, 1000 MiB) copied, 57 s, 18.4 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 57.051 s, 18.4 MB/s

Read:

$ dd status=progress if=/media/9TB-Array/zeros of=/dev/null
990137856 bytes (990 MB, 944 MiB) copied, 13 s, 76.2 MB/s
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 13.9548 s, 75.1 MB/s

$ dd status=progress if=/media/9TB-Array/urand of=/dev/null
1023459328 bytes (1.0 GB, 976 MiB) copied, 18 s, 56.9 MB/s
2048000+0 records in
2048000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 18.4569 s, 56.8 MB/s

$ dd status=progress if=/media/9TB-Array/zeros of=/dev/null bs=1M count=1000
1009778688 bytes (1.0 GB, 963 MiB) copied, 7 s, 144 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 7.33777 s, 143 MB/s

$ dd status=progress if=/media/9TB-Array/urand2 of=/dev/null bs=1M count=1000
965738496 bytes (966 MB, 921 MiB) copied, 8 s, 121 MB/s
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 8.66835 s, 121 MB/s

Previous page: References
Next page: LineageOS Install/Upgrade