Setting up a (new) system

From Linux Raid Wiki
Revision as of 20:02, 2 October 2016 by Anthony Youngman (Talk | contribs)

Jump to: navigation, search

This section describes setting up a RAID system using a distro like gentoo or slackware, that expects you to do a lot of the work yourself.


Planning the layout

Linux sees most storage as block devices. It is also quite happy to layer block devices on top of each other. The devices we need to consider are disk drives, raid arrays, logical volumes, and partitions. Network Block Devices can also be used. It is very important to get clear which layers go where in the stack - linux couldn't care, but it can get very confusing for admins, and confused admins means computer disasters.

Partitioning your system

Every system needs a / partition. Most systems have one or more swap partitions. Many systems have a separate /home partition. You may want to have other partitions. And do you want to use LVM to manage your user partitions?

If you have a separate /home partition, do you want this on a separate array?

Do you want linux to manage the swap space directly, or do you want it on raid?

Setting up the boot disks

Grub needs the first 2MB at the start of an MBR disk, or a partition of its own on a GPT disk, to install itself. UEFI needs its own partition. That means you must partition your boot drives. If you put swap directly on the drives you run the risk of a drive failure taking out any processes that may have been swapped on to that drive. Putting swap on an array will protect against this.

Use fdisk or gdisk (or the partition manager of your choice) to prep your boot disk. If you are using GPT, create a 1MB partition, type EF00 for UEFI, or EF02 for grub. If you are not putting swap on RAID, add your swap partition (I always size my swap as twice the maximum memory the motherboard will take. On a modern hard disk, that's nothing). Put the remaining space into a partition to create your RAID from. If you are not using LVM you can create multiple partitions for multiple arrays if you wish.

Now create your arrays.

mdadm --create /dev/md/root --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3

If you are using LVM, read the LVM howto and partition your array. I didn't, so

mkfs.ext4 /dev/md/root

Now you can install your distro. Make sure you install grub on all the disks. You don't want to lose the first disk and suddenly discover your system won't reboot!

Setting up grub

There are a lot of good resources out there on setting up grub 2 (this page ignores lilo and grub legacy because, well, they are legacy). However, be careful - for example the gentoo page for grub2/raid has currently been cobbled together and needs revising by someone who knows the difference between raid and lvm. (Disclaimer - Wol wrote most of the raid stuff and someone else has merged it with the lvm stuff.)

Grub is only required if you are using BIOS boot. If you are using UEFI, then that will load the initramfs for you and boot linux.

If you are using drives of 2TB or less, you can use the MBR. This will leave 2MB empty at the start of the disk for grub. If you are using drives over 2TB (or plan to be using UEFI in future) you should be using GPT. If you are using GPT you must create the 1MB type EF02 partition. This can then be changed to a type EF00 partition later for use with UEFI.

Edit the file /etc/default/grub and add "domdadm" to the following line


Now run grub-mkconfig, and check that your boot section contains the line

insmod mdraid1x

and that the line that loads linux looks something like

linux   /boot/vmlinuz-4.4.6-gentoo root=UUID=ab538350-d249-413b-86ef-4bd5280600b8 ro  domdadm

If you do a

ls -al /dev/disk/by-uuid

then the uuid should point to an md array.

Finally install grub to both your disks

grub-install /dev/sda
grub-install /dev/sdb

Note that you MUST use an initrd to boot with a version 1.2 array as the kernel can no longer assemble the array itself and needs to call out to userspace. This page does not discuss booting from older version arrays.

Setting up UEFI

[TODO: Write this section]

Setting up a separate home

It's a matter of preference whether to pass bare drives or a partition to the RAID if you're not booting off it. Some people think that's a bad idea, because installers assume a drive is blank if it doesn't have an MBR or GPT (as a gentoo guy, I don't run that sort of software). Others - including some notable raid names - prefer to use bare drives.

mdadm --create /dev/md/home --level=5 --raid-devices=3 /dev/sdc /dev/sdd /dev/sde
mkfs.ext4 /dev/md/home
Personal tools