Choosing your hardware, and what is a device?

From Linux Raid Wiki
Revision as of 17:04, 11 October 2016 by Anthony Youngman (Talk | contribs)

Jump to: navigation, search


What is a device?

As far as linux is concerned, all storage is a "block device". This means that data is written in chunks, typically 4KB in size, and that these chunks may be written to random locations on the device. Block devices are often layered, which means that one device writes to another, which may then write to a third, and so on. One typically thinks of a device as a hard disk, but as far as linux is concerned that is just one of many.

A typical raid system will have several hard drives as the bottom layer. These will then be partitioned (another block device) and the partitions will be combined into a raid array (yet another block device). The array will then have lvm over that (yet another block device) before this is partitioned (yet another device layer).

From this it can be seen that linux doesn't particularly what devices you use to make up an array - you can combine arrays, you can use a network storage device, you can use SSDs, SATA or PATA disks, USB, whatever. However, certain choices may or may not be wise.

(In contrast to block devices, linux also has character devices, which just read or write a continuous stream of characters. Not suitable for storage because you have to start at the beginning and end at the end, so you would have to rewrite everything every time you wanted to change just a little bit - like an old-fashioned tape drive, for example).

When creating your arrays, make sure you have it clear in your mind (and well documented for your successor, who is quite likely to be you six months down the line!) what devices fit where in the hierarchy.

USB devices

Do NOT use these as part of your array. They have a timeout/disconnect mechanism which interacts very badly with the raid code. An array which includes USB is likely to fall over in a heap. USB is okay in a recovery scenario when you are copying from a drive as the disconnect scenario won't apply. Always copy from USB to SATA, as writing to USB is usually much slower than reading.

Solid State Disks

These are very fast disks, and if you build an array from them, you are best off using raid-1 (mirror). This is a scenario where using the other forms of redundant raid (ie 5 or 6) is actually likely to slow access down!

A new facility (2016) of raid is to use an SSD as a journal. [TODO: Improve my understanding of journalling] This flushes writes to the SSD, partly to speed the system up for the user, and partly to optimise writes to slower rotating media, as it allows changes to accumulate in the journal until a full stripe can be flushed to rotating media in one go.

Serial ATA drives

All modern drives are serial ATA. Not all drives are suitable for RAID, though (at least in their default state). When buying disk drives, make sure they support SCT/ERC, otherwise known as TLER. Most drives of 1TB or less are okay, manufacturers seem to have removed this support from their cheaper drives as capacities went over 1TB.

Other devices

Other devices such as network storage will be added when I or other contributors gain experience of them. Obsolete devices such as parallel ATA will only be mentioned in passing - note that it is not wise to put more than one PATA drive per cable/controller.

Hardware Issues

Selecting a motherboard

For home systems, many motherboards nowadays come with five or six SATA ports. Bearing in mind you will probably want to use one port for a DVD-ROM, you will need at least three ports to have a single mirror, or five for two mirrors or a raid-6. An add-in card to add more SATA ports isn't that expensive.

One thing to watch out for - does the motherboard support legacy IDE? Mine does, and it will only boot from CD-ROM in this mode - which disables the fifth SATA port. However, it's a BIOS board, most boards today will be UEFI.

Desktop and Enterprise drives

Disk drives now tend to come in two varieties, desktop drives from which most of the features needed for a decent raid have been deleted, and enterprise drives, which have the features but are designed to run 24/7. So if you want to run raid on a desktop system it's rather difficult to find a drive that is suitable.

If you're going to buy desktop drives (raid drives are a bit more expensive, but not much) you MUST read the Timeout Mismatch section.

Other drives

For example, Western Digital make "Green" and "Purple" drives.

Green drives are designed to be eco-friendly, and shut down as much as possible. This is great for a laptop that caches everything in ram until it absolutely has to flush everything to disk, or if you've got an SSD in front of them so they can stay idle for as long as possible, but in normal use they'll cause problems and wear out far faster than they should - linux usage is almost guaranteed to stress them in unsuitable ways.

Read the hdparm man page - "hdparm -J" in particular, to disable the greens from head-parking.

[Todo: document hdparm -S to disable spindown, and add this to the script in "Timeout Mismatch"]

Purple drives are meant for CCTV systems. Again, it's highly unlikely their default configuration is linux-friendly, but I know nothing about them.


TLER (Time Limited Error Recovery) is a WD creation, which means that drives will return within 7 seconds. Having introduced it, WD subsequently disabled it on most desktop drives, although it is enabled by default on enterprise drives.

SCT/ERC is the generic specification implemented by TLER.

If it's available this feature needs to be enabled. If it isn't enabled or available, the linux defaults will interact badly with the drive, and a single drive failure will usually take down the array.

smartctl -x

This command will tell you what the drive is capable of. If possible, it would be wise to see the output of it on the drive(s) you are thinking of buying. The following is the output from my laptop's Toshiba drive. Note especially where it says SCT Error Recovery Control is supported.

crappit:/home/anthony # smartctl -x /dev/sda
smartctl 6.2 2013-11-07 r3856 [x86_64-linux-4.1.27-27-default] (SUSE RPM)
Copyright (C) 2002-13, Bruce Allen, Christian Franke,

Device Model:     ST2000LM003 HN-M201RAD
Serial Number:    S321J9DG805231
LU WWN Device Id: 5 0004cf 2106b38eb
Firmware Version: 2BC10001
User Capacity:    2,000,398,934,016 bytes [2.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    5400 rpm
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Tue Sep 20 00:05:59 2016 BST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
AAM feature is:   Disabled
APM feature is:   Disabled
Rd look-ahead is: Enabled
Write cache is:   Enabled
ATA Security is:  Disabled, NOT FROZEN [SEC1]
Wt Cache Reorder: Enabled

SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever 
                                        been run.
Total time to complete Offline 
data collection:                (22740) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine 
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 379) minutes.
SCT capabilities:              (0x003f) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.
Personal tools