Monitoring your system

From Linux Raid Wiki
(Difference between revisions)
Jump to: navigation, search
m (update backwards)
m (Add smartctl section)
Line 27: Line 27:
Don't rely on this! Check regularly on a manual basis!
Don't rely on this! Check regularly on a manual basis!
== smartctl ==
== Analysing a Disk Failure ==
== Analysing a Disk Failure ==

Revision as of 23:18, 12 October 2017

Back to Scrubbing the drives

Many of the horror stories that come to the linux raid mailing list are down to a simple lack of monitoring. Okay, it's not unknown for several disks to fail simultaneously, and if your raid array consists of a bunch of drives all bought at the same time, for the array, the odds of that happening are painfully high - batches of disks tend to have similar lifetimes.

But all too often, an array has been running in a degraded state for months, and then a disk fails and tips the array over the edge. The author's brother told him of a raid array, bought and placed in a colo facility, where a technician just happened to walk past and spot two red lights! The raid 6 array had two failed drives! This should never have happened, and of course there was a mad panic while they tried to safely replace the dead drives.


Monitoring Tools


You should get to know /proc/mdstat, looking at it often. This will tell you the state of your arrays, and very importantly it will tell you whether any drives have failed, and whether any arrays are degraded. Check, and check regularly!


xosview is a venerable utility, and one of the author's favourites. It is capable of displaying the state of raid arrays, but unfortunately currently the code is broken - it reads mdstat, and doesn't understand the current output. It is currently (2016) being updated to read the status directly from /sys, and should hopefully soon be able to display raid status correctly. The author leaves xosview running permanently on his desktop to provide an overview of system performance.


mdadm --monitor --scan --mail

This will fire up mdadm to keep an eye on your arrays. It will daemonize and run in the background, sending an email to the specified address if it detects any problems related to a disk failure. This is good for remote monitoring BUT. It won't tell you if anything goes wrong with the monitoring! You cannot assume - even if you put this in your boot-up sequence as you should - that you will be notified about important events. It's not unknown for the daemon to fail.

Don't rely on this! Check regularly on a manual basis!


Analysing a Disk Failure

Back to Scrubbing the drives
Personal tools