View source for Recovering a failed software RAID

== Setting the scene ==

This article will deal with the following case. It starts out as a perfect RAID6 (state 1):

  md0 : active raid6 sdn1[6](S) sdm1[5] sdk1[3] sdj1[2] sdh1[1] sdg1[0]
      305664 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/5] [UUUUU]

For some unknown reason /dev/sdk1 fails and rebuild starts on the spare /dev/sdn1 (state 2):

  md0 : active raid6 sdn1[6] sdm1[5] sdk1[3](F) sdj1[2] sdh1[1] sdg1[0]
      305664 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/4] [UUU_U]
      [===>.................]  recovery = 16.0% (16744/101888) finish=1.7min speed=797K/sec

During the rebuild /dev/sdg1 fails, too. Now all redundancy is lost, and losing another data disk will fail the RAID. The rebuild on /dev/sdn1 continues (state 3):

  md0 : active raid6 sdn1[6] sdm1[5] sdk1[3](F) sdj1[2] sdh1[1] sdg1[0](F)
      305664 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/3] [_UU_U]
      [===========>.........]  recovery = 59.0% (60900/101888) finish=0.6min speed=1018K/sec

Before the rebuild finishes, yet another data harddisk (/dev/sdh1) fails, thus failing the RAID. The rebuild on /dev/sdn1 cannot continue, so /dev/sdn1 reverts to its status as spare (state 4):

  md0 : active raid6 sdn1[6](S) sdm1[5] sdk1[3](F) sdj1[2] sdh1[1](F) sdg1[0](F)
      305664 blocks super 1.2 level 6, 512k chunk, algorithm 2 [5/2] [__U_U]

This is the situation we are going to recover from. The goal is to get back to state 3 with minimal data loss.