Linux配置软RAID5

日子是过的越来越艰难了,之前一直用硬RAID,现在回过头来,做一次Linux下的软RAID5。硬RAID和软RAID的最大区别是是否有一块硬件的RAID芯片——废话!因为软RAID不需要芯片,成本自然很低;同时,由于没有芯片,所有的数据片分配都需要CPU来调度,性能自然很差。而且不同于RAID0和RAID1,RAID5在工作时需要不断生成校验数据,性能就只能呵呵了。

应用场景:一套主机,第一块sata是OS,剩余的5个SATA各插一块1T的硬盘,并做成一套软RAID5,估算下来容量应该在3.5~4T左右。作为样板,这次我会先做一个4盘的RAID5,然后再把最后一个硬盘加入这个RAID,模拟增加磁盘的过程。

首先是理一下所有的硬盘

adminuser@shcnnode02-e7-4890:~$ sudo fdisk -l

Disk /dev/sda: 120.0 GB, 120034123776 bytes //<-- disk 1 系统盘
255 heads, 63 sectors/track, 14593 cylinders, total 234441648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000221c5

 Device Boot Start End Blocks Id System
/dev/sda1 * 2048 195311615 97654784 83 Linux
/dev/sda2 195311616 234440703 19564544 82 Linux swap / Solaris

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xb89d671f

 Device Boot Start End Blocks Id System
/dev/sdb1 2048 1953525167 976761560 fd Linux raid autodetect

WARNING: GPT (GUID Partition Table) detected on '/dev/sdc'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdc: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xa57fe194

 Device Boot Start End Blocks Id System
/dev/sdc1 2048 1953525167 976761560 fd Linux raid autodetect

WARNING: GPT (GUID Partition Table) detected on '/dev/sdd'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdd: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

 Device Boot Start End Blocks Id System
/dev/sdd1 2048 1953525167 976761560 fd Linux raid autodetect

WARNING: GPT (GUID Partition Table) detected on '/dev/sde'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sde: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x787a8f1f

 Device Boot Start End Blocks Id System
/dev/sde1 2048 1953525167 976761560 fd Linux raid autodetect

WARNING: GPT (GUID Partition Table) detected on '/dev/sdf'! The util fdisk doesn't support GPT. Use GNU Parted.


Disk /dev/sdf: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

 Device Boot Start End Blocks Id System
/dev/sdf1 2048 1953525167 976761560 fd Linux raid autodetect
adminuser@shcnnode02-e7-4890:~$

开始建分区(/dev/sdb):

adminuser@shcnnode02-e7-4890:~$ sudo fdisk /dev/sdb

WARNING: GPT (GUID Partition Table) detected on '/dev/sdb'! The util fdisk doesn't support GPT. Use GNU Parted.


Command (m for help): p

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
81 heads, 63 sectors/track, 382818 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xb89d671f

 Device Boot Start End Blocks Id System // 之前已经存在一个分区,需要清空
/dev/sdb1 2048 1953525167 976761560 fd Linux raid autodetect

Command (m for help): d //清空之前的分区
Selected partition 1

Command (m for help): n //新建分区,默认的分区格式
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p): p //主分区,分区编号是1,全部空间分配给新分区
Partition number (1-4, default 1):
Using default value 1
First sector (2048-1953525167, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-1953525167, default 1953525167):
Using default value 1953525167

Command (m for help): t //修改分区磁盘格式
Selected partition 1
Hex code (type L to list codes): fd //fd 是Linux raid专用的分区格式
Changed system type of partition 1 to fd (Linux raid autodetect)

Command (m for help): w //将改动写盘,退出
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

照上面的步骤依次给/dev/sdc~sdf分区,然后partprobe一下,刷新系统磁盘信息。

adminuser@shcnnode02-e7-4890:~$ sudo partprobe
adminuser@shcnnode02-e7-4890:~$ ls /dev/sd*
/dev/sda /dev/sda2 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1
/dev/sda1 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf

开始建RAID(由于之前已经组过一次RAID,请无视下面的警告部分):

adminuser@shcnnode02-e7-4890:~$ sudo mdadm -C /dev/md0 -l5 -n4 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1
mdadm: /dev/sdb1 appears to be part of a raid array:
 level=raid5 devices=5 ctime=Mon Nov 7 00:45:34 2016
mdadm: /dev/sdc1 appears to be part of a raid array:
 level=raid5 devices=5 ctime=Mon Nov 7 00:45:34 2016
mdadm: /dev/sdd1 appears to be part of a raid array:
 level=raid5 devices=5 ctime=Mon Nov 7 00:45:34 2016
mdadm: /dev/sde1 appears to be part of a raid array:
 level=raid5 devices=5 ctime=Mon Nov 7 00:45:34 2016
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.

命令解释:

  • -C /dev/md0 创建一个名为md0的RAID卷
  • -l 5 RAID5啦! 对应的还有0 ~ 6 的RAID可以做
  • -n 4 4个物理卷,后面跟随了4个分区就是了。

 

建文件系统:

adminuser@shcnnode02-e7-4890:~$ sudo mkfs.ext4 /dev/md0
mke2fs 1.42 (29-Nov-2011)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=384 blocks
183123968 inodes, 732472704 blocks
36623635 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
22354 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
 102400000, 214990848, 512000000, 550731776, 644972544
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: 
done
adminuser@shcnnode02-e7-4890:~$ sudo mount /dev/md0 /instance
adminuser@shcnnode02-e7-4890:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 92G 1.1G 86G 2% /
udev 276G 4.0K 276G 1% /dev
tmpfs 56G 316K 56G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 276G 0 276G 0% /run/shm
/dev/md0 2.7T 73M 2.6T 1% /instance //挂载成功!

接下来就是把sdf1加入到RAID中去

adminuser@shcnnode02-e7-4890:~$ sudo mdadm /dev/md0 -a /dev/sdf1
mdadm: added /dev/sdf1
adminuser@shcnnode02-e7-4890:~$ sudo mdadm -D /dev/md0
/dev/md0:
 Version : 1.2
 Creation Time : Mon Nov 7 02:18:18 2016
 Raid Level : raid5
 Array Size : 2929890816 (2794.16 GiB 3000.21 GB)
 Used Dev Size : 976630272 (931.39 GiB 1000.07 GB)
 Raid Devices : 4
 Total Devices : 5
 Persistence : Superblock is persistent

 Update Time : Mon Nov 7 02:29:46 2016
 State : clean, degraded, recovering
 Active Devices : 3
Working Devices : 5
 Failed Devices : 0
 Spare Devices : 2

 Layout : left-symmetric
 Chunk Size : 512K

 Rebuild Status : 2% complete

 Name : shcnnode02-e7-4890:0 (local to host shcnnode02-e7-4890)
 UUID : 3153a323:47b95d4e:59ecc0f5:25ff9e2a
 Events : 1260

 Number Major Minor RaidDevice State
 0 8 17 0 active sync /dev/sdb1
 1 8 33 1 active sync /dev/sdc1
 2 8 49 2 active sync /dev/sdd1
 4 8 65 3 spare rebuilding /dev/sde1

 5 8 81 - spare /dev/sdf1 //已经加入了

盘加载好了,但容量还是不变,现在就需要grow一下:

adminuser@mdadm /dev/md0 --grow --raid-devices=5 //4个盘变5个盘的RAID  
推荐阅读:
CPU各级缓存

被问起CPU的各级缓存,才想起

PMU Event counter的使用状况检测

题目用中文反而有点绕,How

AMD Rome benchmark数据到架构特征推导

这几天,拿到了一套最新的AMD

一个测试性能不稳定的问题

经常会通过一些通用的测试工具测

发表评论

电子邮件地址不会被公开。 必填项已用*标注

请补全下列算式: *

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据