mdadm 命令，允许创建和操作 RAID 阵列，以及将其集成到系统的其余部分（包括监控系统）的脚本和工具。
sdb 磁盘，4 GB，完全可用;
sdc 磁盘，4 GB，也是完全可用;
sdd 磁盘，只有分区 sdd2（大约4 GB）可用;
sde 磁盘，4 GB，完全可用。
#mdadm --create /dev/md0 --level=0 --raid-devices=2 /dev/sdb /dev/sdcmdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md0 started. #mdadm --query /dev/md0/dev/md0: 7.99GiB raid0 2 devices, 0 spares. Use mdadm --detail for more detail. #mdadm --detail /dev/md0/dev/md0: Version : 1.2 Creation Time : Mon Feb 28 01:54:24 2022 Raid Level : raid0 Array Size : 8378368 (7.99 GiB 8.58 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Mon Feb 28 01:54:24 2022 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Layout : -unknown- Chunk Size : 512K Consistency Policy : none Name : debian:0 (local to host debian) UUID : a75ac628:b384c441:157137ac:c04cd98c Events : 0 Number Major Minor RaidDevice State 0 8 0 0 active sync /dev/sdb 1 8 16 1 active sync /dev/sdc #mkfs.ext4 /dev/md0mke2fs 1.46.2 (28-Feb-2021) Discarding device blocks: done Creating filesystem with 2094592 4k blocks and 524288 inodes Filesystem UUID: ef077204-c477-4430-bf01-52288237bea0 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done #mkdir /srv/raid-0#mount /dev/md0 /srv/raid-0#df -h /srv/raid-0Filesystem Size Used Avail Use% Mounted on /dev/md0 7.8G 24K 7.4G 1% /srv/raid-0
mdadm --create 命令需要多个参数：要创建卷的名称（/dev/md*，MD 代表 Multiple Device），RAID 级别，磁盘数量（尽管只在 RAID-1 及以上级别时才有意义），以及要使用的物理驱动器。设备创建后，可以像使用普通分区一样，创建文件系统、挂载文件系统等。请注意，我们创建 RAID-0 卷到 md0 只是巧合，阵列的编号不需要与所选的冗余量相关。还可以使用 mdadm 参数如 /dev/md/linear 代替 /dev/md0 来命名 RAID 阵列。
#mdadm --create /dev/md1 --level=1 --raid-devices=2 /dev/sdd2 /dev/sdemdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90 mdadm: largest drive (/dev/sdc2) exceeds size (4189184K) by more than 1% Continue creating array?ymdadm: Defaulting to version 1.2 metadata mdadm: array /dev/md1 started. #mdadm --query /dev/md1/dev/md1: 4.00GiB raid1 2 devices, 0 spares. Use mdadm --detail for more detail. #mdadm --detail /dev/md1/dev/md1: Version : 1.2 Creation Time : Mon Feb 28 02:07:48 2022 Raid Level : raid1 Array Size : 4189184 (4.00 GiB 4.29 GB) Used Dev Size : 4189184 (4.00 GiB 4.29 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Mon Feb 28 02:08:09 2022 State : clean, resync Active Devices : 2 Working Devices : 2 Failed Devices : 0 Spare Devices : 0 Consistency Policy : resync Rebuild Status : 13% complete Name : debian:1 (local to host debian) UUID : 2dfb7fd5:e09e0527:0b5a905a:8334adb8 Events : 17 Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdd2 1 8 48 1 active sync /dev/sde #mdadm --detail /dev/md1/dev/md1: [...] State : clean [...]
mdadm 提示物理磁盘具有不同的大小；这意味着在较大的磁盘上会丢失一些空间，因此需要确认。
/dev/md1 也可用，并且可以在其上创建文件系统以及复制的一些数据。
mdadm 的 --fail 选项，可以模拟这样的磁盘故障：
#mdadm /dev/md1 --fail /dev/sdemdadm: set /dev/sde faulty in /dev/md1 #mdadm --detail /dev/md1/dev/md1: Version : 1.2 Creation Time : Mon Feb 28 02:07:48 2022 Raid Level : raid1 Array Size : 4189184 (4.00 GiB 4.29 GB) Used Dev Size : 4189184 (4.00 GiB 4.29 GB) Raid Devices : 2 Total Devices : 2 Persistence : Superblock is persistent Update Time : Mon Feb 28 02:15:34 2022 State : clean, degraded Active Devices : 1 Working Devices : 1 Failed Devices : 1 Spare Devices : 0 Consistency Policy : resync Name : debian:1 (local to host debian) UUID : 2dfb7fd5:e09e0527:0b5a905a:8334adb8 Events : 19 Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdd2 - 0 0 1 removed 1 8 48 - faulty /dev/sde
sdd 磁盘发生故障，数据将丢失。我们希望避免这种风险，因此我们将用新的磁盘 sdf 替换发生故障的磁盘：
#mdadm /dev/md1 --add /dev/sdfmdadm: added /dev/sdf #mdadm --detail /dev/md1/dev/md1: Version : 1.2 Creation Time : Mon Feb 28 02:07:48 2022 Raid Level : raid1 Array Size : 4189184 (4.00 GiB 4.29 GB) Used Dev Size : 4189184 (4.00 GiB 4.29 GB) Raid Devices : 2 Total Devices : 3 Persistence : Superblock is persistent Update Time : Mon Feb 28 02:25:34 2022 State : clean, degraded, recovering Active Devices : 1 Working Devices : 2 Failed Devices : 1 Spare Devices : 1 Consistency Policy : resync Rebuild Status : 47% complete Name : debian:1 (local to host debian) UUID : 2dfb7fd5:e09e0527:0b5a905a:8334adb8 Events : 39 Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdd2 2 8 64 1 spare rebuilding /dev/sdf 1 8 48 - faulty /dev/sde #[...][...] #mdadm --detail /dev/md1/dev/md1: Version : 1.2 Creation Time : Mon Feb 28 02:07:48 2022 Raid Level : raid1 Array Size : 4189184 (4.00 GiB 4.29 GB) Used Dev Size : 4189184 (4.00 GiB 4.29 GB) Raid Devices : 2 Total Devices : 3 Persistence : Superblock is persistent Update Time : Mon Feb 28 02:25:34 2022 State : clean Active Devices : 2 Working Devices : 2 Failed Devices : 1 Spare Devices : 0 Consistency Policy : resync Name : debian:1 (local to host debian) UUID : 2dfb7fd5:e09e0527:0b5a905a:8334adb8 Events : 41 Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdd2 2 8 64 1 active sync /dev/sdf 1 8 48 - faulty /dev/sde
sde磁盘即将从阵列中删除，以便最终在两个磁盘上使用经典 RAID 镜像：
#mdadm /dev/md1 --remove /dev/sdemdadm: hot removed /dev/sde from /dev/md1 #mdadm --detail /dev/md1/dev/md1: [...] Number Major Minor RaidDevice State 0 8 34 0 active sync /dev/sdd2 2 8 64 1 active sync /dev/sdf
sde disk failure had been real (instead of simulated) and the system had been restarted without removing this sde disk, this disk could start working again due to having been probed during the reboot. The kernel would then have three physical elements, each claiming to contain half of the same RAID volume. In reality this leads to the RAID starting from the individual disks alternately - distributing the data also alternately, depending on which disk started the RAID in degraded mode Another source of confusion can come when RAID volumes from two servers are consolidated onto one server only. If these arrays were running normally before the disks were moved, the kernel would be able to detect and reassemble the pairs properly; but if the moved disks had been aggregated into an md1 on the old server, and the new server already has an md1, one of the mirrors would be renamed.
/etc/mdadm/mdadm.conf文件，下面列出了一个示例：
例 12.1. mdadm 配置文件
# mdadm.conf # # !NB! Run update-initramfs -u after updating this file. # !NB! This will ensure that initramfs has an uptodate copy. # # Please refer to mdadm.conf(5) for information about this file. # # by default (built-in), scan all partitions (/proc/partitions) and all # containers for MD superblocks. alternatively, specify devices to scan, using # wildcards if desired. DEVICE /dev/sd* # automatically tag new arrays as belonging to the local system HOMEHOST <system> # instruct the monitoring daemon where to send mail alerts MAILADDR root # definitions of existing MD arrays ARRAY /dev/md/0 metadata=1.2 UUID=a75ac628:b384c441:157137ac:c04cd98c name=debian:0 ARRAY /dev/md/1 metadata=1.2 UUID=2dfb7fd5:e09e0527:0b5a905a:8334adb8 name=debian:1 # This configuration was auto-generated on Mon, 28 Feb 2022 01:53:48 +0100 by mkconf
DEVICE 选项，它列出了系统将在启动时自动查找 RAID 卷组件的设备。在我们的示例中，我们用设备文件的显式列表替换了默认值partitions containers，因为我们选择对某些卷使用整个磁盘，而不是仅使用分区。
/dev/md*名称）。
#mdadm --misc --detail --brief /dev/md?ARRAY /dev/md/0 metadata=1.2 UUID=a75ac628:b384c441:157137ac:c04cd98c name=debian:0 ARRAY /dev/md/1 metadata=1.2 UUID=2dfb7fd5:e09e0527:0b5a905a:8334adb8 name=debian:1
/dev 中的设备文件中，因此没有直接使用它们的风险。
/dev 中条目的块设备，它可用作任何其他物理分区（通常用于托管文件系统或交换空间）。
sdb 磁盘，sdb2 分区，4 GB;
sdc 磁盘，sdc3 分区，3 GB;
sdd 磁盘，4 GB，完全可用;
sdf 磁盘，sdf1 分区，4 GB; 以及 sdf2 分区，5 GB。
sdb 和 sdf 色速度比另外两个更快。
pvcreate 准备物理卷：
#pvcreate /dev/sdb2Physical volume "/dev/sdb2" successfully created. #pvdisplay"/dev/sdb2" is a new physical volume of "4.00 GiB" --- NEW Physical volume --- PV Name /dev/sdb2 VG Name PV Size 4.00 GiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID yK0K6K-clbc-wt6e-qk9o-aUh9-oQqC-k1T71B #for i in sdc3 sdd sdf1 sdf2 ; do pvcreate /dev/$i ; donePhysical volume "/dev/sdc3" successfully created. Physical volume "/dev/sdd" successfully created. Physical volume "/dev/sdf1" successfully created. Physical volume "/dev/sdf2" successfully created. #pvdisplay -CPV VG Fmt Attr PSize PFree /dev/sdb2 lvm2 --- 4.00g 4.00g /dev/sdc3 lvm2 --- 3.00g 3.00g /dev/sdd lvm2 --- 4.00g 4.00g /dev/sdf1 lvm2 --- 4.00g 4.00g /dev/sdf2 lvm2 --- 5.00g 5.00g
pvdisplay列出了现有的PV，有两种输出格式。
vgcreate 将这些物理元素组合。将快速磁盘放到一个VG vg_critical；其他 VG vg_normal 包含较慢的元素。
#vgcreate vg_critical /dev/sdb2 /dev/sdf1Volume group "vg_critical" successfully created #vgdisplay--- Volume group --- VG Name vg_critical System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 7.99 GiB PE Size 4.00 MiB Total PE 2046 Alloc PE / Size 0 / 0 Free PE / Size 2046 / 7.99 GiB VG UUID JgFWU3-emKg-9QA1-stPj-FkGX-mGFb-4kzy1G #vgcreate vg_normal /dev/sdc3 /dev/sdd /dev/sdf2Volume group "vg_normal" successfully created #vgdisplay -CVG #PV #LV #SN Attr VSize VFree vg_critical 2 0 0 wz--n- 7.99g 7.99g vg_normal 3 0 0 wz--n- <11.99g <11.99g
vgdisplay 有两种输出格式）。请注意，将同一物理磁盘的两个分区用于两个不同的 VG 是有可能的。 请注意，我们使用 vg_ 前缀来命名我们的 VG，但它只不过是一个约定。
lvcreate命令，以及稍微复杂的语法：
#lvdisplay#lvcreate -n lv_files -L 5G vg_criticalLogical volume "lv_files" created. #lvdisplay--- Logical volume --- LV Path /dev/vg_critical/lv_files LV Name lv_files VG Name vg_critical LV UUID Nr62xe-Zu7d-0u3z-Yyyp-7Cj1-Ej2t-gw04Xd LV Write Access read/write LV Creation host, time debian, 2022-03-01 00:17:46 +0100 LV Status available # open 0 LV Size 5.00 GiB Current LE 1280 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 #lvcreate -n lv_base -L 1G vg_criticalLogical volume "lv_base" created. #lvcreate -n lv_backups -L 11.98G vg_normalRounding up size to full physical extent 11.98 GiB Rounding up size to full physical extent 11.98 GiB Logical volume "lv_backups" created. #lvdisplay -CLV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv_base vg_critical -wi-a----- 1.00g lv_files vg_critical -wi-a----- 5.00g lv_backups vg_normal -wi-a----- 11.98g
lvcreate 作为选项。使用 -n 选项指定要创建的 LV 的名称，通常使用 -L 选项指定其大小。当然，还需要告诉命令要对哪个 VG 进行操作，作为命令行上的最后一个参数。
/dev/mapper/：
#ls -l /dev/mappertotal 0 crw------- 1 root root 10, 236 Mar 1 00:17 control lrwxrwxrwx 1 root root 7 Mar 1 00:19 vg_critical-lv_base -> ../dm-1 lrwxrwxrwx 1 root root 7 Mar 1 00:17 vg_critical-lv_files -> ../dm-0 lrwxrwxrwx 1 root root 7 Mar 1 00:19 vg_normal-lv_backups -> ../dm-2 #ls -l /dev/dm-*brw-rw---- 1 root disk 253, 0 Mar 1 00:17 /dev/dm-0 brw-rw---- 1 root disk 253, 1 Mar 1 00:19 /dev/dm-1 brw-rw---- 1 root disk 253, 2 Mar 1 00:19 /dev/dm-2
#ls -l /dev/vg_criticaltotal 0 lrwxrwxrwx 1 root root 7 Mar 1 00:19 lv_base -> ../dm-1 lrwxrwxrwx 1 root root 7 Mar 1 00:17 lv_files -> ../dm-0 #ls -l /dev/vg_normaltotal 0 lrwxrwxrwx 1 root root 7 Mar 1 00:19 lv_backups -> ../dm-2
#mkfs.ext4 /dev/vg_normal/lv_backupsmke2fs 1.46.2 (28-Feb-2021) Discarding device blocks: done Creating filesystem with 3140608 4k blocks and 786432 inodes Filesystem UUID: 7eaf0340-b740-421e-96b2-942cdbf29cb3 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208 Allocating group tables: done Writing inode tables: done Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done #mkdir /srv/backups#mount /dev/vg_normal/lv_backups /srv/backups#df -h /srv/backupsFilesystem Size Used Avail Use% Mounted on /dev/mapper/vg_normal-lv_backups 12G 24K 12G 1% /srv/backups #[...][...] #cat /etc/fstab[...] /dev/vg_critical/lv_base /srv/base ext4 defaults 0 2 /dev/vg_critical/lv_files /srv/files ext4 defaults 0 2 /dev/vg_normal/lv_backups /srv/backups ext4 defaults 0 2
vg_critical 的可用空间，我们可以扩展 lv_files。为此，将使用命令 lvresize，然后使用 resize2fs 相应地调整文件系统：
#df -h /srv/files/Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_critical-lv_files 4.9G 4.2G 485M 90% /srv/files #lvdisplay -C vg_critical/lv_filesLV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv_files vg_critical -wi-ao---- 5.00g #vgdisplay -C vg_criticalVG #PV #LV #SN Attr VSize VFree vg_critical 2 2 0 wz--n- 7.99g 1.99g #lvresize -L 6G vg_critical/lv_filesSize of logical volume vg_critical/lv_files changed from 5.00 GiB (1280 extents) to 6.00 GiB (1536 extents). Logical volume vg_critical/lv_files successfully resized. #lvdisplay -C vg_critical/lv_filesLV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert lv_files vg_critical -wi-ao---- 6.00g #resize2fs /dev/vg_critical/lv_filesresize2fs 1.46.2 (28-Feb-2021) Filesystem at /dev/vg_critical/lv_files is mounted on /srv/files; on-line resizing required old_desc_blocks = 1, new_desc_blocks = 1 The filesystem on /dev/vg_critical/lv_files is now 1572864 (4k) blocks long. #df -h /srv/files/Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_critical-lv_files 5.9G 4.2G 1.5G 75% /srv/files
#df -h /srv/base/Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_critical-lv_base 974M 883M 25M 98% /srv/base #vgdisplay -C vg_criticalVG #PV #LV #SN Attr VSize VFree vg_critical 2 2 0 wz--n- 7.99g 1016.00m
sdb3 partition, which was so far used outside of LVM, only contained archives that could be moved to lv_backups. We can now recycle it and integrate it to the volume group, and thereby reclaim some available space. This is the purpose of the vgextend command. Of course, the partition must be prepared as a physical volume beforehand. Once the VG has been extended, we can use similar commands as previously to grow the logical volume then the filesystem:
#pvcreate /dev/sdb3Physical volume "/dev/sdb3" successfully created. #vgextend vg_critical /dev/sdb3Volume group "vg_critical" successfully extended #vgdisplay -C vg_criticalVG #PV #LV #SN Attr VSize VFree vg_critical 3 2 0 wz--n- <12.99g <5.99g #lvresize -L 2G vg_critical/lv_base[...] #resize2fs /dev/vg_critical/lv_base[...] #df -h /srv/base/Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_critical-lv_base 2.0G 886M 991M 48% /srv/base
sda 和 sdc。它们沿以下方案进行分区：
#sfdisk -l /dev/sdaDisk /dev/sda: 894.25 GiB, 960197124096 bytes, 1875385008 sectors Disk model: SAMSUNG MZ7LM960 Units: sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disklabel type: gpt Disk identifier: BB14C130-9E9A-9A44-9462-6226349CA012 Device Start End Sectors Size Type /dev/sda1 2048 4095 2048 1M BIOS boot /dev/sda2 4096 100667391 100663296 48G Linux RAID /dev/sda3 100667392 134221823 33554432 16G Linux RAID /dev/sda4 134221824 763367423 629145600 300G Linux RAID /dev/sda5 763367424 1392513023 629145600 300G Linux RAID /dev/sda6 1392513024 1875384974 482871951 230.3G Linux LVM
sda2 and sdc2 (about 48 GB) are assembled into a RAID-1 volume, md0. This mirror is directly used to store the root filesystem.
sda3 and sdc3 partitions are assembled into a RAID-0 volume, md1, and used as swap partition, providing a total 32 GB of swap space. Modern systems can provide plenty of RAM and our system won't need hibernation. So with this amount added, our system will unlikely run out of memory.
sda4 and sdc4 partitions, as well as sda5 and sdc5, are assembled into two new RAID-1 volumes of about 300 GB each, md2 and md3. Both these mirrors are initialized as physical volumes for LVM, and assigned to the vg_raid volume group. This VG thus contains about 600 GB of safe space.
sda6 and sdc6, are directly used as physical volumes, and assigned to another VG called vg_bulk, which therefore ends up with roughly 460 GB of space.
vg_raid 中创建的 LV 都会保留，而在 vg_bulk 中创建的 LV 则不会保留；另一方面，后者将并行分配给两个磁盘，这允许大文件有更高的读取或写入速度。
vg_raid 中创建 lv_var 和 lv_home，以存放对应的文件系统；另一个大型LV lv_movies，将用于存放编辑后的视频。另一个 VG 将拆分为 lv_rushes，用于直接保存数字摄像机中输出的数据，lv_tmp 存放临时文件。工作区的位置是一个不太简单的选择：虽然该卷需要良好的性能，但如果磁盘在编辑会话期间发生故障，是否值得冒失去工作的风险？根据该问题的答案，相关 LV 将在一个 VG 或另一个 VG 上创建。