Recently we needed to transfer an aggregate that was hosted on an external shelf to disks housed internally in our test FAS2020 and the same trick can be used to move an aggregate from one shelf to another.
The trick uses the disk-replace command. It works serially, and so it is a little time consuming, BUT it retains exactly the same layout on disk, since it’s a bit-for-bit copy.
In short the command to use is ‘disk replace start e.g.
disk replace start 0b.19 0c.00.0
So here’s what my disk layout looks like – I am using traditional volumes
Volume vol1 (online, raid_dp) (block checksums) Plex /vol1/plex0 (online, normal, active) RAID group /vol1/plex0/rg0 (normal)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0c.00.6 0c 0 6 SA:1 - SATA 7200 211377/432901760 211977/434130816
parity 0c.00.1 0c 0 1 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.19 0b 1 3 FC:B - ATA 7200 211377/432901760 211921/434014304
data 0c.00.3 0c 0 3 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.11 0c 0 11 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.10 0c 0 10 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.9 0c 0 9 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.8 0c 0 8 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.7 0c 0 7 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.4 0c 0 4 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.29 0b 1 13 FC:B - ATA 7200 211377/432901760 211921/434014304
data 0c.00.5 0c 0 5 SA:1 - SATA 7200 211377/432901760 211977/434130816 Spare disks RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare 0a.16 0a 1 0 FC:A - ATA 7200 211377/432901760 211921/434014304(not zeroed)
spare 0a.20 0a 1 4 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.22 0a 1 6 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.24 0a 1 8 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.26 0a 1 10 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0b.17 0b 1 1 FC:B - ATA 7200 211377/432901760 211921/434014304 (not zeroed)
spare 0b.25 0b 1 9 FC:B - ATA 7200 211377/432901760 211921/434014304
spare 0b.27 0b 1 11 FC:B - ATA 7200 211377/432901760 211921/434014304
spare 0c.00.0 0c 0 0 SA:1 - SATA 7200 211377/432901760 211977/434130816 (not zeroed)
All the disks marked with a ‘CHAN’ type of FC:A or FC:B are actually SATA drives on the other side of a FC->SATA bridge, and are basically SATA disks in an external shelf. The disks marked SA:1 are SATA drive internal to the FAS2020. In the output above I am part-way through migrating the disks. Some of the disks in my volume/aggregate are internal and some are external. On of the disks that is still on the external shelf is disk 0b.19. Using the command below I will transfer the data on that individual disk to an internal SATA disk (0c.00.0). The command I use is.
disk replace start 0b.19 0c.00.0
Then I receive a warning messge
*** You are about to copy and replace the following file system disk *** Disk /vol1/plex0/rg0/0b.19
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
data 0b.19 0b 1 3 FC:B - ATA 7200 211377/432901760 211921/434014304
***Really replace disk 0b.19 with 0c.00.0? ydisk replace: Disk 0b.19 was marked for replacing.
The output of sysconfig -r shows something like this
Volume vol1 (online, raid_dp) (block checksums) Plex /vol1/plex0 (online, normal, active) RAID group /vol1/plex0/rg0 (normal)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0c.00.6 0c 0 6 SA:1 - SATA 7200 211377/432901760 211977/434130816
parity 0c.00.1 0c 0 1 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.19 0b 1 3 FC:B - ATA 7200 211377/432901760 211921/434014304 (replacing)
data 0c.00.3 0c 0 3 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.11 0c 0 11 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.10 0c 0 10 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.9 0c 0 9 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.8 0c 0 8 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.7 0c 0 7 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.4 0c 0 4 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.29 0b 1 13 FC:B - ATA 7200 211377/432901760 211921/434014304
data 0c.00.5 0c 0 5 SA:1 - SATA 7200 211377/432901760 211977/434130816
And then once the replacement gets under way.. I see my copy under way.
Volume vol1 (online, raid_dp) (block checksums) Plex /vol1/plex0 (online, normal, active) RAID group /vol1/plex0/rg0 (normal)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0c.00.6 0c 0 6 SA:1 - SATA 7200 211377/432901760 211977/434130816
parity 0c.00.1 0c 0 1 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.19 0b 1 3 FC:B - ATA 7200 211377/432901760 211921/434014304 (replacing, copy in progress) -> copy 0c.00.0 0c 0 0 SA:1 - SATA 7200 211377/432901760 211977/434130816 (copy 1% completed)
data 0c.00.3 0c 0 3 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.11 0c 0 11 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.10 0c 0 10 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.9 0c 0 9 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.8 0c 0 8 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.7 0c 0 7 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.4 0c 0 4 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0b.29 0b 1 13 FC:B - ATA 7200 211377/432901760 211921/434014304
data 0c.00.5 0c 0 5 SA:1 - SATA 7200 211377/432901760 211977/434130816
And finally when the copy is done, 0c.00.0 is part of the aggregate/volume and 0b.19 is spare.
Volume vol1 (online, raid_dp) (block checksums) Plex /vol1/plex0 (online, normal, active) RAID group /vol1/plex0/rg0 (normal)
RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
dparity 0c.00.6 0c 0 6 SA:1 - SATA 7200 211377/432901760 211977/434130816
parity 0c.00.1 0c 0 1 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.0 0c 0 0 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.3 0c 0 3 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.11 0c 0 11 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.10 0c 0 10 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.9 0c 0 9 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.8 0c 0 8 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.7 0c 0 7 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.4 0c 0 4 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.2 0c 0 2 SA:1 - SATA 7200 211377/432901760 211977/434130816
data 0c.00.5 0c 0 5 SA:1 - SATA 7200 211377/432901760 211977/434130816
Spare disks RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block or zoned checksum traditional volumes or aggregates
spare 0a.16 0a 1 0 FC:A - ATA 7200 211377/432901760 211921/434014304 (not zeroed)
spare 0a.20 0a 1 4 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.22 0a 1 6 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.24 0a 1 8 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0a.26 0a 1 10 FC:A - ATA 7200 211377/432901760 211921/434014304
spare 0b.17 0b 1 1 FC:B - ATA 7200 211377/432901760 211921/434014304 (not zeroed)
spare 0b.19 0b 1 3 FC:B - ATA 7200 211377/432901760 211921/434014304 (not zeroed)
spare 0b.25 0b 1 9 FC:B - ATA 7200 211377/432901760 211921/434014304
spare 0b.27 0b 1 11 FC:B - ATA 7200 211377/432901760 211921/434014304
spare 0b.29 0b 1 13 FC:B - ATA 7200 211377/432901760 211921/434014304 (not zeroed)