We had a 9-disk mix of 3TB and 4TB drives in a storage pool, set for 2 physical disk replication.
We've had single disks fail before, and were able to remove them just fine, both with and without replacing them, provided that there was enough space in the rest of the pool.
We recently had a power...event. The three oldest 3TB drives all failed, and won't even spin up on another machine. Everything else on the server is working fine, physically. However, when we replaced the 3 dead 3TB drives with new 4TB
drives, we were unable to remove the dead drives from the storage pool, either via the UI (click Remove, then the Remove Drive button; Starting Task comes up and makes some progress, then says "Can't remove the drive from the pool. Drive could not
be removed because not all data could be reallocated. Add an additional drive to this pool and reattempt this operation." Given that we've already added 3 4TB drives to the pool, that are empty, this is obviously false.
So we resorted to PowerShell. Assigning the three dead disks to $missingDisks, we then ran "Remove-PhysicalDisk -PhysicalDisks $missingDisks -StoragePoolFriendlyName Replicated" and got "Remove-PhysicalDisk :
One of the physical disks specified could not be removed because it is still in use.". Get-PhysicalDisk shows all three dead drives as "Lost Communication" and "Retired", with everything else showing as "OK"
and "Auto-Select". Tried assigning each of the dead disks individually, then removing, with the same result.
We know that we've probably lost a fair chunk of the data on those three disks; specifically, any data that was only replicated across two of the dead drives. However, there is going to be a fair chunk of data that was replicated to one of the still-live
drives, and the pool repair won't run (just starts and immediately stops within a few microseconds) until these dead drives are removed, and there's a ton of data not on any of the dead drives that's still inaccessible because the pool won't let us access
it at all until it's repaired.
Current state:
PS C:\Windows\system32> Get-StoragePool
FriendlyName OperationalStatus HealthStatus IsPrimordial IsReadOnly
------------ ----------------- ------------ ------------ ----------
Replicated Degraded Warning False False
Primordial OK Healthy True False
PS C:\Windows\system32> Get-VirtualDisk
FriendlyName ResiliencySettingNa OperationalStatus HealthStatus IsManualAttach Size
me
------------ ------------------- ----------------- ------------ -------------- ----
Software Mirror Detached Unhealthy False 50 TB
Business Mirror Detached Unhealthy False 50 TB
Music Mirror Detached Unhealthy False 50 TB
Books Mirror Detached Unhealthy False 50 TB
Photos Mirror Detached Unhealthy False 50 TB
Videos Mirror Detached Unhealthy False 50 TB
Backups Mirror Detached Unhealthy False 50 TB
TV and Movies Mirror Detached Unhealthy False 50 TB
Web Mirror Detached Unhealthy False 50 TB
Users Mirror Detached Unhealthy False 50 TB
PS C:\Windows\system32> Get-PhysicalDisk
FriendlyName CanPool OperationalStatus HealthStatus Usage Size
------------ ------- ----------------- ------------ ----- ----
POOL 4M USB 2.3 False OK Healthy Auto-Select 3.64 TB
POOL4J USB1.4 False OK Healthy Auto-Select 3.64 TB
POOL4I USB1.3 False OK Healthy Auto-Select 3.64 TB
POOL3D False Lost Communication Warning Retired 2.73 TB
PhysicalDisk0 False OK Healthy Auto-Select 298.09 GB
POOL4K USB2.1 False OK Healthy Auto-Select 3.64 TB
POOL 3B False Lost Communication Warning Retired 2.73 TB
POOL4H USB1.2 Z3... False OK Healthy Auto-Select 3.64 TB
POOL4E False OK Healthy Auto-Select 3.64 TB
POOL4G USB1.1 False OK Healthy Auto-Select 3.64 TB
POOL4F False OK Healthy Auto-Select 3.64 TB
POOL4L USB 2.2 False OK Healthy Auto-Select 3.64 TB
POOL3C False Lost Communication Warning Retired 2.73 TB
Manage Storage Spaces - Storage Spaces disk utilization:
Books 515GB, Videos 409GB, Software 218GB, Users 176GB, Music 142GB, Photos 177GB, Web 133GB, TV and Movies 9.13TB, Backups 6.33TB, Business 220GB. All say "Error - Inaccessible; reconnect drives".
Manage Storage Spaces - Physical Drives usage:
POOL 3B 71.5% of 2.72TB (retired), POOL3C 71.5% of 2.72TB (retired), POOL3D 71.5% of 2.72TB (retired), POOL4E 71.0% of 3.63 (OK), POOL4F 71.0% of 3.63TB (OK), POOL4G USB1.1 71.0% of 3.63TB (OK), POOL4H USB1.2 Z302Y837 70.9% of 3.63TB (OK), POOL4I USB1.3
19.3% of 3.63TB (OK), POOL4J USB1.4 19.2% of 3.63TB (OK), POOL4K USB2.1 0.00% of 3.63TB (OK), POOL4L USB2.2 0.00% of 3.63TB (OK), POOL4M USB2.3 0.00% of 3.63TB (OK). The "USB" drives are in 4-drive USB 3.0 external units. The last three
drives (with 0% used) are the new ones.
How do we remove the dead drives, so we can repair the pool, given that all of the standard ways of removing dead drives are failing?