Hi All,
So have been playing with the lovely new Storage Spaces and De-duplication features in Server 2012 and have come across some strange behaviour. The test volume in question contains just over 3.2TB of data, stored as a mirrored Virtual Disk on a Storage Pool consisting of 4x 2TB sata drives. The data is a mixture of user home folders containing anything and everything you can imagine.
I enabled the de-duplication feature on the volume about 3 days ago, manually set the file age to 0 and then triggered a new optimization job using powershell. Whilst the job was running I kept check on the process using the Get-DedupJob and Get-DedupStatus powershell commands. The job is still running now and has taken about 3 days so far. (The server is low on ram and this has caused some lovely paging issues, hence the massive run time.)
During the first 0-50% or so the savings reported kept climbing and gradually ended up as high as 1.4TB which is about what I was expecting (lots of duplicate audio/video files and documents)
BUT - somewhere around 50-70% the savings have completely plummeted back to zero and started growing again. At 86% it now lists only 129GB saved. Its almost as if it has forgotten about the earlier savings it had made, even though it still lists almost 100% of files optimized.
PS C:\Windows\system32> Get-DedupStatus FreeSpace SavedSpace OptimizedFiles InPolicyFiles --------- ---------- -------------- ------------- 7.06 TB 129.45 GB 114973 116706
Can anyone explain this behaviour? Was it simply a rounding error and should of displayed GB instead of TB? (I highly doubt this) Is there something else going on that I should know off?
My next step will be to un-optimize the data and then optimize it again, setting up a script to log the output of get-dedupstatus every hour and see if I can log and reproduce the same issue.
I have had similar results with other smaller data sets where the savings climb up very well to around the 50% mark, then seemingly disappear!