Cyberax/mdtrim — Issue #1: High memory use when trimming a large RAID1 partition
Author: jukkaho
Created: 2011-12-31 23:46:29+00:00
State: open

Memory use gets really high when trimming RAID with two SSD drives and a lot of free space. Python also crashed with some internal double free errors when trying to trim using a file as large as 40 GB. I have successfully tested the program up to 30 GB temporary file sizes and it needed about 4 GB of RAM.

I was able to work around this by removing temporary file deletion from the code and using several ~10 GB files sequentially until I run out of disk space. Then I just removed all the trimmed temporary files at the end. This seemed to restore my write speeds to original performance level.

Additional benefit of using several smaller files is that the disk is almost full for shorter amount of time as checking of the last piece takes less time now.