Free cache directory buffer before reusing it