[#414] Improve the tracking of ancilliary buffers