[compositor] Retain one buffer, also when compositor is hidden