Ensure previous buttons get removed from DOM

I don’t think this causes a memory leak?
2.2.1