Do the debounce the right way, it actually works now

Some other cleanup as well