Attached events to $(document) always, instead of $(windows) for some browsers. Removed the browser detection and switched the event.button to event.which. Thanks to estmatic for the advice!