The css `pointer-events` [1] will be your friend! You should be able to remove this hack with one or two `pointer-events: none` in the css.

[1] https://developer.mozilla.org/en-US/docs/CSS/pointer-events