Simplify mousepressed button handling logic
Break out of mousepressed earlier on button press