Properly send events to interacts when mouse is pressed and button is pressed