Spec says max speed is 2MHz
Use faster SPI lib
Do a digital read

Sometimes the scrolling interrupt obscures the falling edge of the
motion pin. This slows down the loop a tiny bit if the mouse is not
moving, but that’s better than getting the mouse stuck.