I couldn't think of any simple trick here to only allow one connection check per frame.

As it is now if you have no gamepads connected....  every second all 4 game pads are checked on the same frame.  Ideally the work would be more evenly spread out.

That said it is still tremendously better than with no throttling at all.