Fix wrong select() call timeout

This caused an EINVAL failure on arm64 as we tried to specify 5M microseconds
in the struct timeval. Correct it to 500.000 microseconds.