Repopulate now when we are doing idle timer checks.

Otherwise in a weird scenario the idle timer could
be started with a more recent time then the time we
test against causing a connection to disconnect
before its idle timer was over.