Forget incoming connections after too much time without a valid incoming message.