Keep track of response times and outstanding pings.