Remove timestamp entries if the timeouts have already elapsed