improved guards against extremely unlikely packets

This is a small fix for handling stats around extremely unlikely packets. 

We noticed some odd behavior related to this while debugging the time stamp issue.