[fix] track our own time so that we dont hit weird edge cases