Remember seen messages - store latest seen timestamps per buffer