No. Your upvotes will remain, and you can also purchase more if you feel like your account could use more growth in the future.