Yes, you can deactivate the Follow button on your personal profile at any time. By adjusting your privacy settings accordingly, you can limit your audience to your friends only.