Yes, you can refuse. You should not be doing anything that you believe can put you or your privacy at risk. You should communicate your concerns to the management and discuss options that will make you feel comfortable while maintaining job-related communication. One such option to avoid using a personal number for work would be a second phone number.