A scammer with your phone number could commit phone account takeover fraud or use it to trick you into sharing two-factor authentication codes so they can take over other accounts.