No. You can block a contact without adding them to your contacts. All you have to do is: Tap the i button next to the number → select Block this Caller.