Yes. The person you've deleted won't know you've done so because your number will still be in their contact list. The removed contact might still phone you and send you messages. There is one difference: you will see their phone number rather than their name. But that doesn’t mean that WhatsApp contact cannot be deleted permanently; block the contact if you don't want them to phone or text you.