Allow contact numbers to be destroyed

needed after introducing strong params