You can remove your Twitter Contacts if you no longer wish to have them there. To do this, go to privacy and safety, and tap remove contacts under discoverability and contacts.