Snapchat doesn't include an option to turn off Quick Add completely, but you can give it less to work with by revoking its access to your contacts. On an iPhone, go to Settings > Snapchat and turn off Contacts. On an Android device, go to Settings > Apps > Snapchat > Permissions and turn it off there.