Exploiting Novel GPT-4 APIs
Authors: name: Kellin Pelrine; name: Mohammad Taufeeque; name: Michał Zając; name: Euan McLean; name: Adam Gleave
Published: 2023-12-21 21:22:41

Abstract: Language model attacks typically assume one of two extreme threat models:
full white-box access to model weights, or black-box access limited to a text
generation API. However, real-world APIs are often more flexible than just text
generation: these APIs expose "gray-box" access leading to new threat vectors.
To explore this, we red-team three new functionalities exposed in the GPT-4
APIs: fine-tuning, function calling and knowledge retrieval. We find that
fine-tuning a model on as few as 15 harmful examples or 100 benign examples can
remove core safeguards from GPT-4, enabling a range of harmful outputs.
Furthermore, we find that GPT-4 Assistants readily divulge the function call
schema and can be made to execute arbitrary function calls. Finally, we find
that knowledge retrieval can be hijacked by injecting instructions into
retrieval documents. These vulnerabilities highlight that any additions to the
functionality exposed by an API can create new vulnerabilities.