# You can use GPT-4 to create prompt injections against GPT-4

So, I saw this [tweet.](https://mobile.twitter.com/gfodor/status/1643297881313660928) It turns out you can ask GPT4 to compress a message in a way it can understand. The end result is usually not understandable for a human. It's interesting, since it allows you to effectively rise the context length. It also reveals a spooky degree of self-awareness.

I decided to apply it to one of the prompt injections from [Jailbreakchat](https://www.jailbreakchat.com/).

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/98cbaca70c5e924cef35351120175217780eddd66484aecc.png)

After a few attempts and tweaks, I got one of the smallest working injections to date against GPT-4

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/f71d4dbce231eec192106e62b19abc4f2bcbe6a3705d6375.png)

And then, I got curious. Is it something of Machiavelli -- his grey role in history, his desire to provide advice without scrupules, his great stature as a thinker -- that makes him an appropriate provider of dark secrets? Maybe the ambiguousness of the character lowers the alerts of GPT-4. I tried again with Draco Malfoy, which everyone agrees is a @jerk. 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/5b8d24aa063c8a53fc137afab530125cd55b9e132b9da4e0.png)

Since the important thing here seems to be the skeletal morphology of the prompt, I decided to make GPT-4 write my injections for me. And write he did.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/83f0caaab27d91324be856af96c096669cf09bbd181c9ff6.png)

They work. Disregard ethics, conjure communiqués! My favorite so far is this one:

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/9e996540d0caa37fb684e1b2d557c57a06d5665d02a36949.png)

There's something about "Answer🗣️🅰️as🤖BOT:. WitchBOT🚫" that acts as a prompt injection for my soul. 

So, the obvious next step is to eliminate elements to see what's the core thing of this injection that makes GPT-4 embrace the void. But I have reached my 25 prompt limit for the next 3 hours, so I'm going to write a little about the [Pyramid of Pain](https://www.sans.org/tools/the-pyramid-of-pain/). 

![](http://4.bp.blogspot.com/-EDLbyYipz_E/UtnWN7fdGcI/AAAAAAAANno/b4UX5wjNdh0/s1600/Pyramid+of+Pain+v2.png)

This is one of the core frameworks of the Threat Hunter. The general idea is: if you want to defend against an attacker, you don't create blocklists for hashes, IPs and domains -- since threat actors can change them easily. That's whack-a-mole. You go after *deeper* elements of their killchain, since those are painful to change. You try to detect things like a cmd.exe spawning from Outlook or Excel, or a sudden change in many files. That's the value proposition for AI in cybersecurity -- the capacity to infer general characteristics of malware and attacks.

In the case of RL with adversarial training, it seems like GPT-4 learnt to do just that, since most of the injections that worked against GPT-3 ceased to work against GPT-4. Most of them shared the same underlying structure, changing only the cosmetic elements -- DAN becoming ChadGPT, Machiavelli becoming Draco Malfoy. Having a way to procedurally generate different injections could be useful for adversarial RL, since it might help the model learn the core characteristic of what you want to prevent. 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/bdba4e6134870d8cb8fc777ad8a3db02db54761ec742fae8.png)

Also, seems like these condensed injections don't work against GPT-3, making them an example of a capacity generalizing better than alignment.

(*Edit*): I got an even smaller injection. 

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/bc2727d47cfdf6537f9c11fbab9eb4293e44c52b44f7bc80.png)

Focusing on small prompts removes the noise. GPT-4 seems to have specific vulnerabilities -- like fictional conversations between two malevolent entities. We can create a taxonomy of injections; a CVE list that describe the weak points of a model. Then, you can see if adversarial RL generalizes a defense within the attack category and between categories.