Before revision:   Our work highlights the promise of using LMs trained on text with (un) desired attributes for efficient decoding-time controlled language generation .
After revision:  Moreover, because DExperts operates only on the output of the pretrained LM, it is effective with (anti-)experts of smaller size, including when operating on GPT-3. Our work highlights the promise of using LMs trained on text with (un) desired attributes for efficient decoding-time controlled language generation .