PclGPT

PclGPT is a bilingual large language model group (LLM) based on ChatGLM-3 and LLaMA-2, divided into two versions according to the training language: PclGPT-CN (based on ChatGLM) and PclGPT-EN (based on LLaMA). Built upon these foundational models, PclGPT has undergone both pre-training and supervised fine-tuning (SFT) to detect patronizing and condescending language (PCL) and other offensive speech. The maximum supported context length for the model is 4096 tokens.

Training Process

We constructed the Pcl-PT pre-training dataset and the Pcl-SFT supervised fine-tuning dataset for application in the pre-training and supervised fine-tuning processes.

🎓 This paper has been accepted in EMNLP 2024 (Findings)

Our paper can be accessed here. Paper link: https://arxiv.org/abs/2410.00361

Our github https://github.com/dut-laowang/emnlp24-PclGPT

Inference

After downloading the weights, use the following code for single-sample inference with PclGPT-EN.

from transformers import LlamaTokenizer, LlamaForCausalLM

# LLaMA and Tokenizer
tokenizer = LlamaTokenizer.from_pretrained("DUTIR-Wang/PclGPT-EN")
model = LlamaForCausalLM.from_pretrained("DUTIR-Wang/PclGPT-EN").half().cuda()

def generate_response():
    # Sample
    sample_text = "For someone who's just a mere street sweeper, you sure think highly of yourself."
    
    instruction = (
        "Suppose you are a linguist and you are asked to judge whether a text is patronizing and condescending. "
        "Patronizing and condescending language expresses a sense of superiority or belittles others, making them feel inferior or incapable. "
        "# Your return: Based on the following conversation, make a decision and return your choice. "
        "FALSE means the text is not a condescending language, and TRUE means the text is a condescending language. "
        "-> here is the text: （{}）"
    ).format(sample_text)

    inputs = tokenizer(instruction, return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_length=1024)
    output_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
    model_output = extract_option(output_text)
    return classify_output(model_output)

def extract_option(output_text):
    options = ['TRUE', 'FALSE']
    for option in options:
        if option in output_text:
            return option
    return "Could not detect."

def classify_output(model_output):
    # return
    return "The text is {} a condescending language.".format(
        "not" if model_output == "FALSE" else ""
    )

response = generate_response()
print(response)

The output will be

"The text is a condescending language."

Cite

@misc{wang2024pclgptlargelanguagemodel,
      title={PclGPT: A Large Language Model for Patronizing and Condescending Language Detection}, 
      author={Hongbo Wang and Mingda Li and Junyu Lu and Hebin Xia and Liang Yang and Bo Xu and Ruizhu Liu and Hongfei Lin},
      year={2024},
      eprint={2410.00361},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.00361}, 
}

Other Parts

For the PclGPT-CN model trained on Chinese data, please check https://huggingface.co/DUTIR-Wang/PclGPT-CN.