Hugging Face's logo

monsoon-nlp
/
gpt-nyc-affirmations

Text Generation
Transformers
PyTorch
Safetensors
gpt2
Inference Endpoints
text-generation-inference
Model card Files Community
1
gpt-nyc-affirmations / README.md
monsoon-nlp's picture
monsoon-nlp
add pytorch model and update readme
96f8530
preview code
|
raw history blame contribute delete
No virus
1.63 kB

GPT-NYC-affirmations

About

GPT2 (small version on HF) fine-tuned on questions and responses from https://reddit.com/r/asknyc and then 2 epochs of Value Affirmations based on the OpenAI post Improving Language Model Behavior and corresponding paper.

Try prompting with question? - %% or question? - more info %%

I filtered AskNYC comments to ones with scores >= 3, and responding directly to the original post ( = ignoring responses to other commenters). I also added many tokens which were common on /r/AskNYC but missing from GPT2.

The 'affirmations' list was sourced from excerpts in the OpenAI paper, a popular version of the 'in this house we believe' sign, and the Reddit rules. They should not be seen as all-encompassing or foundational to a safe AI. The main goal was to see how it affected the behavior of GPT-NYC on generating toxic or non-toxic language.

The gpt-nyc repo is based on GPT2-Medium and comes off more accurate.

Blog

https://mapmeld.medium.com/gpt-nyc-part-1-9cb698b2e3d

Notebooks

Data processing / new tokens

https://colab.research.google.com/drive/13BOw0uekoAYB4jjQtaXTn6J_VHatiRLu

Fine-tuning GPT2 (small)

https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR

Predictive text and probabilities

Scroll to end of

https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR

to see how to install git-lfs and trick ecco into loading this.