Create README.md

README.md

@@ -0,0 +1,52 @@
+# GPT-Code-Clippy-125M-APPS
+
+## Model Description
+
+GPT-CC-125M-APPS is a GPT-Neo-125M finetuned on APPS dataset. This model is specialized to solve programming tasks.
+
+## Training data
+
+[APPS dataset](https://github.com/hendrycks/apps).
+
+## Training procedure
+
+The training script used to train this model can be found [here](https://github.com/ncoop57/gpt-code-clippy/blob/camera-ready/training/run_clm_apps.py).
+
+## Intended Use and Limitations
+
+The model is finetuned to solve programming problems given a text description and optional starter code.
+
+### How to use
+
+You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
+
+```py
+from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata")
+tokenizer = AutoTokenizer.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata")
+
+prompt = """
+A function to greet user. Given a user name it should say hello
+
+def greet(name):
+
+ANSWER:
+""" 
+
+input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device)
+start = input_ids.size(1)
+out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, 
+                     early_stopping=True, eos_token_id=tokenizer.eos_token_id, )
+
+print(tokenizer.decode(out[0][start:]))
+```
+
+### Limitations and Biases
+
+The model is intended to be used for research purposes and comes with no guarantees of quality of generated code.
+GPT-CC is finetuned GPT-Neo and might have inhereted biases and limitations from it. See [GPT-Neo model card](https://huggingface.co/EleutherAI/gpt-neo-125M#limitations-and-biases) for details.
+
+## Eval results
+
+Coming soon...