Create README.md

README.md

@@ -0,0 +1,61 @@
+# GPT-Code-Clippy-125M-APPS
+
+## Model Description
+
+GPT-CC-125M-APPS is a GPT-Neo-1.3B finetuned on APPS dataset. This model is specialized to solve programming tasks.
+
+## Training data
+
+[APPS dataset](https://github.com/hendrycks/apps).
+
+## Training procedure
+
+The training script used to train this model can be found [here](https://github.com/ncoop57/gpt-code-clippy/blob/camera-ready/training/run_clm_apps.py).
+
+## Intended Use and Limitations
+
+The model is finetuned to solve programming problems given a text description and optional starter code.
+
+### How to use
+
+You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:
+
+```py
+
+from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM
+
+model = AutoModelForCausalLM.from_pretrained("flax-community/gpt-code-clippy-1.3B-apps-alldata")
+
+tokenizer = AutoTokenizer.from_pretrained("flax-community/gpt-code-clippy-1.3B-apps-alldata")
+
+prompt = """
+
+A function to greet user. Given a user name it should say hello
+
+def greet(name):
+
+ANSWER:
+
+""" 
+
+input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device)
+
+start = input_ids.size(1)
+
+out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, 
+
+                     early_stopping=True, eos_token_id=tokenizer.eos_token_id, )
+
+print(tokenizer.decode(out[0][start:]))
+
+```
+
+### Limitations and Biases
+
+The model is intended to be used for research purposes and comes with no guarantees of quality of generated code.
+
+GPT-CC is finetuned GPT-Neo and might have inhereted biases and limitations from it. See [GPT-Neo model card](https://huggingface.co/EleutherAI/gpt-neo-1.3B#limitations-and-biases) for details.
+
+## Eval results
+
+Coming soon...