| # GPT-Code-Clippy-125M-APPS | |
| ## Model Description | |
| GPT-CC-125M-APPS is a GPT-Neo-125M finetuned on APPS dataset. This model is specialized to solve programming tasks. | |
| ## Training data | |
| [APPS dataset](https://github.com/hendrycks/apps). | |
| ## Training procedure | |
| The training script used to train this model can be found [here](https://github.com/ncoop57/gpt-code-clippy/blob/camera-ready/training/run_clm_apps.py). | |
| ## Intended Use and Limitations | |
| The model is finetuned to solve programming problems given a text description and optional starter code. | |
| ### How to use | |
| You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run: | |
| ```py | |
| from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM | |
| model = AutoModelForCausalLM.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata") | |
| tokenizer = AutoTokenizer.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata") | |
| prompt = """ | |
| A function to greet user. Given a user name it should say hello | |
| def greet(name): | |
| ANSWER: | |
| """ | |
| input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device) | |
| start = input_ids.size(1) | |
| out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, | |
| early_stopping=True, eos_token_id=tokenizer.eos_token_id, ) | |
| print(tokenizer.decode(out[0][start:])) | |
| ``` | |
| ### Limitations and Biases | |
| The model is intended to be used for research purposes and comes with no guarantees of quality of generated code. | |
| GPT-CC is finetuned GPT-Neo and might have inhereted biases and limitations from it. See [GPT-Neo model card](https://huggingface.co/EleutherAI/gpt-neo-125M#limitations-and-biases) for details. | |
| ## Eval results | |
| Coming soon... | |