# GPT-Code-Clippy-125M-APPS

## Model Description

GPT-CC-125M-APPS is a GPT-Neo-125M finetuned on APPS dataset. This model is specialized to solve programming tasks.

## Training data

[APPS dataset](https://github.com/hendrycks/apps).

## Training procedure

The training script used to train this model can be found [here](https://github.com/ncoop57/gpt-code-clippy/blob/camera-ready/training/run_clm_apps.py).

## Intended Use and Limitations

The model is finetuned to solve programming problems given a text description and optional starter code.

### How to use

You can use this model directly with a pipeline for text generation. This example generates a different sequence each time it's run:

```py
from transformers import AutoModelForCausalLM, AutoTokenizer, FlaxAutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata")
tokenizer = AutoTokenizer.from_pretrained("flax-community/gpt-code-clippy-125M-apps-alldata")

prompt = """
A function to greet user. Given a user name it should say hello

def greet(name):

ANSWER:
""" 

input_ids = tokenizer(prompt, return_tensors='pt').input_ids.to(device)
start = input_ids.size(1)
out = model.generate(input_ids, do_sample=True, max_length=50, num_beams=2, 
                     early_stopping=True, eos_token_id=tokenizer.eos_token_id, )

print(tokenizer.decode(out[0][start:]))
```

### Limitations and Biases

The model is intended to be used for research purposes and comes with no guarantees of quality of generated code.
GPT-CC is finetuned GPT-Neo and might have inhereted biases and limitations from it. See [GPT-Neo model card](https://huggingface.co/EleutherAI/gpt-neo-125M#limitations-and-biases) for details.

## Eval results

Coming soon...