Details

This is a roBERTa-base model trained on the python part of CodeSearchNet and reached a dev perplexity of 3.296

This model was used for the Programming Puzzles enumerative solver baseline detailed in Programming Puzzles paper.

See also the Python Programming Puzzles (P3) Repository for more details.

Usage

You can either load the model and further fine-tune it for a target task (as done for the puzzle solver), or you can experiment with mask-filling directly with this model as in the following example:

from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline

tokenizer = AutoTokenizer.from_pretrained("tals/roberta_python")
model = AutoModelWithLMHead.from_pretrained("tals/roberta_python")

demo = pipeline("fill-mask", model=model, tokenizer=tokenizer)

code = """sum= 0
for i in range(<mask>):
    sum += i
assert sum == 6
"""
demo(code)

BibTeX entry and citation info

@article{schuster2021programming,
      title={Programming Puzzles}, 
      author={Tal Schuster and Ashwin Kalyan and Oleksandr Polozov and Adam Tauman Kalai},
      year={2021},
      eprint={2106.05784},
      archivePrefix={arXiv},    
      url={https://arxiv.org/abs/2106.05784}  
}
New

Select AutoNLP in the “Train” menu to fine-tune this model automatically.

Downloads last month
12
Hosted inference API
Fill-Mask
Mask token: <mask>
Examples
Examples
This model can be loaded on the Inference API on-demand.