CarperAI

/

FIM-NeoX-1.3B

like 26

+1

thanks for bringing this to our attention! Looking into this and will get back to you asap.

Thank you for raising this concern. It seems like it's an issue with the tokenizer. Unfortunately all of our engineers are OOO for the long weekend, we should have a patch out Tuesday or Wednesday. Thanks.

There was an issue where the sentinel <|SUF|>, <|PRE|>, and <|MID|> tokens were not the correct ids in the uploaded tokenizer and model card! Please try clearing the Huggingface cache and redownloading the model :))

This is what I get, attempting to try out open-ended generation on a simple code function

def score(x,y) -> int:
    """
    

and also infilling with

def score(x,y) -> int:
    """
    <|MID|> (infill here)
    """

    score = x + y
    return score

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model = AutoModelForCausalLM.from_pretrained("CarperAI/FIM-NeoX-1.3B")
tok = AutoTokenizer.from_pretrained("CarperAI/

# infilling demo
prefix = 'def score(x, y) -> int:\n"""\n'
suffix = '"""\n\n    score = x + y\n    return score'

 model_input = [50277, *tok(suffix)["input_ids"], 50278, *tok(prefix)["input_ids"], 50279]
 output = tok.decode(model.generate(torch.IntTensor(model_input).unsqueeze(0), max_length=40)[0])

print(output)

'<|SUF|>"""\n\n score = x + y\n return score<|PRE|>def score(x, y) -> int:\n"""\n<|MID|> score(x, y) -> int\n<|endoftext|>'

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# non-infilling demo
prefix = 'def score(x, y) -> int:\n"""\n'
model_input = [*tok(prefix)["input_ids"]]
output = tok.decode(model.generate(torch.IntTensor(model_input).unsqueeze(0), max_length=100)[0])
print(output)

'def score(x, y) -> int:\n"""\n Return the score of the given point.\n """\n return sum(x * y for x, y in zip(x_list, y_list))\n\ndef get_point_score(x, y) -> int:\n """\n Return the score of the given point.\n """\n return sum(x * y for x, y in zip(x_list, y'

Hope this helps! I will also update the model card with this example :)