bigcode

/

starcoder2-15b

like 492

Hey,

This is the code that I'm using to run FIM but I suspect I am not doing this correctly. I am following instructions from a santacoder discussion on how to use the FIM tokens: https://huggingface.co/bigcode/santacoder/discussions/10#63c324e0e25fc176427928aa

# pip install -q git+https://github.com/huggingface/transformers.git 

import os
import os.path
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig

model = AutoModelForCausalLM.from_pretrained(
    checkpoint, 
    #quantization_config=quantization_config,
    cache_dir=MODEL_CACHE_DIR,
)

def generate_completion(text: str):
    inputs = tokenizer.encode(text, return_tensors="pt")
    outputs = model.generate(inputs, max_length=128)
    return tokenizer.decode(outputs[0])

text = "<fim-prefix>def fib(n):<fim-suffix>    else:\n        return fib(n - 2) + fib(n - 1)<fim-middle>"

print(generate_completion(text))

However the generated output doesn't give the appropriate code, instead it outputs this:

<fim-prefix>def fib(n):<fim-suffix>    else:
        return fib(n - 2) + fib(n - 1)<fim-middle>

<fim-prefix>print(fib(10))<fim-suffix>
```

## 2.2.2. 递归的优点

递归的优点是逻辑简单易懂，缺点是过深的调用会导致栈溢出。

针对尾递归优化的语言可以通过尾递归防止栈溢出

Is there a way to fix this? I was unable to find documentation around this.

Thanks!