Repository: huggingface/text-generation-inference

Issue:
SantaCoder Object has no attribute has_position_ids
### System Info

I am using text-generation-inference for a locally deployed StarCoder model, and I am encountering an error when running inference. Specifically, I am receiving the error message 'SantaCoder Object has no attribute has_position_ids'. After inspecting the code, I have determined the cause of the error.

The StarCoder model has a model-type of 'gpt-bigcode', which maps to the SantaCoder model in text-generation-inference. The SantaCoder model is a child class of CausalLM, which is a child class of Model. In CausalLM's __init__ function, an attribute called 'has_position_ids' is defined and used in CausalLM's forward function. However, SantaCoder does not call CausalLM's __init__ function. Instead, it calls Model's __init__ function using the syntax `super(CausalLM, self).__init__()`. As a result, when SantaCoder's forward function is called (which actually calls CausalLM's forward function), the 'has_position_ids' attribute is not defined, leading to the error.

It is unclear whether this is a bug or if I am using it incorrectly.

### Information

- [ ] Docker
- [X] The CLI directly

### Tasks

- [ ] An officially supported command
- [ ] My own modifications

### Reproduction

1. download starcoder 
2. text-generation-launcher --model-id /root/autodl-tmp/models/starcoder --sharded false --trust-remote-code `true` --port 8088
3. use a client to send request

from text_generation import Client

API_URL = 'http://localhost:8088/'

generate_kwargs = dict(
    temperature=0.2,
    max_new_tokens=512,
    top_p=0.95,
    repetition_penalty=1.1,
    do_sample=True,
    seed=42,
)


def print_output(stream):
    output = ''
    for response in stream:
        if response.token.text == "
":
            break
        else:
            output += response.token.text
    return output


client = Client(API_URL)
prompt = 'def hello():'

stream = client.generate_stream(prompt, **generate_kwargs)

print_output(stream)

### Expected behavior

return the completion code.


Discussion:

Resolving patch:
diff --git a/server/text_generation_server/models/causal_lm.py b/server/text_generation_server/models/causal_lm.py
--- a/server/text_generation_server/models/causal_lm.py
+++ b/server/text_generation_server/models/causal_lm.py
@@ -496,11 +496,6 @@ def __init__(
             else:
                 tokenizer.add_special_tokens({"pad_token": "[PAD]"})
 
-        self.has_position_ids = (
-            inspect.signature(model.forward).parameters.get("position_ids", None)
-            is not None
-        )
-
         super(CausalLM, self).__init__(
             model=model,
             tokenizer=tokenizer,
diff --git a/server/text_generation_server/models/model.py b/server/text_generation_server/models/model.py
--- a/server/text_generation_server/models/model.py
+++ b/server/text_generation_server/models/model.py
@@ -1,3 +1,4 @@
+import inspect
 import torch
 
 from abc import ABC, abstractmethod
@@ -29,6 +30,12 @@ def __init__(
         self.device = device
         self.rank = rank
         self.world_size = world_size
+
+        self.has_position_ids = (
+            inspect.signature(model.forward).parameters.get("position_ids", None)
+            is not None
+        )
+
         self.check_initialized()
 
     @property


Test patch: