Model initialization is not with ray server.
When I am trying to initiate the mode, i get this error
(ServeController pid=11740) ValueError: The checkpoint you are trying to load has model type span-marker but Transformers does not recognize this architecture. This could be because of an issue with the checkpoint, or because your version of Transformers is out of date
This is the code --
@serve
.deployment
class MultilingualNER:
def init(self):
model_path = "lxyuan/span-marker-bert-base-multilingual-uncased-multinerd"
self.model = pipeline("ner", model=model_path, tokenizer=model_path)
def recognize(self, text: str) -> dict:
return self.model(text)
This issue is not happening for other Hugging Face models.
Hi,
That's right. The span-marker model architecture isn't included in Transformers. You can load the model using SpanMarkerModel and use it with a custom NER class.
# First, ensure you have the span_marker library installed
# You can install it using pip:
# pip install span_marker
from span_marker import SpanMarkerModel
model = SpanMarkerModel.from_pretrained("lxyuan/span-marker-bert-base-multilingual-uncased-multinerd")
model.predict("Yesterday, Elon Musk, Tesla CEO, met with Angela Merkel, former Chancellor, in Berlin to discuss sustainable energy solutions.")
>>>
[
{'span': 'Elon Musk', 'label': 'PER', 'score': 0.9999874830245972, 'char_start_index': 11, 'char_end_index': 20}
{'span': 'Tesla', 'label': 'ORG', 'score': 0.9990750551223755, 'char_start_index': 22, 'char_end_index': 27}
{'span': 'Angela Merkel', 'label': 'PER', 'score': 0.9999912977218628, 'char_start_index': 42, 'char_end_index': 55}
{'span': 'Berlin', 'label': 'LOC', 'score': 0.9999889135360718, 'char_start_index': 79, 'char_end_index': 85}
]
Hi, Thanks for the reply.
This worked for me.
from span_marker import SpanMarkerModel
@serve
.deployment
class SpanMarkerDeployment:
def __init__(self):
# Load the model when the deployment is initialized
self.english_multilingiual_model = SpanMarkerModel.from_pretrained("lxyuan/span-marker-bert-base-multilingual-uncased-multinerd")
def predict(self, text: str) -> str:
# Use the model to predict and return results
return self.english_multilingiual_model.predict(text)