mamba-2.8b-hfは英語Universal Dependenciesの係り受けリンクをFew-Shot Promptingで生成できるか

Ercong Nie, Shuzhou Yuan, Bolei Ma, Helmut Schmid, Michael Färber, Frauke Kreuter, Hinrich Schütze『[Decomposed Prompting: Unveiling Multilingual Linguistic Structure Knowledge in English-Centric Large Language Models](https://arxiv.org/abs/2402.18397)』を横目に、[mamba-2.8b-hf](https://huggingface.co/state-spaces/mamba-2.8b-hf)を使って、英語Universal Dependenciesの係り受けリンクでDecomposed Promptingを試してみた。Google Colaboratory (GPU版)だと、こんな感じ。

```
!pip install transformers accelerate causal-conv1d mamba-ssm
ex="""
###text: And if you send me a story, that would be great to!
In Universal Dependencies of the text, ‘if’ is linked from ‘send’ with the label of mark.
###text: I'll admit I wasn't expecting much from this place, but they really did do a good job.
In Universal Dependencies of the text, ‘good’ is linked from ‘job’ with the label of amod.
###text: "I don't know." The girl shrugged once again.
In Universal Dependencies of the text, ‘girl’ is linked from ‘shrugged’ with the label of nsubj.
###text: The dancers were falling back around a Polish agriculturalist who was teaching a gangling Englishman and two young Africans an Eastern European peasant dance.
In Universal Dependencies of the text, ‘around’ is linked from ‘agriculturalist’ with the label of case.
###text: Antigua was awesome.
In Universal Dependencies of the text, ‘was’ is linked from ‘awesome’ with the label of cop.
###text: The food is fresh and taste great.
In Universal Dependencies of the text, ‘The’ is linked from ‘food’ with the label of det.
###text: Now I have wife and son.
In Universal Dependencies of the text, ‘Now’ is linked from ‘have’ with the label of advmod.
###text: However, this fruitful period was short-lived, as Greece suffered badly under the Ottoman Empire, only to recover in the 19th century as the capital of independent Greece.
In Universal Dependencies of the text, ‘suffered’ is linked from ‘lived’ with the label of advcl.
###text: I survived it without a problem.
In Universal Dependencies of the text, ‘.’ is linked from ‘survived’ with the label of punct.
###text: The food is fresh and taste great.
In Universal Dependencies of the text, ‘and’ is linked from ‘taste’ with the label of cc.
###text: you can view at dresscod.com
In Universal Dependencies of the text, ‘dresscod.com’ is linked from ‘view’ with the label of obl.
###text: "I don't know." The girl shrugged once again.
In Universal Dependencies of the text, ‘shrugged’ is linked from ‘shrugged’ with the label of root.
###text: I'll admit I wasn't expecting much from this place, but they really did do a good job.
In Universal Dependencies of the text, ‘n't’ is linked from ‘expecting’ with the label of advmod.
###text: Antigua was awesome.
In Universal Dependencies of the text, ‘Antigua’ is linked from ‘awesome’ with the label of nsubj.
###text: The dancers were falling back around a Polish agriculturalist who was teaching a gangling Englishman and two young Africans an Eastern European peasant dance.
In Universal Dependencies of the text, ‘two’ is linked from ‘Africans’ with the label of nummod.
###text: Yes, the Cyclone is almost certain to lose strength as it surges over land.
In Universal Dependencies of the text, ‘Yes’ is linked from ‘certain’ with the label of discourse.
###text: —-== Posted via Newsfeed.Com - Unlimited - Uncensored - Secure Usenet News ==—-
In Universal Dependencies of the text, ‘—-==’ is linked from ‘Posted’ with the label of punct.
"""
from transformers import AutoTokenizer,AutoModelForCausalLM,TextGenerationPipeline
tkz=AutoTokenizer.from_pretrained("state-spaces/mamba-2.8b-hf")
mdl=AutoModelForCausalLM.from_pretrained("state-spaces/mamba-2.8b-hf",torch_dtype="auto",device_map="auto")
tgn=TextGenerationPipeline(model=mdl,tokenizer=tkz,max_new_tokens=128)
nlp=lambda x,y=None:"\n".join(tgn(ex+f"###text: {x}\nIn Universal Dependencies of the text"+(f", ‘{y}’" if y else ","))[0]["generated_text"][len(ex):].split("\n")[0:2])
print(nlp("I have a pen.","I"))
print(nlp("I have a pen.","have"))
print(nlp("I have a pen.","a"))
print(nlp("I have a pen.","pen"))
print(nlp("I have a pen.","."))
```

「I have a pen.」を係り受け解析してみたところ、私(安岡孝一)の手元では以下の結果が出力された。

> ###text: I have a pen.
> In Universal Dependencies of the text, ‘I’ is linked from ‘have’ with the label of det.
> ###text: I have a pen.
> In Universal Dependencies of the text, ‘have’ is linked from ‘pen’ with the label of prep.
> ###text: I have a pen.
In Universal Dependencies of the text, ‘a’ is linked from ‘pen’ with the label of det.
> ###text: I have a pen.
> In Universal Dependencies of the text, ‘pen’ is linked from ‘have’ with the label of det.
> ###text: I have a pen.
> In Universal Dependencies of the text, ‘.’ is linked from ‘have’ with the label of det.

残念ながら、リンクもラベルも無茶苦茶だ。[4月6日](https://qiita.com/KoichiYasuoka/items/d6a2f3eb36fdec3ab4a3)や[4月24日](https://qiita.com/KoichiYasuoka/items/555e2fa6855a95b35df4)の記事と較べても、mamba-2.8b-hfは明らかに劣っている。やっぱり、なかなかうまく行かないなあ。