allenai

/

cosmo-xl

like 82

It works fine, when I run on my local pc using transformers

Do you know how to setup chat history properly?
I write
dialogue = [
"Jakob said. I have a black dog.",
"Philip said. My name is Philip",
"Jakob said. Cool!",
"Philip said. My dog's name is Andi"
]

I'm not sure how the Inference API works, but Cosmo uses the special token to separate the utterances in the dialogue history.
The dialogue history should be given in a string format as the following:
utterance1 <TURN> utterance2 <TURN> ... <TURN> last utterance
Please follow the "How to use" section in the model card :)

The utterances don't have to be prepended with "Jakob said." or "David said."
You can simply prompt Cosmo with a role instruction.

Below is an example according to the "How to use" section:

situation = ""  # You can put some situation description
instruction = "You are Jakob and you are talking with Philip."
dialogue = [
    "I have a black dog.",
   "My name is Philip.",
   "Cool!",
   "My dog's name is Andi"
]

response = generate(situation, instruction, dialogue)
print(response)

How does Cosmo know what Jakob and Philip said. Philip and Jakob can send many messages at the same time.

instruction = "You are Jakob and you are talking with Philip."
dialogue = [
"Philip: I have a black rat.",
"Jakob: I have a black bird.",
"Jakob: I have a black cat.",
"Jakob: I have a black dog.",
"Philip: My name is Philip.",
"Jakob: Cool!",
"Philip: What animals do you have?"
]

VS

instruction = "You are Jakob and you are talking with Philip."
dialogue = [
"I have a black rat.",
"I have a black bird.",
"I have a black cat.",
"I have a black dog.",
"My name is Philip.",
"Cool!",
"What animals do you have?"
]

VS
(Or do you suggest to combine the many messages?)
instruction = "You are Jakob and you are talking with Philip."
dialogue = [
"I have a black rat.",
"I have a black bird. I have a black cat. I have a black dog.",
"My name is Philip.",
"Cool!",
"What animals do you have?"
]

same here. if the inference requires special tokens etc, perhaps better to set the pipeline_tag as text-to-text generation ?

Yes, the inference API for Cosmo is not working correctly :(
We added a simple demo code for having a chat with Cosmo in our official repository, please try it out!