Cannot reproduce inference examples from paper

#20
by Anandamoy - opened

I could not reproduce the inference results demonstrated in the paper. Is the deployed model different from the one used for evaluation?

FROMAGe-1680167292215.png

FROMAGe-1680167398257.png

This is strange, I just tried the demo and got a different result. I'm not quite sure why exactly, since this should be deterministic. Did you adjust any of the parameters (e.g., temperature?)
Screen Shot 2023-03-30 at 10.53.48 AM.png

To answer your question: the model here is different from the one in the paper. This one is generally better at dialogue settings. For the original model and reproduction of the examples shown, please check out the GitHub notebook.

Sign up or log in to comment