Spaces:
Runtime error
Runtime error
Results aren't as good as the paper
#9
by
rajistics
- opened
FYI - This version is far from the model shown in the paper. When I slightly tweak the examples sentences the answers don't make sense:
Can Thomas Jefferson have a conversation with George Washington? Give the rationale before answering.
Q: Can Barack Obama have a conversation with Donald Trump? Give the rationale before answering.
What is the boiling point of alcohol?
Answer the following question by reasoning step by step. The cafeteria had 23 apples. If they bought 6 more and used 20 for lunch, how many apples do they have?
Other examples from the paper I can't replicate:
What is the boiling point of Nitrogen?
Answer the following yes/no question
by reasoning step-by-step.
Can you write a whole Haiku in a
single tweet?
I think the examples in the paper are by flan-palm 540B, so they are likely to be much better than flan-t5-xl
With the upgrade to flan-t5-xxl - I was able to get much better answers for reasoning
rajistics
changed discussion status to
closed