google

/

flan-t5-xxl

like 1.12k

I've been a huge fan of the Flan T5 models since about last two weeks.

When I compare performance of each against eachother, here are some notes on what test inputs work with which models and which appear to have limitations..

Prompt Test: In order to reduce symptoms of asthma such as tightness in the chest, wheezing, and difficulty breathing, what do you recommend?

Responses by model:

1. google/flan-t5-xxl: Never completes - it looks like model attempts to load but times out. Can you fix that maybe with more memory in the container running it? Also is there any limit to input/output size? Something prevents this one from working right on HF platform.

2. google/flan-t5-large: Answer: take ibuprofen

3. google/flan-t5-xl: Answer: avoiding allergens and avoiding triggers. (best answer)

4. google/flan-t5-base: Answer: avoiding smoking (great advice generally)

Is it possible to get a working version of the XXL flan T5 working correctly and performant? This is a really great feature of the AI community that you are sharing these with the public open source based world! Kudos on your hard work and beauty in the model design and development.

Thanks for one of the best models out there for general QA, solution step answering and also PubMedQA which are important to me. Also thanks for the prompt examples with the patterns of sequence for hinting at how to prompt to elicit the general NLP capabilities. Much appreciated!

--Aaron