t5 - base- askscience
- t5-v1_1 trained on the entirety of the askscience sub-section of the eli5 dataset for one epoch.
- compare to bart on eli5 here
- note that for the inference API, the model is restricted to outputting 96 tokens - by using the model in python with the transformers library, you can get longer outputs.
- for inputs, the model was presented with the post title and the post selftext encoded as:
question: <post title> context: <post selftext>. You may see better results if queries are posed in this fashion.
- The top two replies were aggregated and presented to the model as the output text.
- Training for longer will be explored, but given that the dataset has 127k examples and the loss flatlines at 0.5 epochs so this model should be fairly viable.
- Downloads last month
This model can be loaded on the Inference API on-demand.