In need of muchmuch larger context
#4
by
matbee
- opened
Is the dataset available for fine-tuning a larger context base model?
Or, are there plans to release a much larger context model that is >= 16k context, and/or more than 7b parameters? I feel like a 30b model would stomp this 7b model.
edit: realized I wasnt using the large model and have reached 100% on my evaluation
Hi @matbee , no the dataset is not available. We do have plan to deal with (much) larger documents. Glad the large model works for you use case though!