Nice results
#1
by
darraghd
- opened
Very nice work, after some initial teething problems, like here, I have this running on A100. The results on my prompts are showing to be slightly better than the google/tf-flan-xxl with full precision. Maybe this is be down to chance but there was an improvement on a number of prompts.
In any case, the memory usage is way down, 40GB for full precision and a short prompt, down to around 18GB for your 8-bit version. This is great, it allows me to experiment with much larger prompts...
Thanks a lot for releasing this !!
darraghd
changed discussion status to
closed