Text Generation
Transformers
PyTorch
mosaic_gpt
custom_code

Benchmark

#4
by zokica - opened

Hello,

Thanks for your hard work, first.

Did you do the test and how it compares with models such as Llama 7B?

Mosaic ML, Inc. org

We have some internal numbers, but I think the existing LLM benchmarks do a poor job evaluating the models in the way that we seek to use them (content generation, instruction following). We leave it to members of the community to evaluate the model as they like and come to their own conclusions about the dataset and resulting model.

jfrankle changed discussion status to closed

Sign up or log in to comment