model,throughput,response_length,latency,energy
BAIR/koala-13b,25.50117012651914,258.85426460711886,10.063550845041279,1946.198744123579
BAIR/koala-7b,32.92721737057136,247.0456682337139,7.462044263397255,1195.704790463391
BlinkDL/RWKV-4-Raven-7B-v12-Eng98%-Other2%-20230521-ctx8192.pth,64.47852685919943,209.4788448623237,2.9212886777848954,736.6553304231044
FreedomIntelligence/phoenix-inst-chat-7b,54.276457252393016,233.51108126259234,4.103907452767313,1002.3285728676934
camel-ai/CAMEL-13B-Combined-Data,26.30022114657458,282.4801880456682,10.72763999556464,2349.1830879784957
databricks/dolly-v2-12b,24.809807917158277,143.01410342511753,5.488031835448774,1002.4862169241038
lmsys/vicuna-13B,26.471361088447697,269.1239086635326,10.13281609787714,2151.007414036257
lmsys/vicuna-7B,33.42934778322944,272.4801880456682,8.138669158259418,1384.4621272666218
metaai/llama-13B,21.603271794897424,78.05742108797851,3.1706939710784794,671.8356437206298
metaai/llama-7B,27.65154362364814,67.52249832102082,2.1186345380820564,369.4284586971132
tatsu-lab/alpaca-7B,33.28849736927566,124.2155809267965,3.679997416454322,596.6560735392795
Neutralzz/BiLLa-7B-SFT,33.545066744328324,142.316319677636,4.214619093185649,677.7428566151748
OpenAssistant/oasst-sft-1-pythia-12b,24.918246219702905,251.10006715916722,9.773108534123995,2057.1658650100785
h2oai/h2ogpt-gm-oasst1-en-2048-open-llama-7b-preview-300bt-v2,33.01663685477516,206.82236400268636,6.229473677366736,998.7828710543929
lmsys/fastchat-t5-3b-v1.0,19.17742489592329,312.91034251175284,20.878918582659214,1744.028456346554
nomic-ai/gpt4all-13b-snoozy,26.07277221094992,216.6400268636669,8.208376720764958,1738.0828875083905
openaccess-ai-collective/manticore-13b-chat-pyg,26.16054700304064,276.17058428475485,10.500794145583235,2117.0177760241745
StabilityAI/stablelm-tuned-alpha-7b,45.937935977440574,244.92578912021492,5.171557462488085,1199.1363797850827
project-baize/baize-v2-7B,31.98482659469149,316.2058428475487,9.718332352424522,1593.2951074546695
togethercomputer/RedPajama-INCITE-7B-Chat,27.099977162771395,275.3183344526528,9.708286378669253,1423.8720110812499