model,throughput,response_length,latency,energy
BAIR/koala-13b,21.207721076321384,265.5732034922767,12.1433556952537,2144.8338435191445
BAIR/koala-7b,26.5628207031951,261.0698455339154,9.620738416075923,1345.554839153794
RWKV/rwkv-raven-7b,61.52127877000437,214.78475486903963,3.0815914876914072,735.766777367353
FreedomIntelligence/phoenix-inst-chat-7b,55.838321060384565,240.34419073203492,4.122390997297159,1030.8907246474148
Neutralzz/BiLLa-7B-SFT,33.57049533371462,161.80826057756883,4.794732123814392,881.4395631296157
OpenAssistant/oasst-sft-1-pythia-12b,25.404288065688412,259.58764271323037,9.929468848113123,2060.029685359295
StabilityAI/stablelm-tuned-alpha-7b,45.42078300695383,243.87541974479515,5.208388584671265,1124.4006343183476
camel-ai/CAMEL-13B-Combined-Data,24.501197269962507,291.92008059100067,13.17006271185142,2526.4594200805864
databricks/dolly-v2-12b,22.10974954935886,153.75822699798522,7.943902623603982,1242.9209395567464
h2oai/h2ogpt-gm-oasst1-en-2048-open-llama-7b-preview-300bt-v2,33.19728218417469,212.45601074546676,6.392467083725633,951.077651108126
lmsys/fastchat-t5-3b-v1.0,19.291850692305534,314.29684351914034,20.682375825357692,1802.9754949630653
lmsys/vicuna-13B,21.611375785831733,280.739758226998,12.738298498406273,2600.83922397583
lmsys/vicuna-7B,27.25945669782727,284.9194089993284,10.297412690598657,1531.7009963062367
metaai/llama-13B,23.201365367977594,76.47078576225655,2.9742001407414107,631.5696930826057
metaai/llama-7B,27.470901624921254,60.65144392209537,1.9862404549640509,335.25658764271293
nomic-ai/gpt4all-13b-snoozy,26.567282559123655,247.79583613163197,9.280741749712535,2004.7329180658162
openaccess-ai-collective/manticore-13b-chat-pyg,26.280767439646972,288.8153122901276,10.947609359217866,2189.247364674278
project-baize/baize-v2-7B,31.591810721211225,326.29952988582943,10.17013366094905,1588.2741897246513
tatsu-lab/alpaca-7B,28.0006749636751,132.84586971121558,4.627808613438331,684.1749345198082
togethercomputer/RedPajama-INCITE-7B-Chat,27.48563290418453,274.39456010745465,9.540005102660757,1457.8563905305575
Salesforce/xgen-7b-8k-inst,49.54232207970749,275.2686366689053,5.597276540872152,1246.0923297515117