model,latency
Phi-3-mini-128k-instruct,0.6615315832127351
Meta-Llama-3.1-70B-Instruct,0.8105055275945272
Meta-Llama-3.1-405B-Instruct-Turbo,0.7886103946545832
mistral-medium-2312,3.3167870515212137
gemma-2-2b-it,0.3139821517919896
Meta-Llama-3-8B-Instruct-hf,2.116447543256806
Qwen1.5-1.8B-Chat,0.5502705450000004
CodeLlama-34b-Instruct-hf,3.851887315425931
Qwen2-72B-Instruct,0.9480584860151365
Meta-Llama-3.1-8B-Instruct,0.20630574840608434
gemma-1.1-7b-it,0.1782953878345513
gemma-2-27b-it,0.9922771009345804
openchat-3.5-1210,0.28049827691029705
codegemma-7b-it,0.30489740508652785
dolphin-2.5-mixtral-8x7b,1.100729847312237
sheep-duck-llama-2-70b-v1.1,5.524607914346896
gpt-4-0613,0.6484411465828742
Meta-Llama-3-70B-Instruct-hf,3.348256158662245
openchat_3.5,0.31728768684620623
Yi-34B-Chat,1.2871676207135385
gpt-4o-2024-05-13,0.5570990732389735
openchat-3.5-0106,0.2920951450556654
Mixtral-8x7B-Instruct-v0.1,0.9392967660636317
gpt-4,0.9930876319528372
Qwen1.5-14B-Chat,0.37289333481152975
mistral-large-2402,0.3967416598893948
claude-3-5-sonnet-20240620,0.879929281888541
gemma-7b-it,0.6112263564356414
gpt-4-0125-preview,1.0418927523113601
gpt-4-1106-preview,0.7767265743542753
gemini-1.5-pro-latest,82.90283880578006
Nous-Hermes-2-Mixtral-8x7B-SFT,0.860892263303385
claude-2.1,1.6836316221022491
gemma-1.1-2b-it,0.11925699461279458
claude-3-opus-20240229,2.955845827917778
command-r,0.18832414914586068
gpt-4o-mini-2024-07-18,0.5342684150088973
sheep-duck-llama-2-13b,2.9462099794520453
SUS-Chat-34B,2.27951476106911
claude-3-haiku-20240307,0.869549739619107
gpt-4o-2024-08-06,0.5169587531894035
Yi-1.5-34B-Chat,9.232551962889337
Qwen2-7B-Instruct,0.3589407217948713
Mixtral-8x22B-Instruct-v0.1,1.0759354563573875
vicuna-13b-v1.5,1.4753938719676492
gemma-2-9b-it,0.3692553324432539
Mistral-7B-Instruct-v0.2,0.7635151196709047
Qwen1.5-32B-Chat,0.5092292557397938
Qwen1.5-0.5B-Chat,0.173469139375476
Starling-LM-7B-beta,1.365002297029707
gemini-1.0-pro,0.6346876567117742
gemini-1.5-flash-latest,37.2042672100488
aya-23-35B,0.5755088395104249
llama-2-70b-chat-hf,4.724659620079609
Yi-1.5-9B-Chat,8.02352422018858
WizardLM-70b-v1.0,3.9249772038834863
Yi-1.5-6B-Chat,6.496741103848931
tulu-2-dpo-70b,7.848597339328551
Qwen1.5-72B-Chat,12.689668927658234
Mistral-7B-Instruct-v0.1,0.2828647550771723
gpt-3.5,0.6829601016193744
aya-23-8B,0.4818848185613409
Qwen1.5-7B-Chat,0.38989076908838965
claude-3-sonnet-20240229,1.419486012822594
vicuna-33b-v1.3,0.8235025152162343
WizardLM-13b-v1.2,3.565436762576296
Mistral-Large-Instruct-2407,1.2444667688634197
command-r-plus,0.3104016019283745