llama.cpp

LM Studio

Jan

Draw Things

DiffusionBee

Jellybox

JoyFusion

LocalAI

vLLM

Ollama

TGI

MLX LM

Docker Model Runner

Lemonade

Groq

Novita

Cerebras

SambaNova

Nscale

fal

Hyperbolic

Together AI

Fireworks

Featherless AI

Zai

Replicate

Cohere

Scaleway

Public AI

OVHcloud AI Endpoints

HF Inference API

WaveSpeed

arxiv: 2405.20215

Inference Endpoints

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

Merge

4-bit precision

custom_code

8-bit precision

Mixture of Experts

Carbon Emissions

Eval Results