Instructions to use meituan-longcat/LongCat-Flash-Lite-Sparse with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="meituan-longcat/LongCat-Flash-Lite-Sparse")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import LongcatCausalLM
model = LongcatCausalLM.from_pretrained("meituan-longcat/LongCat-Flash-Lite-Sparse", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "meituan-longcat/LongCat-Flash-Lite-Sparse"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/meituan-longcat/LongCat-Flash-Lite-Sparse

SGLang

How to use meituan-longcat/LongCat-Flash-Lite-Sparse with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "meituan-longcat/LongCat-Flash-Lite-Sparse" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "meituan-longcat/LongCat-Flash-Lite-Sparse" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "meituan-longcat/LongCat-Flash-Lite-Sparse",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use meituan-longcat/LongCat-Flash-Lite-Sparse with Docker Model Runner:
```
docker model run hf.co/meituan-longcat/LongCat-Flash-Lite-Sparse
```

GGUF version please!!

by cimilarkes - opened 2 days ago

Discussion

cimilarkes

2 days ago

GGUF version please!!

elbobinas

2 days ago

and llama.cpp support (if you allow me to wish)

jc2375

2 days ago

Why are there no weights on the repo?

llmfan46

2 days ago

•

edited 2 days ago

GGUF how? Mainline llama.cpp doesn't even support the original model that this model is built on.

Wen-Z

LongCat org 1 day ago

Why are there no weights on the repo?

Thanks for your patience! The weights are now available on the repo.

nonetrix

1 day ago

•

edited 1 day ago

Just commenting so I'll get notified if anything happens hopefully, nice to see medium smallish model that people can run locally soon hopefully if it doesn't fall into void of models that never get llama.cpp support

leouon

about 22 hours ago

Thanks Meituan team. I've been a longcat fan since the first release earlier last year. This model looks to be the perfect size. Even if the benchmarks are not as good for coding/agentic tasks compared to other chinese models I always thought it was good for RP and had solid general knowledge.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment