Instructions to use mahin1234/lychee-gpt with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mahin1234/lychee-gpt with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mahin1234/lychee-gpt")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("mahin1234/lychee-gpt", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mahin1234/lychee-gpt with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mahin1234/lychee-gpt"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mahin1234/lychee-gpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mahin1234/lychee-gpt

SGLang

How to use mahin1234/lychee-gpt with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mahin1234/lychee-gpt" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mahin1234/lychee-gpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mahin1234/lychee-gpt" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mahin1234/lychee-gpt",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mahin1234/lychee-gpt with Docker Model Runner:
```
docker model run hf.co/mahin1234/lychee-gpt
```

🍒 Lychee GPT

Lychee GPT is a state-of-the-art conversational AI model, fine-tuned on a massive 31B parameter foundation model. Developed by MX LLMS, it delivers exceptional performance in both Bengali and English, featuring a massive 256K context window and stable generation of 11,000+ output tokens in a single pass.

Model Details

Model Description

Lychee GPT is the flagship model of MX LLMS, created to push the boundaries of long-context and long-output text generation for Bengali and English speakers. It is optimized for advanced instruction-following, complex reasoning, and dynamic conversational AI.

Developed by: MX LLMS & MD Mushfiqur Rahim
Funded by: Open-source community
Shared by: Hugging Face Hub
Model type: Causal Language Model (CLM) with 4-bit QLoRA fine-tuning
Language(s): Bengali (bn), English (en)
License: Apache 2.0
Finetuned from model: A 31B parameter foundation model

Model Sources

Repository: mahin1234/lychee-gpt
Demo: Coming soon on Hugging Face Spaces

Uses

Direct Use

This model is designed for:

Chatbots & Virtual Assistants (Bengali & English)
Long-form content generation (articles, stories, reports)
Code assistance & reasoning tasks
Educational and research purposes

Downstream Use

Can be fine-tuned further for specific domains like legal, medical, or customer support.

Out-of-Scope Use

High-risk decision-making (medical, legal, financial).
Malicious content generation (hate speech, misinformation, harassment).
Generating content without human oversight where factual accuracy is critical.

Bias, Risks, and Limitations

Lychee GPT inherits biases from its base foundation model. While fine-tuning on curated datasets reduces some harmful outputs, users should:

Use with human oversight in sensitive applications.
Avoid relying solely on the model for critical decisions.
Be aware that the model may produce hallucinations or outdated information.

Recommendations

Users (both direct and downstream) should implement content moderation and validation layers when deploying in production. Always verify factual claims generated by the model.

How to Get Started with the Model

Use the code below to load and run Lychee GPT instantly. The LoRA adapter will auto-detect the base model.

**Installation:**https://colab.research.google.com/#fileId=https://huggingface.co/mahin1234/lychee-gpt.ipynb kaggle.com/notebooks/new?accelerator=nvidiaTeslaT4&src=https://huggingface.co/mahin1234/lychee-gpt.ipynb

Training Data

The model was fine-tuned on a custom conversational dataset curated from publicly available sources, including Wikipedia (Bengali & English), open-source GitHub repositories, and curated prompt-response pairs from open foundation models. All data was filtered for quality and cultural relevance.

Downloads last month: -; Downloads are not tracked for this model. How to track

mahin1234
/

lychee-gpt