Instructions to use HuggingFaceBio/Carbon-X-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HuggingFaceBio/Carbon-X-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HuggingFaceBio/Carbon-X-Base")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-X-Base")
model = AutoModelForCausalLM.from_pretrained("HuggingFaceBio/Carbon-X-Base", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use HuggingFaceBio/Carbon-X-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HuggingFaceBio/Carbon-X-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HuggingFaceBio/Carbon-X-Base

SGLang

How to use HuggingFaceBio/Carbon-X-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HuggingFaceBio/Carbon-X-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HuggingFaceBio/Carbon-X-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HuggingFaceBio/Carbon-X-Base",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use HuggingFaceBio/Carbon-X-Base with Docker Model Runner:
```
docker model run hf.co/HuggingFaceBio/Carbon-X-Base
```

Fix EOS override persistence on save and reload

by edbeeching HF Staff - opened 12 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

-0

edbeeching

Hugging Face Biology Research org 12 days ago

Preserve the observed EOS override across save_pretrained() and from_pretrained().

This is a narrow alternative to the broader tokenizer persistence PR: four added lines in tokenizer.py.

Copy the wrapper's effective EOS token into the underlying BPE tokenizer before saving.
Restore EOS from tokenizer_config.json in the existing custom loader, accepting string and legacy dictionary representations. The existing final kwargs merge preserves explicit caller overrides.

Constructor defaults, PAD handling, other attributes, DNA encoding, and vocabulary IDs retain their existing behavior. The custom save/load implementation is retained.

Reproduction

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("HuggingFaceBio/Carbon-X-Base", trust_remote_code=True)
tok.eos_token = "<|im_end|>"
tok.save_pretrained("./checkpoint")

tok = AutoTokenizer.from_pretrained("./checkpoint", trust_remote_code=True)
print(tok.eos_token)  # Expected: <|im_end|>; current main returns <|endoftext|>.

Validation

Three focused tokenizer tests passed on Transformers 5.8.1 and 5.12.1:

Reproduce the reset on the original tokenizer.
Verify the fix preserves <|im_end|> / ID 151645, serializes the EOS setting, respects explicit reload overrides, and leaves the vocabulary and representative DNA encodings unchanged.
Check the unchanged default EOS and loading an EOS stored as a legacy AddedToken dictionary.

This PR has tokenizer-only validation. The SFT GPU validation currently running targets the broader PR #1.

Fix EOS override persistence on save and reload9c20b244

edbeeching changed pull request status to closed 12 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment