Instructions to use ThomasYn/GenomeOcean-4B-v1.2-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ThomasYn/GenomeOcean-4B-v1.2-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ThomasYn/GenomeOcean-4B-v1.2-FP8", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ThomasYn/GenomeOcean-4B-v1.2-FP8", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("ThomasYn/GenomeOcean-4B-v1.2-FP8", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ThomasYn/GenomeOcean-4B-v1.2-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ThomasYn/GenomeOcean-4B-v1.2-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThomasYn/GenomeOcean-4B-v1.2-FP8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/ThomasYn/GenomeOcean-4B-v1.2-FP8

SGLang

How to use ThomasYn/GenomeOcean-4B-v1.2-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ThomasYn/GenomeOcean-4B-v1.2-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThomasYn/GenomeOcean-4B-v1.2-FP8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ThomasYn/GenomeOcean-4B-v1.2-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ThomasYn/GenomeOcean-4B-v1.2-FP8",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use ThomasYn/GenomeOcean-4B-v1.2-FP8 with Docker Model Runner:
```
docker model run hf.co/ThomasYn/GenomeOcean-4B-v1.2-FP8
```

GenomeOcean-4B-v1.2 - FP8 W8A8

This repository contains a FP8 W8A8 quantization of https://huggingface.co/DOEJGI/GenomeOcean-4B-v1.2, a 4.25B-parameter Mistral causal language model for microbial genomic sequences.

Quantization

Method: FP8 E4M3 per-tensor static weights with per-token dynamic FP8 activations; lm_head remains BF16.
Runtime used for validation: vLLM
Source precision: BF16
Release size: 3.99 GiB
Weight conversion: deterministic direct E4M3 conversion with compressed-tensors metadata
Container: repository-local Apptainer image, vLLM build 1acd67a795ebccdf9b9db7697ae9082058301657

Validation

The model was loaded from the release directory and used for real greedy inference inside the same Apptainer environment used by the quantization workflow.

Load succeeded: True
Inference succeeded: True
Prompt: ATGCGATCGATCGATCGATCGATCGATCGATCG

ATG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG TTTG

Stratified Metagenome Proxy Perplexity

Model	PPL [95% CI]	Relative to BF16 [95% CI]
BF16	102.7640 [91.3996, 115.6145]	baseline
FP8	105.2026 [93.6835, 118.3596]	+2.37% [+2.17%, +2.58%]
Q4_K_M	109.3034 [97.4647, 122.7864]	+6.36% [+5.84%, +6.91%]
Q8_0	102.9155 [91.4151, 115.8558]	+0.15% [+0.10%, +0.20%]

This paired benchmark uses the seven local metagenomic assemblies listed in the source v1.2 model card: Antarctic, GRE, Harvard Forest, Mendota, NEON, Oilcane, and Tara. These are officially listed training-data sources, but they are not verified as the unpublished validation split that produced the source card's PPL of 143.77.

Sampling uses fixed seed 20260711, equal allocation of 32 windows per environment, deterministic random byte seeks, 8,192 ACGT bases per window, a minimum 1 MB within-source gap, and 1,024 source-tokenizer tokens per sample. The 224 windows contain 229,152 scored next-token targets. The token corpus SHA-256 is 297c5261ae721b6f0a7083d3e44f92e82e6f17553dc72306ae1c26f2d1b6a083. Confidence intervals use 10,000 paired bootstrap replicates, stratified by environment. This release's measured PPL is 105.2026.

These measurements are conditional in-domain deployment-format regressions, not official validation or held-out generalization estimates. Pairing controls for window difficulty but cannot remove possible interactions between training-set membership and quantization error. The confidence intervals condition on these sampled windows and treat sufficiently separated windows as approximately independent; they do not describe uncertainty over unseen metagenomes. Sampling also favors long, clean ACGT regions, uses equal environment weighting, evaluates only 1,024-token contexts, and does not exercise [CLS], [SEP], [MASK], or scaffold-gap handling. Because safetensors FP8/BF16 and GGUF use different vLLM loading paths, the comparison measures the complete deployed format/runtime path rather than isolated weight-rounding error.

Full per-window and per-environment results are in metagenome_ppl.json; the cross-model comparison and sampling manifest are in metagenome_ppl_summary.json and metagenome_ppl_manifest.json.

Secondary Reference-Genome Proxy

Model	PPL	Relative to BF16
Source BF16	64.9039	baseline
FP8 W8A8	71.8874	+10.76%

This earlier deterministic benchmark uses 40 windows from five reference genomes, 512 tokenizer tokens per window, and 20,440 scored next-token targets. Its token corpus SHA-256 is 125735fd43676e999bf7064c965ffbd1e264179a24acd391368b434dad4f654c. It is retained in proxy_ppl.json for historical comparison; the larger stratified metagenome benchmark above is the primary quantization-quality result.

Usage

The FP8 release is intended for vLLM on FP8-capable NVIDIA GPUs. The GGUF releases are intended for a recent llama.cpp build. GenomeOcean is a base genomic language model, not a chat model; provide DNA sequence prompts rather than chat messages.

See the source model card for training data, special-token semantics, citation, limitations, and the full LBNL BSD license terms. The original LICENSE file is included in this repository.

Downloads last month: 68

Safetensors

Model size

4B params

Tensor type

BF16

F32

F8_E4M3

Model tree for ThomasYn/GenomeOcean-4B-v1.2-FP8

Base model

DOEJGI/GenomeOcean-4B-v1.2

Quantized

(3)

this model