Instructions to use mkd-hossain/Keural-Cortex-8B-64K with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use mkd-hossain/Keural-Cortex-8B-64K with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="mkd-hossain/Keural-Cortex-8B-64K")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("mkd-hossain/Keural-Cortex-8B-64K")
model = AutoModelForCausalLM.from_pretrained("mkd-hossain/Keural-Cortex-8B-64K", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use mkd-hossain/Keural-Cortex-8B-64K with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "mkd-hossain/Keural-Cortex-8B-64K"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-hossain/Keural-Cortex-8B-64K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/mkd-hossain/Keural-Cortex-8B-64K

SGLang

How to use mkd-hossain/Keural-Cortex-8B-64K with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "mkd-hossain/Keural-Cortex-8B-64K" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-hossain/Keural-Cortex-8B-64K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "mkd-hossain/Keural-Cortex-8B-64K" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "mkd-hossain/Keural-Cortex-8B-64K",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use mkd-hossain/Keural-Cortex-8B-64K with Docker Model Runner:
```
docker model run hf.co/mkd-hossain/Keural-Cortex-8B-64K
```

Keural Cortex 8B — 64K context (Phase 5 final)

Korean-focused continued pretraining of Qwen/Qwen3-8B-Base with the context window extended from 32,768 to 65,536 tokens, by MKD Co., Ltd.

This is the completed Phase 5 model: 41B tokens of CPT followed by 2.5B tokens of context extension. It is a base model — no instruction tuning.


Parameters	8.19 B
Context length	65,536 (native, YaRN factor 2.0)
CPT tokens	41.0 B
Context-extension tokens	2.50 B
Precision	bfloat16
Completed	2026-09-12

Evaluation

All measurements use lm-evaluation-harness 0.4.12 / transformers 4.55.2 in bfloat16, with settings identical to the Qwen3-8B-Base baseline, so the comparisons are like-for-like. Two reference points are shown: step10000 is the CPT checkpoint this phase started from, so it isolates what context extension changed; base is Qwen3-8B-Base.

Korean — context extension improved Korean

Benchmark	Base	step10000	this model	vs step10000
KMMLU	53.91	53.57	54.32	+0.75 ✱
HaeRae	63.06	64.44	64.62	+0.18
KoBEST BoolQ	57.05	64.25	63.46	−0.78
KoBEST COPA	72.30	70.20	70.20	0.00
KoBEST HellaSwag	58.80	59.60	59.80	+0.20
KoBEST SentiNeg	83.88	81.36	91.69	+10.33 ✱
Mean				+1.78 (vs base +2.52)

✱ significant at 95%. Across KMMLU's 45 subtasks: 31 improved, 12 regressed. Three of four subject groups rose (STEM +0.96, applied science +1.16, other +0.75; humanities −0.58).

KMMLU 54.32 is above the base model's 53.91 — the first Korean knowledge gain in this project, and it came from 2.5B tokens of context-extension data rather than the preceding 41B tokens of CPT. See Why Korean improved here below.

English — no regression

Benchmark	Base	step10000	this model	vs step10000
MMLU	74.70	74.02	73.79	−0.23
GSM8K (1,319 samples)	84.69	82.34	82.71	+0.38
ARC-Challenge	57.25	59.64	58.79	−0.85
HellaSwag	78.59	78.12	78.40	+0.28
WinoGrande	72.38	72.61	72.77	+0.16
Mean				−0.05

No English result is statistically significant. Context extension is the operation most likely to damage short-context behaviour; it did not.

Long context — needle-in-a-haystack

A random 6-digit code is buried at a given depth in a document of a given length; the model must reproduce it. Measured on CPU via llama.cpp (Q8_0):

context	depth 10%	depth 50%	depth 90%
4,096	PASS	PASS	PASS
16,384	PASS	PASS	PASS
32,768	PASS	PASS	PASS
65,536	PASS	PASS	PASS

12/12 retrieved, measured on this exact checkpoint.

The 32,768 and 65,536 rows are beyond what the base model could do at all, so this is direct evidence the YaRN scaling took effect rather than merely being configured. Retrieval is exact even at 90% depth.

Probe: eval/needle_cpu.py. It is framed as pure completion rather than question-answering (a base model has no instruction tuning and will continue the document instead of answering), and scored through llama.cpp's HTTP API rather than the CLI (both CLI tools echo the prompt, which contains the needle, so stdout-based scoring would pass on echo alone). Positive and negative controls are asserted before any grid is believed.

Rollback gate: PASS

The project gate is Korean up and English within −0.5 pt. Result: Korean +1.78, English −0.05, long context verified. This model ships rather than the pre-extension 32K checkpoint.

Why Korean improved here

The preceding 41B-token CPT did not improve Korean knowledge — KMMLU was negative at every checkpoint, and the gains were lexical (HaeRae rare_word +3.70, loan_word +2.96) rather than factual. Its Korean data was three web-crawl sources.

This phase used 2.5B tokens that are 58% Korean and 58% AI Hub — curated books and the modu corpus rather than web crawl. Same model, same recipe, same learning rate, 16× fewer tokens, and Korean knowledge moved. The HaeRae subtasks track the corpus: history +3.19 (curated books contain history) against general_knowledge −3.41.

The practical lesson, on this project's own evidence: for Korean knowledge the lever is data composition, not token volume.

⚠️ What this model is and is not

It is a base model. Qwen3-8B-Base has no instruction following, no chat format, no tool calling and no thinking mode; continued pretraining adds none of them. It continues text rather than answering instructions. Post-training (SFT/DPO) has not been done.

The chat_template.jinja is inherited from the base tokenizer and does not mean the model is chat-tuned. Driven through it, the model will imitate assistant phrasing because Qwen3-8B-Base absorbed large amounts of assistant text during its 36T-token pretraining — that is imitation, not instruction tuning.

It has not been compared against Qwen3-8B-Instruct. Every number here is against Qwen3-8B-Base — a base-to-base comparison. Comparing a base model to an instruct model is only meaningful on knowledge benchmarks, and has not been done.

The Korean gain is real but modest. KMMLU +0.41 over base is a genuine, statistically significant improvement, not a transformation. This is not a model that beats Qwen3-8B-Instruct, and it is not intended as one.

Benchmark decontamination has not been run. Treat any benchmark number from this project as provisional until n-gram overlap against the evaluation suites has been checked.

Training

Phase 4 — CPT (41B tokens)

Full-parameter FSDP on 4× H200, 18 days 10 hours, zero restarts. Sequence length 4,096, global batch 3,145,728 tokens/step, peak LR 1.0e-05 on a WSD schedule, AdamW (β 0.9/0.95, wd 0.1), grad clip 1.0 (never triggered — max observed norm 0.48). Corpus: 38.784B tokens across 19 sources, 42.45% Korean.

Phase 5 — context extension (2.5B tokens)

Started from step_0010000 of the CPT rather than the final checkpoint — that checkpoint measured best on every Korean aggregation, and being mid-stable-phase (undecayed) it is the more natural handoff to a phase that applies its own WSD schedule.

Parameter	Value
RoPE scaling	YaRN, factor 2.0 → 65,536 positions
Sequence length	65,536
Global batch	3,145,728 tokens/step (1 × 65,536 × 4 GPU × 12 accum)
Steps	795
Peak LR	1.0e-05, WSD, min ratio 0.1
Throughput	10.3k tok/s
Runtime	~2d 23h, zero crashes

Data: 2.501B tokens, 12 sources, every document ≥32,768 tokens. 58.1% Korean, and notably 58% of the total is AI Hub (aihub_modu 0.899B, aihub_books 0.553B) — curated, edited Korean rather than web crawl.

Note on the loss curve: training loss sat flat at ~1.94 throughout Phase 5, which in isolation looks like nothing was learned. The evaluations above show otherwise — Korean improved and 64K retrieval works. Average token loss is simply not the right instrument for either positional behaviour or a data-composition shift, which is why the gate is defined on evaluations rather than on loss.

A required implementation detail: HF computes the loss by materializing the full seq_len × vocab logits tensor and upcasting to fp32 — 37.09 GiB at seq_len 65,536, which OOMs with micro_bsz already at 1. Phase 5 used liger-kernel's fused linear cross-entropy, which never allocates that tensor.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

m = "mkd-hossain/Keural-Cortex-8B-64K"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, torch_dtype="bfloat16", device_map="cuda")

# Base model: complete text, do not instruct.
out = model.generate(**tok("대한민국의 수도는", return_tensors="pt").to("cuda"), max_new_tokens=64)
print(tok.decode(out[0], skip_special_tokens=True))

GGUF / llama.cpp

config.json includes rope_scaling.original_max_position_embeddings: 32768. transformers infers that field, but llama.cpp's converter requires it and fails with KeyError otherwise — it is present here so conversion works:

python3 convert_hf_to_gguf.py <this_model> --outfile out-f16.gguf --outtype f16
./llama-quantize out-f16.gguf out-q8_0.gguf Q8_0 8

On AMX-capable CPUs, Q8_0 outperforms Q4_K_M despite being larger (AMX-INT8 hardware path). Build llama.cpp with -DGGML_NATIVE=ON or those kernels are not compiled in.

Related models

Model	Tokens	Context	Purpose
this	41B + 2.5B	65,536	Phase 5 final
`Keural-Cortex-8B-Base`	41B	32,768	CPT final
`Keural-Cortex-8B-step10000`	31.5B	32,768	best Korean checkpoint; Phase 5 started here
`Keural-Cortex-8B-step5000`	15.7B	32,768	intermediate
`Keural-Cortex-8B-pilot`	3B	32,768	recipe validation

Full audit record — configuration, training dynamics, corpus provenance, evaluation and findings — at github.com/MKD-CORP/Keural-Cortex-8B under CPT_details/.

License

Apache 2.0, inherited from Qwen/Qwen3-8B-Base. Derivative work of Qwen3-8B-Base by Alibaba Cloud; attribution required.

Intended use

Research, long-context experimentation, and as a base for further fine-tuning. Not a user-facing assistant — it has no instruction tuning — and not a basis for capability claims until decontamination is complete.

Downloads last month: -

Safetensors

Model size

2B params

Tensor type

BF16

Model tree for mkd-hossain/Keural-Cortex-8B-64K

Base model

Qwen/Qwen3-8B-Base

Finetuned

(563)

this model