K2-geoscience-7B-4bit-mlx

This is a 4-bit MLX quantization of K2 (daven3/k2), the first open-source foundation language model for the geoscience domain, converted for fast local inference on Apple Silicon.

K2 is a 7B model built by further-pretraining LLaMA-7B on ~5.5B tokens of geoscience literature (earth-science papers and Wikipedia) and instruction-tuning with the GeoSignal dataset. See the original work for full details:

Deng et al., K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization, WSDM 2024. Paper: https://arxiv.org/abs/2306.05064 · Code: https://github.com/davendw49/k2

Conversion details

• Method: mlx_lm.convert with -q --q-bits 4
• Result: ~4.5 bits per weight
• Precision of base load: float16
• Converted on: Apple Silicon (MLX)
• Tokenizer: the daven3/k2 repo does not ship a tokenizer. K2 uses the unmodified LLaMA-7B SentencePiece tokenizer (vocab size 32000), so the standard LLaMA tokenizer files are bundled here for a self-contained model. Credit to the LLaMA authors for the tokenizer.

Usage

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("xquantize/K2-geoscience-7B-4bit-mlx")

# K2 (this release) is a BASE model — prompt it with completion-style text,
# not chat turns.
prompt = "Plate tectonics is the scientific theory that"
print(generate(model, tokenizer, prompt=prompt, max_tokens=120, verbose=True))

Note on usage style

This is the base / further-pretrained K2 model, not an instruction-tuned chat model. It continues text rather than answering questions conversationally. For best results, give it a sentence to continue (e.g. "P-waves and S-waves differ in that...") rather than a chat-style question. Output can be repetitive, as is typical for 7B base models of this generation.

Performance

On an Apple Silicon Mac, this 4-bit build runs at ~34 tokens/sec with a peak memory footprint of ~4 GB, making a geoscience-specialized 7B usable on consumer hardware.

Limitations

K2 is a 2023/2024-era LLaMA-7B derivative; its general fluency and knowledge reflect that generation, not current frontier models. It may produce inaccurate or outdated statements and should not be relied on for authoritative scientific or safety-critical decisions. Quantization to 4 bits may introduce additional minor quality degradation relative to the full-precision model.

License & attribution

The base model K2 is distributed under the LLaMA license (LLaMA-2 community license terms apply to redistribution of LLaMA-derived weights). All credit for the model belongs to the original K2 authors (Deng et al., davendw49/k2). This repository only provides an MLX-format 4-bit quantization for local inference and claims no ownership of the underlying model.