ryujis/LLM-jp-4-33B-GGUF

⸻

license: apache-2.0 base_model:

• llm-jp/llm-jp-4-33b-base library_name: llama.cpp tags:
• gguf
• quantized
• japanese
• llm-jp
• llama.cpp
• ollama language:
• ja
• en

⸻

LLM-jp-4-33B-Base GGUF — Q4_K_M

This repository provides an unofficial GGUF Q4_K_M quantization of:

llm-jp/llm-jp-4-33b-base

The original model was developed by the Research and Development Center for Large Language Models at the National Institute of Informatics (NII), Japan / LLM-jp.

This quantized model is not an official quantization released by LLM-jp.

Model

• Base model: llm-jp/llm-jp-4-33b-base
• Model family: LLM-jp-4
• Parameters: approximately 33.2B
• Architecture: Dense Transformer / LlamaForCausalLM
• Context length: 65,536 tokens
• Format: GGUF
• Quantization: Q4_K_M
• Original precision: BF16
• Languages: Japanese / English
• License: Apache License 2.0

File

llm-jp-4-33b-base-Q4_K_M.gguf

About this model

LLM-jp-4-33B Base is a pretrained language model developed by LLM-jp.

The Base model has undergone pre-training and mid-training, but is not a post-trained instruction-following model.

Users looking primarily for conversational or instruction-following behavior should also consider the post-trained LLM-jp-4 models released by the original developers.

Ollama

The model can be run directly from Hugging Face with recent versions of Ollama:

ollama run hf.co/ryujis/LLM-jp-4-33B-GGUF:Q4_K_M

llama.cpp

With a recent llama.cpp installation:

llama cli -hf ryujis/LLM-jp-4-33B-GGUF:Q4_K_M

To start an OpenAI-compatible server:

llama serve -hf ryujis/LLM-jp-4-33B-GGUF:Q4_K_M

Download

Using the Hugging Face CLI:

hf download ryujis/LLM-jp-4-33B-GGUF
llm-jp-4-33b-base-Q4_K_M.gguf

Quantization

This repository contains a Q4_K_M GGUF conversion intended to reduce memory requirements while retaining practical model quality.

This is a community-created quantization and has not been produced or endorsed by the original LLM-jp developers.

Original model

Original repository:

llm-jp/llm-jp-4-33b-base

Please refer to the original model card for architecture details, training data information, evaluation results, risks, limitations, and citation information.

License

The original llm-jp-4-33b-base model is distributed under the Apache License 2.0.

This GGUF quantization follows the licensing terms of the original model.

Disclaimer

This is an unofficial community quantization.

The quantization process may alter model behavior or output quality compared with the original BF16 model. Users should independently evaluate outputs for their intended use.