Instructions to use pfnet/Qwen2.5-1.5B-pfn-qfin with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pfnet/Qwen2.5-1.5B-pfn-qfin with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pfnet/Qwen2.5-1.5B-pfn-qfin")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("pfnet/Qwen2.5-1.5B-pfn-qfin")
model = AutoModelForCausalLM.from_pretrained("pfnet/Qwen2.5-1.5B-pfn-qfin", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pfnet/Qwen2.5-1.5B-pfn-qfin with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pfnet/Qwen2.5-1.5B-pfn-qfin"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/Qwen2.5-1.5B-pfn-qfin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/pfnet/Qwen2.5-1.5B-pfn-qfin

SGLang

How to use pfnet/Qwen2.5-1.5B-pfn-qfin with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pfnet/Qwen2.5-1.5B-pfn-qfin" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/Qwen2.5-1.5B-pfn-qfin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pfnet/Qwen2.5-1.5B-pfn-qfin" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/Qwen2.5-1.5B-pfn-qfin",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use pfnet/Qwen2.5-1.5B-pfn-qfin with Docker Model Runner:
```
docker model run hf.co/pfnet/Qwen2.5-1.5B-pfn-qfin
```

Qwen2.5-1.5B-pfn-qfin

Model Description

Qwen2.5-1.5B-pfn-qfin is an fine-tuned model based on Qwen/Qwen2.5-1.5B. This is the base model, which is good at generating continuous sentences. Qwen2.5-1.5B-pfn-qfin is fine-tuned on about 400M tokens from multiple special datasets generated by Preferred Networks, which is clear to use for commercial usage. The fine-tuned were carried out at a 2048 context length. This model is released under PLaMo Community License.

Benchmarking

The benchmark score is obtained using Japanese Language Model Financial Evaluation Harness For the benchmark, 0-shot and default prompts are used.

Task	Metric	Qwen2.5-1.5B	Ours
chabsa	f1	0.7269	0.7578
cma_basics	acc	0.3684	0.3947
cpa_audit	acc	0.1382	0.2111
fp2	acc	0.4035	0.4386
security_sales_1	acc	0.2463	0.2421
----------------	------	------	------
OVER ALL		0.3767	0.4089

Usage

Install the required libraries as follows:

>>> python -m pip install "transformers>=4.37.0"

Execute the following python code:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("pfnet/Qwen2.5-1.5B-pfn-qfin", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("pfnet/Qwen2.5-1.5B-pfn-qfin", device_map="auto", trust_remote_code=True)
text = "日本銀行は"
input_ids = tokenizer(text, return_tensors="pt").input_ids
with torch.no_grad():
  generated_tokens = model.generate(
      inputs=input_ids.to(model.device),
      max_new_tokens=32,
      do_sample=True,
      top_k=50,
      top_p=0.95,
      temperature=1.0,
      pad_token_id=tokenizer.pad_token_id,
      bos_token_id=tokenizer.bos_token_id,
      eos_token_id=tokenizer.eos_token_id
  )[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text)

Bias, Risks, and Limitations

Qwen2.5-1.5B-pfn-qfin is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, Qwen2.5-1.5B-pfn-qfin’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. This model is not designed for legal, tax, investment, financial, or other advice. Therefore, before deploying any applications of Qwen2.5-1.5B-pfn-qfin, developers should perform safety testing and tuning tailored to their specific applications of the model.