Instructions to use pfnet/nekomata-7b-pfn-qfin-inst-merge with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use pfnet/nekomata-7b-pfn-qfin-inst-merge with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="pfnet/nekomata-7b-pfn-qfin-inst-merge", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use pfnet/nekomata-7b-pfn-qfin-inst-merge with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "pfnet/nekomata-7b-pfn-qfin-inst-merge"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/nekomata-7b-pfn-qfin-inst-merge",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/pfnet/nekomata-7b-pfn-qfin-inst-merge

SGLang

How to use pfnet/nekomata-7b-pfn-qfin-inst-merge with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "pfnet/nekomata-7b-pfn-qfin-inst-merge" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/nekomata-7b-pfn-qfin-inst-merge",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "pfnet/nekomata-7b-pfn-qfin-inst-merge" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "pfnet/nekomata-7b-pfn-qfin-inst-merge",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use pfnet/nekomata-7b-pfn-qfin-inst-merge with Docker Model Runner:
```
docker model run hf.co/pfnet/nekomata-7b-pfn-qfin-inst-merge
```

nekomata-7b-pfn-qfin-inst-merge

Model Description

nekomata-7b-pfn-qfin-inst-merge is a merged model using rinna/nekomata-7b, rinna/nekomata-7b-instruction, and pfnet/nekomata-7b-pfn-qfin. This is the instruction model, which is good at generating answers for instructions. This model is released under Tongyi Qianwen LICENSE AGREEMENT.

You should also see: pfnet/nekomata-14b-pfn-qfin-inst-merge

Usage

Install the required libraries as follows:

>>> python -m pip install numpy sentencepiece torch transformers accelerate transformers_stream_generator tiktoken einops

Execute the following python code:

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", trust_remote_code=True)

# Use GPU with bf16 (recommended for supported devices)
# model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", device_map="auto", trust_remote_code=True, bf16=True)

# Use GPU with fp16
# model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", device_map="auto", trust_remote_code=True, fp16=True)

# Use GPU with fp32
# model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", device_map="auto", trust_remote_code=True, fp32=True)

# Use CPU
# model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", device_map="cpu", trust_remote_code=True)

# Automatically select device and precision
model = AutoModelForCausalLM.from_pretrained("pfnet/nekomata-7b-pfn-qfin-inst-merge", device_map="auto", trust_remote_code=True)

text = """以下は、タスクを説明する指示と、文脈のある入力の組み合わせです。要求を適切に満たす応答を書きなさい。

### 指示:
次の質問に答えてください。

### 入力:
デリバティブ取引のリスク管理について教えてください。

### 応答:"""
input_ids = tokenizer(text, return_tensors="pt").input_ids.to(model.device)
with torch.no_grad():
  generated_tokens = model.generate(
      inputs=input_ids,
      max_new_tokens=512,
      do_sample=False,
      temperature=1.0,
      repetition_penalty=1.1,
      top_k=50,
      pad_token_id=tokenizer.pad_token_id,
      bos_token_id=tokenizer.bos_token_id,
      eos_token_id=tokenizer.eos_token_id
  )[0]
generated_text = tokenizer.decode(generated_tokens)
print(generated_text.split("### 応答:")[1])
# デリバティブ取引では、原資産の価格変動が大きく影響します。そのため、リスク管理においては、原資産の価格変動リスクを適切にコントロールすることが重要となります。具体的には、ポジションサイズやレバレッジの制限、ヘッジ手法の活用などが挙げられます。また、市場情報の収集・分析を行い、適切なタイミングでポジションを調整することも大切です。さらに、取引先の信用リスクにも注意が必要であり、相手方の財務状況や信用度などを確認し、必要に応じて保証や担保を設定することが求められます。以上のようなリスク管理を行うことで、デリバティブ取引における損失リスクを最小化することができます。ただし、リスク管理はあくまで自己責任であるため、十分な知識と経験を持つことが望まれます。また、金融機関等の専門家によるアドバイスを受けることも有効です。以上が、デリバティブ取引におけるリスク管理の概要です。詳細については、金融庁や日本証券業協会などの公式サイトをご参照ください。

Model Details

Model size: 7b
Context length: 2048
Developed by: Preferred Networks, Inc
Model type: Causal decoder-only
Language(s): Japanese and English
License: Tongyi Qianwen LICENSE AGREEMENT

Bias, Risks, and Limitations

nekomata-7b-pfn-qfin-inst-merge is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, nekomata-7b-pfn-qfin-inst-merge’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. This model is not designed for legal, tax, investment, financial, or other advice. Therefore, before deploying any applications of nekomata-7b-pfn-qfin-inst-merge, developers should perform safety testing and tuning tailored to their specific applications of the model.