README.md

---
library_name: transformers
tags:
- unsloth
license: apache-2.0
datasets:
- llm-jp/magpie-sft-v1.0
language:
- ja
base_model:
- google/gemma-2-9b
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description
gemma-2-9b-nyan100

gemma-2-9b-nyan100 は、Google の Gemma-2-9b を基に、日本語の指示追従タスクに特化して微調整されたモデルです。本モデルは、特に日本語での指示応答や対話生成、文書要約などのタスクに優れた性能を発揮します。

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** [Hizaneko]
- **Funded by [optional]:** [More Information Needed]
- **Shared by [optional]:** [More Information Needed]
- **Model type:** [指示追従型大規模言語モデル (Instruction-Following LLM)]
- **Language(s) (NLP):** [日本語]
- **License:** [Gemma 利用規約 に従う]
- **Finetuned from model [optional]:** [google/gemma-2-9b]

### Model Sources [optional]

<!-- Provide the basic links for the model. -->

- **Repository:** [More Information Needed]
- **Paper [optional]:** [More Information Needed]
- **Demo [optional]:** [More Information Needed]

## Uses

!pip uninstall unsloth -y
!pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

# Google Colab のデフォルトで入っているパッケージをアップグレード（Moriyasu さんありがとうございます）
!pip install --upgrade torch
!pip install --upgrade xformers

# notebookでインタラクティブな表示を可能とする（ただし、うまく動かない場合あり）
# Google Colabでは実行不要
!pip install ipywidgets --upgrade

# Install Flash Attention 2 for softcapping support
import torch
if torch.cuda.get_device_capability()[0] >= 8:
    !pip install --no-deps packaging ninja einops "flash-attn>=2.6.3"

# Hugging Face Token を指定
#HF_TOKEN = "" #@param {type:"string"}

# あるいはGoogle Colab シークレットを使う場合、左のサイドバーより🔑マークをクリック
# HF_TOKEN という名前で Value に Hugging Face Token を入れてください。
# ノートブックからのアクセスのトグルをオンにし、下記の二行のコードのコメントアウトを外してください。
from google.colab import userdata
HF_TOKEN=userdata.get('HF_TOKEN')

# google/gemma-2-9bを4bit量子化のqLoRA設定でロード。

from unsloth import FastLanguageModel
import torch
#max_seq_length = 512 # unslothではRoPEをサポートしているのでコンテキスト長は自由に設定可能
max_seq_length = 1024
dtype = None # Noneにしておけば自動で設定
load_in_4bit = True # 今回は9Bモデルを扱うためTrue

# HFからモデルリポジトリをダウンロード
!huggingface-cli login --token $HF_TOKEN
!huggingface-cli download google/gemma-2-9b --local-dir gemma-2-9b/
model_id = "./gemma-2-9b"
new_model_id = "gemma-2-9b-nyan100" #Fine-Tuningしたモデルにつけたい名前
# FastLanguageModel インスタンスを作成
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name=model_id,
    dtype=dtype,
    load_in_4bit=load_in_4bit,
    trust_remote_code=True,
)

# SFT用のモデルを用意
model = FastLanguageModel.get_peft_model(
    model,
    r = 32,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 32,
    lora_dropout = 0.05,
    #lora_dropout = 0.1,
    bias = "none",
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    use_rslora = False,
    loftq_config = None,
    max_seq_length = max_seq_length,
)

from datasets import load_dataset

# データセットのロード
dataset_name = "llm-jp/magpie-sft-v1.0"
dataset = load_dataset(dataset_name)

# データセットの10分の1を使用（train split前提）
train_length = len(dataset["train"])
#dataset["train"] = dataset["train"].select(range(train_length // 10))
dataset["train"] = dataset["train"].select(range(train_length // 100))

# フォーマット整形関数の定義
def format_dataset(examples):
    conversations = examples["conversations"]  # conversationsカラムを取得
    user_inputs = []
    assistant_outputs = []

    for turn in conversations:
        if turn["role"] == "user":
            user_inputs.append(turn["content"])
        elif turn["role"] == "assistant":
            assistant_outputs.append(turn["content"])

    input_text = " ".join(user_inputs)     # ユーザー発話を結合
    output_text = " ".join(assistant_outputs)  # アシスタント発話を結合

    return {
        "text": input_text,   # 入力部分
        "output": output_text # 出力部分
    }

# データセットを整形
formatted_dataset = dataset.map(
    format_dataset,
    num_proc=4,
    remove_columns=["conversations"]
)

# 結果の表示
print(formatted_dataset)

# プロンプトフォーマットの定義
prompt = """### 指示
{}
### 回答
{}"""

EOS_TOKEN = tokenizer.eos_token  # トークナイザーのEOSトークン

# プロンプト生成関数
def formatting_prompts_func(examples):
    input_text = examples["text"]
    output_text = examples["output"]
    formatted_text = prompt.format(input_text, output_text) + EOS_TOKEN
    return {"formatted_text": formatted_text}

# プロンプト適用
final_dataset = formatted_dataset.map(
    formatting_prompts_func,
    num_proc=4
)

from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset=final_dataset["train"],
    max_seq_length = max_seq_length,
    dataset_text_field="formatted_text",
    packing = False,
    args = TrainingArguments(
        per_device_train_batch_size = 2,
        gradient_accumulation_steps = 4,
        num_train_epochs = 1,
        logging_steps = 10,
        warmup_steps = 10,
        save_steps=100,
        save_total_limit=2,
        max_steps=-1,
        learning_rate = 2e-4,
        #learning_rate = 1e-4,
        fp16 = not is_bfloat16_supported(),
        bf16 = is_bfloat16_supported(),
        group_by_length=True,
        seed = 3407,
        output_dir = "outputs",
        report_to = "none",
    ),
)

trainer_stats = trainer.train()

# ELYZA-tasks-100-TVの読み込み。事前にファイルをアップロードしてください
# データセットの読み込み。
# omnicampusの開発環境では、左にタスクのjsonlをドラッグアンドドロップしてから実行。
import json
datasets = []
with open("/content/elyza-tasks-100-TV_0.jsonl", "r") as f:
    item = ""
    for line in f:
      line = line.strip()
      item += line
      if item.endswith("}"):
        datasets.append(json.loads(item))
        item = ""

# 学習したモデルを用いてタスクを実行
from tqdm import tqdm

# 推論するためにモデルのモードを変更
FastLanguageModel.for_inference(model)

results = []
for dt in tqdm(datasets):
  input = dt["input"]

  #prompt = f"""### 指示\n{input}\n### 回答\n"""
  prompt = f"""### 指示\n{input} 簡潔に回答してください \n### 回答\n"""

  inputs = tokenizer([prompt], return_tensors = "pt").to(model.device)

  outputs = model.generate(**inputs, max_new_tokens = 512, use_cache = True, do_sample=False, repetition_penalty=1.2)
  prediction = tokenizer.decode(outputs[0], skip_special_tokens=True).split('\n### 回答')[-1]

  results.append({"task_id": dt["task_id"], "input": input, "output": prediction})

# jsonlで保存
with open(f"{new_model_id}_output.jsonl", 'w', encoding='utf-8') as f:
    for result in results:
        json.dump(result, f, ensure_ascii=False)
        f.write('\n')
        
#モデルとトークナイザーをHugging Faceにアップロード。
# 一旦privateでアップロードしてください。
# 最終成果物が決まったらpublicにするようお願いします。
# 現在公開しているModel_Inference_Template.ipynbはunslothを想定していないためそのままでは動かない可能性があります。
model.push_to_hub_merged(
    new_model_id,
    tokenizer=tokenizer,
    save_method="lora",
    token=HF_TOKEN,
    private=True
)

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

[More Information Needed]

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

<!-- This section is meant to convey both technical and sociotechnical limitations. -->

[More Information Needed]

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

## Training Details

### Training Data
データセット: llm-jp/magpie-sft-v1.0
データ量: 約50,000件の日本語サンプルのうちランダムに抽出した５０００件


[More Information Needed]

### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing [optional]

[More Information Needed]


#### Training Hyperparameters
LoRA 設定:
	r=32
	lora_alpha=32
	lora_dropout=0.05
	バッチサイズ: デバイスごとに 2
	勾配累積ステップ: 4
	学習率: 2e-4
	学習エポック数: 1

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary



## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** [NVIDIA L4]
- **Hours used:** [約１時間]
- **Cloud Provider:** [More Information Needed]
- **Compute Region:** [More Information Needed]
- **Carbon Emitted:** [More Information Needed]

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

[More Information Needed]

#### Software

[More Information Needed]

## Citation [optional]

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

**BibTeX:**

[More Information Needed]

**APA:**

[More Information Needed]

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[More Information Needed]

## Model Card Contact

[More Information Needed]