Instructions to use jjee2/lora_recycle with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jjee2/lora_recycle with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="jjee2/lora_recycle",
	filename="Aratron1811__llama-3.1-8B-Instruct-abliterated-comrade/Meta-Llama-3.1-8B-Instruct-abliterated-TQ2_0.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use jjee2/lora_recycle with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jjee2/lora_recycle:TQ2_0
# Run inference directly in the terminal:
llama-cli -hf jjee2/lora_recycle:TQ2_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf jjee2/lora_recycle:TQ2_0
# Run inference directly in the terminal:
llama-cli -hf jjee2/lora_recycle:TQ2_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf jjee2/lora_recycle:TQ2_0
# Run inference directly in the terminal:
./llama-cli -hf jjee2/lora_recycle:TQ2_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf jjee2/lora_recycle:TQ2_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf jjee2/lora_recycle:TQ2_0

Use Docker

docker model run hf.co/jjee2/lora_recycle:TQ2_0

LM Studio
Jan
Ollama
How to use jjee2/lora_recycle with Ollama:
```
ollama run hf.co/jjee2/lora_recycle:TQ2_0
```

Unsloth Studio new

How to use jjee2/lora_recycle with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jjee2/lora_recycle to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for jjee2/lora_recycle to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for jjee2/lora_recycle to start chatting

Docker Model Runner
How to use jjee2/lora_recycle with Docker Model Runner:
```
docker model run hf.co/jjee2/lora_recycle:TQ2_0
```

Lemonade

How to use jjee2/lora_recycle with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jjee2/lora_recycle:TQ2_0

Run and chat with the model

lemonade run user.lora_recycle-TQ2_0

List all available models

lemonade list

jjee2 commited on 25 days ago

Commit

e49a72e

verified ·

1 Parent(s): cdaf1fd

Add chchen__Llama-3.1-8B-Instruct-SAA-800

Browse files

Files changed (19) hide show

.gitattributes +1 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/.gitattributes +36 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/.ipynb_checkpoints/lora_orpo-checkpoint.yaml +44 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/README.md +84 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/adapter_config.json +34 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/adapter_model.safetensors +3 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/all_results.json +22 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/eval_results.json +17 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/lora_orpo.yaml +44 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/special_tokens_map.json +17 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/tokenizer.json +3 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/tokenizer_config.json +2065 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/train_results.json +8 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/trainer_log.jsonl +55 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/trainer_state.json +969 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/training_args.bin +3 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/training_eval_loss.png +0 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/training_loss.png +0 -0
chchen__Llama-3.1-8B-Instruct-SAA-800/training_rewards_accuracies.png +0 -0

.gitattributes CHANGED Viewed

@@ -587,3 +587,4 @@ chchen__Llama-3.1-8B-Instruct-SAA-700/tokenizer.json filter=lfs diff=lfs merge=l
 clembench-playpen__meta-llama-Meta-Llama-3.1-8B-Instruct_SFT_D90001/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 taeyoon12321421__gorani-8b-finetuned/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 Nevil9__PsyLlama/tokenizer.json filter=lfs diff=lfs merge=lfs -text

 clembench-playpen__meta-llama-Meta-Llama-3.1-8B-Instruct_SFT_D90001/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 taeyoon12321421__gorani-8b-finetuned/tokenizer.json filter=lfs diff=lfs merge=lfs -text
 Nevil9__PsyLlama/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+chchen__Llama-3.1-8B-Instruct-SAA-800/tokenizer.json filter=lfs diff=lfs merge=lfs -text

chchen__Llama-3.1-8B-Instruct-SAA-800/.gitattributes ADDED Viewed

	@@ -0,0 +1,36 @@

+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

chchen__Llama-3.1-8B-Instruct-SAA-800/.ipynb_checkpoints/lora_orpo-checkpoint.yaml ADDED Viewed

	@@ -0,0 +1,44 @@

+### model
+model_name_or_path: meta-llama/Llama-3.1-8B-Instruct
+### method
+stage: dpo
+pref_loss: orpo
+do_train: true
+finetuning_type: lora
+lora_target: all
+### dataset
+dataset: bct_non_cot_dpo_800
+dataset_dir: data_private
+template: llama3
+cutoff_len: 1024
+# max_samples: 1000
+overwrite_cache: true
+preprocessing_num_workers: 16
+### output
+output_dir: saves/Llama-3.1-8B-Instruct/lora/saa-800
+logging_steps: 10
+save_steps: 50
+plot_loss: true
+overwrite_output_dir: true
+save_total_limit: 3
+load_best_model_at_end: true
+push_to_hub: true
+hub_model_id: chchen/Llama-3.1-8B-Instruct-SAA-800
+### train
+per_device_train_batch_size: 2
+gradient_accumulation_steps: 8
+learning_rate: 0.000005
+num_train_epochs: 10.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+### eval
+val_size: 0.1
+per_device_eval_batch_size: 2
+evaluation_strategy: steps
+eval_steps: 50

chchen__Llama-3.1-8B-Instruct-SAA-800/README.md ADDED Viewed

	@@ -0,0 +1,84 @@

+---
+base_model: meta-llama/Llama-3.1-8B-Instruct
+library_name: peft
+license: llama3.1
+tags:
+- llama-factory
+- lora
+- trl
+- dpo
+- generated_from_trainer
+model-index:
+- name: Llama-3.1-8B-Instruct-SAA-800
+  results: []
+---
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+# Llama-3.1-8B-Instruct-SAA-800
+This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on the bct_non_cot_dpo_800 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1529
+- Rewards/chosen: -0.0121
+- Rewards/rejected: -0.0692
+- Rewards/accuracies: 0.8125
+- Rewards/margins: 0.0571
+- Logps/rejected: -0.6922
+- Logps/chosen: -0.1209
+- Logits/rejected: -0.3690
+- Logits/chosen: -0.3184
+- Sft Loss: 0.0162
+- Odds Ratio Loss: 1.3676
+## Model description
+More information needed
+## Intended uses & limitations
+More information needed
+## Training and evaluation data
+More information needed
+## Training procedure
+### Training hyperparameters
+The following hyperparameters were used during training:
+- learning_rate: 5e-06
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 8
+- total_train_batch_size: 16
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: cosine
+- lr_scheduler_warmup_ratio: 0.1
+- num_epochs: 10.0
+### Training results
+| Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Sft Loss | Odds Ratio Loss |
+|:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|:--------:|:---------------:|
+| 1.4945        | 1.1111 | 50   | 1.2763          | -0.1228        | -0.1709          | 0.7875             | 0.0481          | -1.7095        | -1.2281      | -0.4548         | -0.3794       | 0.1502   | 11.2609         |
+| 0.2666        | 2.2222 | 100  | 0.2491          | -0.0212        | -0.0729          | 0.8250             | 0.0517          | -0.7288        | -0.2119      | -0.4319         | -0.3666       | 0.0242   | 2.2484          |
+| 0.1014        | 3.3333 | 150  | 0.1632          | -0.0129        | -0.0606          | 0.8125             | 0.0477          | -0.6058        | -0.1292      | -0.3820         | -0.3284       | 0.0168   | 1.4635          |
+| 0.1429        | 4.4444 | 200  | 0.1534          | -0.0121        | -0.0584          | 0.8125             | 0.0463          | -0.5841        | -0.1211      | -0.3818         | -0.3298       | 0.0158   | 1.3752          |
+| 0.1007        | 5.5556 | 250  | 0.1530          | -0.0121        | -0.0641          | 0.8125             | 0.0520          | -0.6407        | -0.1206      | -0.3740         | -0.3235       | 0.0159   | 1.3704          |
+| 0.1385        | 6.6667 | 300  | 0.1534          | -0.0122        | -0.0688          | 0.8000             | 0.0566          | -0.6881        | -0.1217      | -0.3725         | -0.3214       | 0.0161   | 1.3729          |
+| 0.0918        | 7.7778 | 350  | 0.1537          | -0.0122        | -0.0689          | 0.8125             | 0.0567          | -0.6889        | -0.1217      | -0.3698         | -0.3191       | 0.0162   | 1.3742          |
+| 0.0752        | 8.8889 | 400  | 0.1534          | -0.0121        | -0.0690          | 0.8000             | 0.0568          | -0.6896        | -0.1213      | -0.3706         | -0.3195       | 0.0162   | 1.3723          |
+| 0.1052        | 10.0   | 450  | 0.1529          | -0.0121        | -0.0692          | 0.8125             | 0.0571          | -0.6922        | -0.1209      | -0.3690         | -0.3184       | 0.0162   | 1.3676          |
+### Framework versions
+- PEFT 0.12.0
+- Transformers 4.45.2
+- Pytorch 2.3.0
+- Datasets 2.19.0
+- Tokenizers 0.20.0

chchen__Llama-3.1-8B-Instruct-SAA-800/adapter_config.json ADDED Viewed

	@@ -0,0 +1,34 @@

+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "meta-llama/Llama-3.1-8B-Instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.0,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "gate_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/adapter_model.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8c7f819f71972c0e5af945940bf13ecd407f8a5f7f1c83e200fa0cb8fc301005
+size 83945296

chchen__Llama-3.1-8B-Instruct-SAA-800/all_results.json ADDED Viewed

	@@ -0,0 +1,22 @@

+{
+    "epoch": 10.0,
+    "eval_logits/chosen": -0.3184443712234497,
+    "eval_logits/rejected": -0.36903008818626404,
+    "eval_logps/chosen": -0.1208958849310875,
+    "eval_logps/rejected": -0.6922038793563843,
+    "eval_loss": 0.1529119908809662,
+    "eval_odds_ratio_loss": 1.3675727844238281,
+    "eval_rewards/accuracies": 0.8125,
+    "eval_rewards/chosen": -0.012089588679373264,
+    "eval_rewards/margins": 0.05713079497218132,
+    "eval_rewards/rejected": -0.06922038644552231,
+    "eval_runtime": 3.3385,
+    "eval_samples_per_second": 23.963,
+    "eval_sft_loss": 0.016154715791344643,
+    "eval_steps_per_second": 11.982,
+    "total_flos": 8.074648418018918e+16,
+    "train_loss": 0.3428536836306254,
+    "train_runtime": 994.0751,
+    "train_samples_per_second": 7.243,
+    "train_steps_per_second": 0.453
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/eval_results.json ADDED Viewed

	@@ -0,0 +1,17 @@

+{
+    "epoch": 10.0,
+    "eval_logits/chosen": -0.3184443712234497,
+    "eval_logits/rejected": -0.36903008818626404,
+    "eval_logps/chosen": -0.1208958849310875,
+    "eval_logps/rejected": -0.6922038793563843,
+    "eval_loss": 0.1529119908809662,
+    "eval_odds_ratio_loss": 1.3675727844238281,
+    "eval_rewards/accuracies": 0.8125,
+    "eval_rewards/chosen": -0.012089588679373264,
+    "eval_rewards/margins": 0.05713079497218132,
+    "eval_rewards/rejected": -0.06922038644552231,
+    "eval_runtime": 3.3385,
+    "eval_samples_per_second": 23.963,
+    "eval_sft_loss": 0.016154715791344643,
+    "eval_steps_per_second": 11.982
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/lora_orpo.yaml ADDED Viewed

	@@ -0,0 +1,44 @@

+### model
+model_name_or_path: meta-llama/Llama-3.1-8B-Instruct
+### method
+stage: dpo
+pref_loss: orpo
+do_train: true
+finetuning_type: lora
+lora_target: all
+### dataset
+dataset: bct_non_cot_dpo_800
+dataset_dir: data_private
+template: llama3
+cutoff_len: 1024
+# max_samples: 1000
+overwrite_cache: true
+preprocessing_num_workers: 16
+### output
+output_dir: saves/Llama-3.1-8B-Instruct/lora/saa-800
+logging_steps: 10
+save_steps: 50
+plot_loss: true
+overwrite_output_dir: true
+save_total_limit: 3
+load_best_model_at_end: true
+push_to_hub: true
+hub_model_id: chchen/Llama-3.1-8B-Instruct-SAA-800
+### train
+per_device_train_batch_size: 2
+gradient_accumulation_steps: 8
+learning_rate: 0.000005
+num_train_epochs: 10.0
+lr_scheduler_type: cosine
+warmup_ratio: 0.1
+bf16: true
+### eval
+val_size: 0.1
+per_device_eval_batch_size: 2
+evaluation_strategy: steps
+eval_steps: 50

chchen__Llama-3.1-8B-Instruct-SAA-800/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,17 @@

+{
+  "bos_token": {
+    "content": "<|begin_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|eot_id|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<|eot_id|>"
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

chchen__Llama-3.1-8B-Instruct-SAA-800/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,2065 @@

+{
+  "added_tokens_decoder": {
+    "128000": {
+      "content": "<|begin_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128001": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128002": {
+      "content": "<|reserved_special_token_0|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128003": {
+      "content": "<|reserved_special_token_1|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128004": {
+      "content": "<|finetune_right_pad_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128005": {
+      "content": "<|reserved_special_token_2|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128006": {
+      "content": "<|start_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128007": {
+      "content": "<|end_header_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128008": {
+      "content": "<|eom_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128009": {
+      "content": "<|eot_id|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128010": {
+      "content": "<|python_tag|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128011": {
+      "content": "<|reserved_special_token_3|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128012": {
+      "content": "<|reserved_special_token_4|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128013": {
+      "content": "<|reserved_special_token_5|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128014": {
+      "content": "<|reserved_special_token_6|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128015": {
+      "content": "<|reserved_special_token_7|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128016": {
+      "content": "<|reserved_special_token_8|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128017": {
+      "content": "<|reserved_special_token_9|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128018": {
+      "content": "<|reserved_special_token_10|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128019": {
+      "content": "<|reserved_special_token_11|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128020": {
+      "content": "<|reserved_special_token_12|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128021": {
+      "content": "<|reserved_special_token_13|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128022": {
+      "content": "<|reserved_special_token_14|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128023": {
+      "content": "<|reserved_special_token_15|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128024": {
+      "content": "<|reserved_special_token_16|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128025": {
+      "content": "<|reserved_special_token_17|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128026": {
+      "content": "<|reserved_special_token_18|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128027": {
+      "content": "<|reserved_special_token_19|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128028": {
+      "content": "<|reserved_special_token_20|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128029": {
+      "content": "<|reserved_special_token_21|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128030": {
+      "content": "<|reserved_special_token_22|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128031": {
+      "content": "<|reserved_special_token_23|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128032": {
+      "content": "<|reserved_special_token_24|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128033": {
+      "content": "<|reserved_special_token_25|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128034": {
+      "content": "<|reserved_special_token_26|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128035": {
+      "content": "<|reserved_special_token_27|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128036": {
+      "content": "<|reserved_special_token_28|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128037": {
+      "content": "<|reserved_special_token_29|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128038": {
+      "content": "<|reserved_special_token_30|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128039": {
+      "content": "<|reserved_special_token_31|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128040": {
+      "content": "<|reserved_special_token_32|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128041": {
+      "content": "<|reserved_special_token_33|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128042": {
+      "content": "<|reserved_special_token_34|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128043": {
+      "content": "<|reserved_special_token_35|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128044": {
+      "content": "<|reserved_special_token_36|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128045": {
+      "content": "<|reserved_special_token_37|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128046": {
+      "content": "<|reserved_special_token_38|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128047": {
+      "content": "<|reserved_special_token_39|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128048": {
+      "content": "<|reserved_special_token_40|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128049": {
+      "content": "<|reserved_special_token_41|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128050": {
+      "content": "<|reserved_special_token_42|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128051": {
+      "content": "<|reserved_special_token_43|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128052": {
+      "content": "<|reserved_special_token_44|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128053": {
+      "content": "<|reserved_special_token_45|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128054": {
+      "content": "<|reserved_special_token_46|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128055": {
+      "content": "<|reserved_special_token_47|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128056": {
+      "content": "<|reserved_special_token_48|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128057": {
+      "content": "<|reserved_special_token_49|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128058": {
+      "content": "<|reserved_special_token_50|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128059": {
+      "content": "<|reserved_special_token_51|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128060": {
+      "content": "<|reserved_special_token_52|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128061": {
+      "content": "<|reserved_special_token_53|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128062": {
+      "content": "<|reserved_special_token_54|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128063": {
+      "content": "<|reserved_special_token_55|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128064": {
+      "content": "<|reserved_special_token_56|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128065": {
+      "content": "<|reserved_special_token_57|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128066": {
+      "content": "<|reserved_special_token_58|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128067": {
+      "content": "<|reserved_special_token_59|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128068": {
+      "content": "<|reserved_special_token_60|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128069": {
+      "content": "<|reserved_special_token_61|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128070": {
+      "content": "<|reserved_special_token_62|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128071": {
+      "content": "<|reserved_special_token_63|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128072": {
+      "content": "<|reserved_special_token_64|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128073": {
+      "content": "<|reserved_special_token_65|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128074": {
+      "content": "<|reserved_special_token_66|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128075": {
+      "content": "<|reserved_special_token_67|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128076": {
+      "content": "<|reserved_special_token_68|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128077": {
+      "content": "<|reserved_special_token_69|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128078": {
+      "content": "<|reserved_special_token_70|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128079": {
+      "content": "<|reserved_special_token_71|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128080": {
+      "content": "<|reserved_special_token_72|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128081": {
+      "content": "<|reserved_special_token_73|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128082": {
+      "content": "<|reserved_special_token_74|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128083": {
+      "content": "<|reserved_special_token_75|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128084": {
+      "content": "<|reserved_special_token_76|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128085": {
+      "content": "<|reserved_special_token_77|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128086": {
+      "content": "<|reserved_special_token_78|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128087": {
+      "content": "<|reserved_special_token_79|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128088": {
+      "content": "<|reserved_special_token_80|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128089": {
+      "content": "<|reserved_special_token_81|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128090": {
+      "content": "<|reserved_special_token_82|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128091": {
+      "content": "<|reserved_special_token_83|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128092": {
+      "content": "<|reserved_special_token_84|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128093": {
+      "content": "<|reserved_special_token_85|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128094": {
+      "content": "<|reserved_special_token_86|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128095": {
+      "content": "<|reserved_special_token_87|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128096": {
+      "content": "<|reserved_special_token_88|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128097": {
+      "content": "<|reserved_special_token_89|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128098": {
+      "content": "<|reserved_special_token_90|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128099": {
+      "content": "<|reserved_special_token_91|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128100": {
+      "content": "<|reserved_special_token_92|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128101": {
+      "content": "<|reserved_special_token_93|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128102": {
+      "content": "<|reserved_special_token_94|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128103": {
+      "content": "<|reserved_special_token_95|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128104": {
+      "content": "<|reserved_special_token_96|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128105": {
+      "content": "<|reserved_special_token_97|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128106": {
+      "content": "<|reserved_special_token_98|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128107": {
+      "content": "<|reserved_special_token_99|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128108": {
+      "content": "<|reserved_special_token_100|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128109": {
+      "content": "<|reserved_special_token_101|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128110": {
+      "content": "<|reserved_special_token_102|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128111": {
+      "content": "<|reserved_special_token_103|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128112": {
+      "content": "<|reserved_special_token_104|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128113": {
+      "content": "<|reserved_special_token_105|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128114": {
+      "content": "<|reserved_special_token_106|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128115": {
+      "content": "<|reserved_special_token_107|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128116": {
+      "content": "<|reserved_special_token_108|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128117": {
+      "content": "<|reserved_special_token_109|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128118": {
+      "content": "<|reserved_special_token_110|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128119": {
+      "content": "<|reserved_special_token_111|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128120": {
+      "content": "<|reserved_special_token_112|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128121": {
+      "content": "<|reserved_special_token_113|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128122": {
+      "content": "<|reserved_special_token_114|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128123": {
+      "content": "<|reserved_special_token_115|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128124": {
+      "content": "<|reserved_special_token_116|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128125": {
+      "content": "<|reserved_special_token_117|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128126": {
+      "content": "<|reserved_special_token_118|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128127": {
+      "content": "<|reserved_special_token_119|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128128": {
+      "content": "<|reserved_special_token_120|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128129": {
+      "content": "<|reserved_special_token_121|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128130": {
+      "content": "<|reserved_special_token_122|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128131": {
+      "content": "<|reserved_special_token_123|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128132": {
+      "content": "<|reserved_special_token_124|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128133": {
+      "content": "<|reserved_special_token_125|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128134": {
+      "content": "<|reserved_special_token_126|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128135": {
+      "content": "<|reserved_special_token_127|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128136": {
+      "content": "<|reserved_special_token_128|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128137": {
+      "content": "<|reserved_special_token_129|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128138": {
+      "content": "<|reserved_special_token_130|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128139": {
+      "content": "<|reserved_special_token_131|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128140": {
+      "content": "<|reserved_special_token_132|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128141": {
+      "content": "<|reserved_special_token_133|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128142": {
+      "content": "<|reserved_special_token_134|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128143": {
+      "content": "<|reserved_special_token_135|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128144": {
+      "content": "<|reserved_special_token_136|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128145": {
+      "content": "<|reserved_special_token_137|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128146": {
+      "content": "<|reserved_special_token_138|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128147": {
+      "content": "<|reserved_special_token_139|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128148": {
+      "content": "<|reserved_special_token_140|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128149": {
+      "content": "<|reserved_special_token_141|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128150": {
+      "content": "<|reserved_special_token_142|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128151": {
+      "content": "<|reserved_special_token_143|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128152": {
+      "content": "<|reserved_special_token_144|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128153": {
+      "content": "<|reserved_special_token_145|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128154": {
+      "content": "<|reserved_special_token_146|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128155": {
+      "content": "<|reserved_special_token_147|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128156": {
+      "content": "<|reserved_special_token_148|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128157": {
+      "content": "<|reserved_special_token_149|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128158": {
+      "content": "<|reserved_special_token_150|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128159": {
+      "content": "<|reserved_special_token_151|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128160": {
+      "content": "<|reserved_special_token_152|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128161": {
+      "content": "<|reserved_special_token_153|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128162": {
+      "content": "<|reserved_special_token_154|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128163": {
+      "content": "<|reserved_special_token_155|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128164": {
+      "content": "<|reserved_special_token_156|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128165": {
+      "content": "<|reserved_special_token_157|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128166": {
+      "content": "<|reserved_special_token_158|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128167": {
+      "content": "<|reserved_special_token_159|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128168": {
+      "content": "<|reserved_special_token_160|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128169": {
+      "content": "<|reserved_special_token_161|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128170": {
+      "content": "<|reserved_special_token_162|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128171": {
+      "content": "<|reserved_special_token_163|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128172": {
+      "content": "<|reserved_special_token_164|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128173": {
+      "content": "<|reserved_special_token_165|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128174": {
+      "content": "<|reserved_special_token_166|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128175": {
+      "content": "<|reserved_special_token_167|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128176": {
+      "content": "<|reserved_special_token_168|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128177": {
+      "content": "<|reserved_special_token_169|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128178": {
+      "content": "<|reserved_special_token_170|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128179": {
+      "content": "<|reserved_special_token_171|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128180": {
+      "content": "<|reserved_special_token_172|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128181": {
+      "content": "<|reserved_special_token_173|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128182": {
+      "content": "<|reserved_special_token_174|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128183": {
+      "content": "<|reserved_special_token_175|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128184": {
+      "content": "<|reserved_special_token_176|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128185": {
+      "content": "<|reserved_special_token_177|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128186": {
+      "content": "<|reserved_special_token_178|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128187": {
+      "content": "<|reserved_special_token_179|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128188": {
+      "content": "<|reserved_special_token_180|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128189": {
+      "content": "<|reserved_special_token_181|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128190": {
+      "content": "<|reserved_special_token_182|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128191": {
+      "content": "<|reserved_special_token_183|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128192": {
+      "content": "<|reserved_special_token_184|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128193": {
+      "content": "<|reserved_special_token_185|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128194": {
+      "content": "<|reserved_special_token_186|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128195": {
+      "content": "<|reserved_special_token_187|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128196": {
+      "content": "<|reserved_special_token_188|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128197": {
+      "content": "<|reserved_special_token_189|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128198": {
+      "content": "<|reserved_special_token_190|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128199": {
+      "content": "<|reserved_special_token_191|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128200": {
+      "content": "<|reserved_special_token_192|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128201": {
+      "content": "<|reserved_special_token_193|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128202": {
+      "content": "<|reserved_special_token_194|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128203": {
+      "content": "<|reserved_special_token_195|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128204": {
+      "content": "<|reserved_special_token_196|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128205": {
+      "content": "<|reserved_special_token_197|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128206": {
+      "content": "<|reserved_special_token_198|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128207": {
+      "content": "<|reserved_special_token_199|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128208": {
+      "content": "<|reserved_special_token_200|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128209": {
+      "content": "<|reserved_special_token_201|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128210": {
+      "content": "<|reserved_special_token_202|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128211": {
+      "content": "<|reserved_special_token_203|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128212": {
+      "content": "<|reserved_special_token_204|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128213": {
+      "content": "<|reserved_special_token_205|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128214": {
+      "content": "<|reserved_special_token_206|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128215": {
+      "content": "<|reserved_special_token_207|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128216": {
+      "content": "<|reserved_special_token_208|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128217": {
+      "content": "<|reserved_special_token_209|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128218": {
+      "content": "<|reserved_special_token_210|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128219": {
+      "content": "<|reserved_special_token_211|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128220": {
+      "content": "<|reserved_special_token_212|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128221": {
+      "content": "<|reserved_special_token_213|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128222": {
+      "content": "<|reserved_special_token_214|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128223": {
+      "content": "<|reserved_special_token_215|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128224": {
+      "content": "<|reserved_special_token_216|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128225": {
+      "content": "<|reserved_special_token_217|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128226": {
+      "content": "<|reserved_special_token_218|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128227": {
+      "content": "<|reserved_special_token_219|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128228": {
+      "content": "<|reserved_special_token_220|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128229": {
+      "content": "<|reserved_special_token_221|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128230": {
+      "content": "<|reserved_special_token_222|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128231": {
+      "content": "<|reserved_special_token_223|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128232": {
+      "content": "<|reserved_special_token_224|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128233": {
+      "content": "<|reserved_special_token_225|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128234": {
+      "content": "<|reserved_special_token_226|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128235": {
+      "content": "<|reserved_special_token_227|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128236": {
+      "content": "<|reserved_special_token_228|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128237": {
+      "content": "<|reserved_special_token_229|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128238": {
+      "content": "<|reserved_special_token_230|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128239": {
+      "content": "<|reserved_special_token_231|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128240": {
+      "content": "<|reserved_special_token_232|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128241": {
+      "content": "<|reserved_special_token_233|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128242": {
+      "content": "<|reserved_special_token_234|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128243": {
+      "content": "<|reserved_special_token_235|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128244": {
+      "content": "<|reserved_special_token_236|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128245": {
+      "content": "<|reserved_special_token_237|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128246": {
+      "content": "<|reserved_special_token_238|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128247": {
+      "content": "<|reserved_special_token_239|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128248": {
+      "content": "<|reserved_special_token_240|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128249": {
+      "content": "<|reserved_special_token_241|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128250": {
+      "content": "<|reserved_special_token_242|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128251": {
+      "content": "<|reserved_special_token_243|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128252": {
+      "content": "<|reserved_special_token_244|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128253": {
+      "content": "<|reserved_special_token_245|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128254": {
+      "content": "<|reserved_special_token_246|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128255": {
+      "content": "<|reserved_special_token_247|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "<|begin_of_text|>",
+  "chat_template": "{{- bos_token }}\n{%- if custom_tools is defined %}\n    {%- set tools = custom_tools %}\n{%- endif %}\n{%- if not tools_in_user_message is defined %}\n    {%- set tools_in_user_message = true %}\n{%- endif %}\n{%- if not date_string is defined %}\n    {%- set date_string = \"26 Jul 2024\" %}\n{%- endif %}\n{%- if not tools is defined %}\n    {%- set tools = none %}\n{%- endif %}\n\n{#- This block extracts the system message, so we can slot it into the right place. #}\n{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content']|trim %}\n    {%- set messages = messages[1:] %}\n{%- else %}\n    {%- set system_message = \"\" %}\n{%- endif %}\n\n{#- System message + builtin tools #}\n{{- \"<|start_header_id|>system<|end_header_id|>\\n\\n\" }}\n{%- if builtin_tools is defined or tools is not none %}\n    {{- \"Environment: ipython\\n\" }}\n{%- endif %}\n{%- if builtin_tools is defined %}\n    {{- \"Tools: \" + builtin_tools | reject('equalto', 'code_interpreter') | join(\", \") + \"\\n\\n\"}}\n{%- endif %}\n{{- \"Cutting Knowledge Date: December 2023\\n\" }}\n{{- \"Today Date: \" + date_string + \"\\n\\n\" }}\n{%- if tools is not none and not tools_in_user_message %}\n    {{- \"You have access to the following functions. To call a function, please respond with JSON for a function call.\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n{%- endif %}\n{{- system_message }}\n{{- \"<|eot_id|>\" }}\n\n{#- Custom tools are passed in a user message with some extra guidance #}\n{%- if tools_in_user_message and not tools is none %}\n    {#- Extract the first user message so we can plug it in here #}\n    {%- if messages | length != 0 %}\n        {%- set first_user_message = messages[0]['content']|trim %}\n        {%- set messages = messages[1:] %}\n    {%- else %}\n        {{- raise_exception(\"Cannot put tools in the first user message when there's no first user message!\") }}\n{%- endif %}\n    {{- '<|start_header_id|>user<|end_header_id|>\\n\\n' -}}\n    {{- \"Given the following functions, please respond with a JSON for a function call \" }}\n    {{- \"with its proper arguments that best answers the given prompt.\\n\\n\" }}\n    {{- 'Respond in the format {\"name\": function name, \"parameters\": dictionary of argument name and its value}.' }}\n    {{- \"Do not use variables.\\n\\n\" }}\n    {%- for t in tools %}\n        {{- t | tojson(indent=4) }}\n        {{- \"\\n\\n\" }}\n    {%- endfor %}\n    {{- first_user_message + \"<|eot_id|>\"}}\n{%- endif %}\n\n{%- for message in messages %}\n    {%- if not (message.role == 'ipython' or message.role == 'tool' or 'tool_calls' in message) %}\n        {{- '<|start_header_id|>' + message['role'] + '<|end_header_id|>\\n\\n'+ message['content'] | trim + '<|eot_id|>' }}\n    {%- elif 'tool_calls' in message %}\n        {%- if not message.tool_calls|length == 1 %}\n            {{- raise_exception(\"This model only supports single tool-calls at once!\") }}\n        {%- endif %}\n        {%- set tool_call = message.tool_calls[0].function %}\n        {%- if builtin_tools is defined and tool_call.name in builtin_tools %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- \"<|python_tag|>\" + tool_call.name + \".call(\" }}\n            {%- for arg_name, arg_val in tool_call.arguments | items %}\n                {{- arg_name + '=\"' + arg_val + '\"' }}\n                {%- if not loop.last %}\n                    {{- \", \" }}\n                {%- endif %}\n                {%- endfor %}\n            {{- \")\" }}\n        {%- else  %}\n            {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' -}}\n            {{- '{\"name\": \"' + tool_call.name + '\", ' }}\n            {{- '\"parameters\": ' }}\n            {{- tool_call.arguments | tojson }}\n            {{- \"}\" }}\n        {%- endif %}\n        {%- if builtin_tools is defined %}\n            {#- This means we're in ipython mode #}\n            {{- \"<|eom_id|>\" }}\n        {%- else %}\n            {{- \"<|eot_id|>\" }}\n        {%- endif %}\n    {%- elif message.role == \"tool\" or message.role == \"ipython\" %}\n        {{- \"<|start_header_id|>ipython<|end_header_id|>\\n\\n\" }}\n        {%- if message.content is mapping or message.content is iterable %}\n            {{- message.content | tojson }}\n        {%- else %}\n            {{- message.content }}\n        {%- endif %}\n        {{- \"<|eot_id|>\" }}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|start_header_id|>assistant<|end_header_id|>\\n\\n' }}\n{%- endif %}\n",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|eot_id|>",
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|eot_id|>",
+  "padding_side": "right",
+  "split_special_tokens": false,
+  "tokenizer_class": "PreTrainedTokenizerFast"
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/train_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 10.0,
+    "total_flos": 8.074648418018918e+16,
+    "train_loss": 0.3428536836306254,
+    "train_runtime": 994.0751,
+    "train_samples_per_second": 7.243,
+    "train_steps_per_second": 0.453
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/trainer_log.jsonl ADDED Viewed

	@@ -0,0 +1,55 @@

+{"current_steps": 10, "total_steps": 450, "loss": 1.8058, "accuracy": 0.75, "learning_rate": 1.111111111111111e-06, "epoch": 0.2222222222222222, "percentage": 2.22, "elapsed_time": "0:00:21", "remaining_time": "0:15:42"}
+{"current_steps": 20, "total_steps": 450, "loss": 1.7416, "accuracy": 0.7562500238418579, "learning_rate": 2.222222222222222e-06, "epoch": 0.4444444444444444, "percentage": 4.44, "elapsed_time": "0:00:43", "remaining_time": "0:15:26"}
+{"current_steps": 30, "total_steps": 450, "loss": 1.7505, "accuracy": 0.800000011920929, "learning_rate": 3.3333333333333333e-06, "epoch": 0.6666666666666666, "percentage": 6.67, "elapsed_time": "0:01:03", "remaining_time": "0:14:47"}
+{"current_steps": 40, "total_steps": 450, "loss": 1.736, "accuracy": 0.706250011920929, "learning_rate": 4.444444444444444e-06, "epoch": 0.8888888888888888, "percentage": 8.89, "elapsed_time": "0:01:24", "remaining_time": "0:14:22"}
+{"current_steps": 50, "total_steps": 450, "loss": 1.4945, "accuracy": 0.7875000238418579, "learning_rate": 4.998119881260576e-06, "epoch": 1.1111111111111112, "percentage": 11.11, "elapsed_time": "0:01:44", "remaining_time": "0:13:58"}
+{"current_steps": 50, "total_steps": 450, "eval_loss": 1.276250958442688, "epoch": 1.1111111111111112, "percentage": 11.11, "elapsed_time": "0:01:48", "remaining_time": "0:14:25"}
+{"current_steps": 60, "total_steps": 450, "loss": 1.1242, "accuracy": 0.831250011920929, "learning_rate": 4.983095894354858e-06, "epoch": 1.3333333333333333, "percentage": 13.33, "elapsed_time": "0:02:10", "remaining_time": "0:14:07"}
+{"current_steps": 70, "total_steps": 450, "loss": 0.7274, "accuracy": 0.800000011920929, "learning_rate": 4.953138276568462e-06, "epoch": 1.5555555555555556, "percentage": 15.56, "elapsed_time": "0:02:30", "remaining_time": "0:13:38"}
+{"current_steps": 80, "total_steps": 450, "loss": 0.6374, "accuracy": 0.762499988079071, "learning_rate": 4.908427196539701e-06, "epoch": 1.7777777777777777, "percentage": 17.78, "elapsed_time": "0:02:51", "remaining_time": "0:13:11"}
+{"current_steps": 90, "total_steps": 450, "loss": 0.3774, "accuracy": 0.8187500238418579, "learning_rate": 4.849231551964771e-06, "epoch": 2.0, "percentage": 20.0, "elapsed_time": "0:03:12", "remaining_time": "0:12:49"}
+{"current_steps": 100, "total_steps": 450, "loss": 0.2666, "accuracy": 0.8062499761581421, "learning_rate": 4.775907352415367e-06, "epoch": 2.2222222222222223, "percentage": 22.22, "elapsed_time": "0:03:34", "remaining_time": "0:12:29"}
+{"current_steps": 100, "total_steps": 450, "eval_loss": 0.24907442927360535, "epoch": 2.2222222222222223, "percentage": 22.22, "elapsed_time": "0:03:37", "remaining_time": "0:12:40"}
+{"current_steps": 110, "total_steps": 450, "loss": 0.1984, "accuracy": 0.8062499761581421, "learning_rate": 4.688895578255228e-06, "epoch": 2.4444444444444446, "percentage": 24.44, "elapsed_time": "0:04:00", "remaining_time": "0:12:23"}
+{"current_steps": 120, "total_steps": 450, "loss": 0.1528, "accuracy": 0.8125, "learning_rate": 4.588719528532342e-06, "epoch": 2.6666666666666665, "percentage": 26.67, "elapsed_time": "0:04:19", "remaining_time": "0:11:54"}
+{"current_steps": 130, "total_steps": 450, "loss": 0.1557, "accuracy": 0.824999988079071, "learning_rate": 4.475981673796899e-06, "epoch": 2.888888888888889, "percentage": 28.89, "elapsed_time": "0:04:40", "remaining_time": "0:11:30"}
+{"current_steps": 140, "total_steps": 450, "loss": 0.1518, "accuracy": 0.824999988079071, "learning_rate": 4.351360032772512e-06, "epoch": 3.111111111111111, "percentage": 31.11, "elapsed_time": "0:05:00", "remaining_time": "0:11:05"}
+{"current_steps": 150, "total_steps": 450, "loss": 0.1014, "accuracy": 0.875, "learning_rate": 4.215604094671835e-06, "epoch": 3.3333333333333335, "percentage": 33.33, "elapsed_time": "0:05:20", "remaining_time": "0:10:40"}
+{"current_steps": 150, "total_steps": 450, "eval_loss": 0.16315264999866486, "epoch": 3.3333333333333335, "percentage": 33.33, "elapsed_time": "0:05:23", "remaining_time": "0:10:46"}
+{"current_steps": 160, "total_steps": 450, "loss": 0.1441, "accuracy": 0.793749988079071, "learning_rate": 4.069530311680247e-06, "epoch": 3.5555555555555554, "percentage": 35.56, "elapsed_time": "0:05:46", "remaining_time": "0:10:27"}
+{"current_steps": 170, "total_steps": 450, "loss": 0.1251, "accuracy": 0.824999988079071, "learning_rate": 3.914017188716347e-06, "epoch": 3.7777777777777777, "percentage": 37.78, "elapsed_time": "0:06:07", "remaining_time": "0:10:05"}
+{"current_steps": 180, "total_steps": 450, "loss": 0.1306, "accuracy": 0.7875000238418579, "learning_rate": 3.7500000000000005e-06, "epoch": 4.0, "percentage": 40.0, "elapsed_time": "0:06:28", "remaining_time": "0:09:42"}
+{"current_steps": 190, "total_steps": 450, "loss": 0.0949, "accuracy": 0.887499988079071, "learning_rate": 3.578465164203134e-06, "epoch": 4.222222222222222, "percentage": 42.22, "elapsed_time": "0:06:48", "remaining_time": "0:09:18"}
+{"current_steps": 200, "total_steps": 450, "loss": 0.1429, "accuracy": 0.8187500238418579, "learning_rate": 3.400444312011776e-06, "epoch": 4.444444444444445, "percentage": 44.44, "elapsed_time": "0:07:10", "remaining_time": "0:08:58"}
+{"current_steps": 200, "total_steps": 450, "eval_loss": 0.1533508002758026, "epoch": 4.444444444444445, "percentage": 44.44, "elapsed_time": "0:07:13", "remaining_time": "0:09:02"}
+{"current_steps": 210, "total_steps": 450, "loss": 0.1171, "accuracy": 0.84375, "learning_rate": 3.217008081777726e-06, "epoch": 4.666666666666667, "percentage": 46.67, "elapsed_time": "0:07:39", "remaining_time": "0:08:45"}
+{"current_steps": 220, "total_steps": 450, "loss": 0.1029, "accuracy": 0.831250011920929, "learning_rate": 3.0292596805735275e-06, "epoch": 4.888888888888889, "percentage": 48.89, "elapsed_time": "0:08:01", "remaining_time": "0:08:22"}
+{"current_steps": 230, "total_steps": 450, "loss": 0.0878, "accuracy": 0.8374999761581421, "learning_rate": 2.8383282493753282e-06, "epoch": 5.111111111111111, "percentage": 51.11, "elapsed_time": "0:08:20", "remaining_time": "0:07:59"}
+{"current_steps": 240, "total_steps": 450, "loss": 0.1183, "accuracy": 0.8687499761581421, "learning_rate": 2.6453620722761897e-06, "epoch": 5.333333333333333, "percentage": 53.33, "elapsed_time": "0:08:41", "remaining_time": "0:07:36"}
+{"current_steps": 250, "total_steps": 450, "loss": 0.1007, "accuracy": 0.8374999761581421, "learning_rate": 2.4515216705704396e-06, "epoch": 5.555555555555555, "percentage": 55.56, "elapsed_time": "0:09:02", "remaining_time": "0:07:13"}
+{"current_steps": 250, "total_steps": 450, "eval_loss": 0.15295907855033875, "epoch": 5.555555555555555, "percentage": 55.56, "elapsed_time": "0:09:05", "remaining_time": "0:07:16"}
+{"current_steps": 260, "total_steps": 450, "loss": 0.127, "accuracy": 0.8374999761581421, "learning_rate": 2.2579728232420524e-06, "epoch": 5.777777777777778, "percentage": 57.78, "elapsed_time": "0:09:28", "remaining_time": "0:06:55"}
+{"current_steps": 270, "total_steps": 450, "loss": 0.0778, "accuracy": 0.893750011920929, "learning_rate": 2.0658795558326745e-06, "epoch": 6.0, "percentage": 60.0, "elapsed_time": "0:09:49", "remaining_time": "0:06:32"}
+{"current_steps": 280, "total_steps": 450, "loss": 0.1018, "accuracy": 0.875, "learning_rate": 1.876397139855047e-06, "epoch": 6.222222222222222, "percentage": 62.22, "elapsed_time": "0:10:10", "remaining_time": "0:06:10"}
+{"current_steps": 290, "total_steps": 450, "loss": 0.0805, "accuracy": 0.887499988079071, "learning_rate": 1.6906651448541977e-06, "epoch": 6.444444444444445, "percentage": 64.44, "elapsed_time": "0:10:30", "remaining_time": "0:05:48"}
+{"current_steps": 300, "total_steps": 450, "loss": 0.1385, "accuracy": 0.8062499761581421, "learning_rate": 1.509800584902108e-06, "epoch": 6.666666666666667, "percentage": 66.67, "elapsed_time": "0:10:51", "remaining_time": "0:05:25"}
+{"current_steps": 300, "total_steps": 450, "eval_loss": 0.15343782305717468, "epoch": 6.666666666666667, "percentage": 66.67, "elapsed_time": "0:10:55", "remaining_time": "0:05:27"}
+{"current_steps": 310, "total_steps": 450, "loss": 0.075, "accuracy": 0.90625, "learning_rate": 1.3348912007436538e-06, "epoch": 6.888888888888889, "percentage": 68.89, "elapsed_time": "0:11:19", "remaining_time": "0:05:06"}
+{"current_steps": 320, "total_steps": 450, "loss": 0.0902, "accuracy": 0.856249988079071, "learning_rate": 1.1669889179957725e-06, "epoch": 7.111111111111111, "percentage": 71.11, "elapsed_time": "0:11:40", "remaining_time": "0:04:44"}
+{"current_steps": 330, "total_steps": 450, "loss": 0.0812, "accuracy": 0.8999999761581421, "learning_rate": 1.0071035207430352e-06, "epoch": 7.333333333333333, "percentage": 73.33, "elapsed_time": "0:12:01", "remaining_time": "0:04:22"}
+{"current_steps": 340, "total_steps": 450, "loss": 0.0655, "accuracy": 0.9125000238418579, "learning_rate": 8.561965785773413e-07, "epoch": 7.555555555555555, "percentage": 75.56, "elapsed_time": "0:12:21", "remaining_time": "0:03:59"}
+{"current_steps": 350, "total_steps": 450, "loss": 0.0918, "accuracy": 0.856249988079071, "learning_rate": 7.151756636052529e-07, "epoch": 7.777777777777778, "percentage": 77.78, "elapsed_time": "0:12:42", "remaining_time": "0:03:37"}
+{"current_steps": 350, "total_steps": 450, "eval_loss": 0.1536635458469391, "epoch": 7.777777777777778, "percentage": 77.78, "elapsed_time": "0:12:45", "remaining_time": "0:03:38"}
+{"current_steps": 360, "total_steps": 450, "loss": 0.118, "accuracy": 0.84375, "learning_rate": 5.848888922025553e-07, "epoch": 8.0, "percentage": 80.0, "elapsed_time": "0:13:11", "remaining_time": "0:03:17"}
+{"current_steps": 370, "total_steps": 450, "loss": 0.0877, "accuracy": 0.887499988079071, "learning_rate": 4.661198243425813e-07, "epoch": 8.222222222222221, "percentage": 82.22, "elapsed_time": "0:13:33", "remaining_time": "0:02:55"}
+{"current_steps": 380, "total_steps": 450, "loss": 0.0857, "accuracy": 0.8999999761581421, "learning_rate": 3.595827511743341e-07, "epoch": 8.444444444444445, "percentage": 84.44, "elapsed_time": "0:13:53", "remaining_time": "0:02:33"}
+{"current_steps": 390, "total_steps": 450, "loss": 0.1054, "accuracy": 0.862500011920929, "learning_rate": 2.6591839919146963e-07, "epoch": 8.666666666666666, "percentage": 86.67, "elapsed_time": "0:14:13", "remaining_time": "0:02:11"}
+{"current_steps": 400, "total_steps": 450, "loss": 0.0752, "accuracy": 0.887499988079071, "learning_rate": 1.8569007682777417e-07, "epoch": 8.88888888888889, "percentage": 88.89, "elapsed_time": "0:14:33", "remaining_time": "0:01:49"}
+{"current_steps": 400, "total_steps": 450, "eval_loss": 0.15339744091033936, "epoch": 8.88888888888889, "percentage": 88.89, "elapsed_time": "0:14:37", "remaining_time": "0:01:49"}
+{"current_steps": 410, "total_steps": 450, "loss": 0.1064, "accuracy": 0.856249988079071, "learning_rate": 1.1938028665396172e-07, "epoch": 9.11111111111111, "percentage": 91.11, "elapsed_time": "0:15:00", "remaining_time": "0:01:27"}
+{"current_steps": 420, "total_steps": 450, "loss": 0.0919, "accuracy": 0.8687499761581421, "learning_rate": 6.738782355044048e-08, "epoch": 9.333333333333334, "percentage": 93.33, "elapsed_time": "0:15:21", "remaining_time": "0:01:05"}
+{"current_steps": 430, "total_steps": 450, "loss": 0.0601, "accuracy": 0.9375, "learning_rate": 3.0025376307977474e-08, "epoch": 9.555555555555555, "percentage": 95.56, "elapsed_time": "0:15:43", "remaining_time": "0:00:43"}
+{"current_steps": 440, "total_steps": 450, "loss": 0.0779, "accuracy": 0.8687499761581421, "learning_rate": 7.517647080519941e-09, "epoch": 9.777777777777779, "percentage": 97.78, "elapsed_time": "0:16:03", "remaining_time": "0:00:21"}
+{"current_steps": 450, "total_steps": 450, "loss": 0.1052, "accuracy": 0.875, "learning_rate": 0.0, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "0:16:23", "remaining_time": "0:00:00"}
+{"current_steps": 450, "total_steps": 450, "eval_loss": 0.1529119908809662, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "0:16:26", "remaining_time": "0:00:00"}
+{"current_steps": 450, "total_steps": 450, "epoch": 10.0, "percentage": 100.0, "elapsed_time": "0:16:34", "remaining_time": "0:00:00"}

chchen__Llama-3.1-8B-Instruct-SAA-800/trainer_state.json ADDED Viewed

	@@ -0,0 +1,969 @@

+{
+  "best_metric": 0.1529119908809662,
+  "best_model_checkpoint": "saves/Llama-3.1-8B-Instruct/lora/saa-800/checkpoint-450",
+  "epoch": 10.0,
+  "eval_steps": 50,
+  "global_step": 450,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.2222222222222222,
+      "grad_norm": 6.713047504425049,
+      "learning_rate": 1.111111111111111e-06,
+      "logits/chosen": -0.4419662356376648,
+      "logits/rejected": -0.5169282555580139,
+      "logps/chosen": -1.7502187490463257,
+      "logps/rejected": -2.110954523086548,
+      "loss": 1.8058,
+      "odds_ratio_loss": 15.742452621459961,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": -0.17502185702323914,
+      "rewards/margins": 0.03607357665896416,
+      "rewards/rejected": -0.21109545230865479,
+      "sft_loss": 0.23150739073753357,
+      "step": 10
+    },
+    {
+      "epoch": 0.4444444444444444,
+      "grad_norm": 7.350757122039795,
+      "learning_rate": 2.222222222222222e-06,
+      "logits/chosen": -0.390705406665802,
+      "logits/rejected": -0.4706195294857025,
+      "logps/chosen": -1.6888424158096313,
+      "logps/rejected": -2.1693763732910156,
+      "loss": 1.7416,
+      "odds_ratio_loss": 15.1643705368042,
+      "rewards/accuracies": 0.7562500238418579,
+      "rewards/chosen": -0.16888423264026642,
+      "rewards/margins": 0.048053398728370667,
+      "rewards/rejected": -0.21693763136863708,
+      "sft_loss": 0.22513218224048615,
+      "step": 20
+    },
+    {
+      "epoch": 0.6666666666666666,
+      "grad_norm": 8.413761138916016,
+      "learning_rate": 3.3333333333333333e-06,
+      "logits/chosen": -0.42878788709640503,
+      "logits/rejected": -0.5077282190322876,
+      "logps/chosen": -1.698947548866272,
+      "logps/rejected": -2.1924192905426025,
+      "loss": 1.7505,
+      "odds_ratio_loss": 15.268714904785156,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": -0.1698947548866272,
+      "rewards/margins": 0.04934718459844589,
+      "rewards/rejected": -0.2192419320344925,
+      "sft_loss": 0.22367195785045624,
+      "step": 30
+    },
+    {
+      "epoch": 0.8888888888888888,
+      "grad_norm": 6.453053951263428,
+      "learning_rate": 4.444444444444444e-06,
+      "logits/chosen": -0.40729817748069763,
+      "logits/rejected": -0.480067640542984,
+      "logps/chosen": -1.677170753479004,
+      "logps/rejected": -1.9914417266845703,
+      "loss": 1.736,
+      "odds_ratio_loss": 15.256231307983398,
+      "rewards/accuracies": 0.706250011920929,
+      "rewards/chosen": -0.1677170693874359,
+      "rewards/margins": 0.03142711520195007,
+      "rewards/rejected": -0.1991441696882248,
+      "sft_loss": 0.21038159728050232,
+      "step": 40
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "grad_norm": 5.584766864776611,
+      "learning_rate": 4.998119881260576e-06,
+      "logits/chosen": -0.4123212695121765,
+      "logits/rejected": -0.49063047766685486,
+      "logps/chosen": -1.4437320232391357,
+      "logps/rejected": -1.951922059059143,
+      "loss": 1.4945,
+      "odds_ratio_loss": 13.130969047546387,
+      "rewards/accuracies": 0.7875000238418579,
+      "rewards/chosen": -0.14437320828437805,
+      "rewards/margins": 0.050819020718336105,
+      "rewards/rejected": -0.19519223272800446,
+      "sft_loss": 0.18137237429618835,
+      "step": 50
+    },
+    {
+      "epoch": 1.1111111111111112,
+      "eval_logits/chosen": -0.3793938159942627,
+      "eval_logits/rejected": -0.45482414960861206,
+      "eval_logps/chosen": -1.228076696395874,
+      "eval_logps/rejected": -1.7094627618789673,
+      "eval_loss": 1.276250958442688,
+      "eval_odds_ratio_loss": 11.260908126831055,
+      "eval_rewards/accuracies": 0.7875000238418579,
+      "eval_rewards/chosen": -0.12280768156051636,
+      "eval_rewards/margins": 0.04813859239220619,
+      "eval_rewards/rejected": -0.17094627022743225,
+      "eval_runtime": 3.3465,
+      "eval_samples_per_second": 23.905,
+      "eval_sft_loss": 0.15016020834445953,
+      "eval_steps_per_second": 11.953,
+      "step": 50
+    },
+    {
+      "epoch": 1.3333333333333333,
+      "grad_norm": 3.5215771198272705,
+      "learning_rate": 4.983095894354858e-06,
+      "logits/chosen": -0.4166947901248932,
+      "logits/rejected": -0.49852150678634644,
+      "logps/chosen": -1.0756200551986694,
+      "logps/rejected": -1.5570634603500366,
+      "loss": 1.1242,
+      "odds_ratio_loss": 9.97706127166748,
+      "rewards/accuracies": 0.831250011920929,
+      "rewards/chosen": -0.10756200551986694,
+      "rewards/margins": 0.04814432933926582,
+      "rewards/rejected": -0.15570633113384247,
+      "sft_loss": 0.1265355795621872,
+      "step": 60
+    },
+    {
+      "epoch": 1.5555555555555556,
+      "grad_norm": 3.005864381790161,
+      "learning_rate": 4.953138276568462e-06,
+      "logits/chosen": -0.45257633924484253,
+      "logits/rejected": -0.5196300148963928,
+      "logps/chosen": -0.6806862950325012,
+      "logps/rejected": -1.1254621744155884,
+      "loss": 0.7274,
+      "odds_ratio_loss": 6.491485595703125,
+      "rewards/accuracies": 0.800000011920929,
+      "rewards/chosen": -0.06806863099336624,
+      "rewards/margins": 0.044477589428424835,
+      "rewards/rejected": -0.11254620552062988,
+      "sft_loss": 0.07826091349124908,
+      "step": 70
+    },
+    {
+      "epoch": 1.7777777777777777,
+      "grad_norm": 3.241006851196289,
+      "learning_rate": 4.908427196539701e-06,
+      "logits/chosen": -0.4250833988189697,
+      "logits/rejected": -0.49341732263565063,
+      "logps/chosen": -0.586665153503418,
+      "logps/rejected": -0.9428589940071106,
+      "loss": 0.6374,
+      "odds_ratio_loss": 5.749706268310547,
+      "rewards/accuracies": 0.762499988079071,
+      "rewards/chosen": -0.05866651609539986,
+      "rewards/margins": 0.03561938554048538,
+      "rewards/rejected": -0.09428589791059494,
+      "sft_loss": 0.06239296868443489,
+      "step": 80
+    },
+    {
+      "epoch": 2.0,
+      "grad_norm": 3.3662965297698975,
+      "learning_rate": 4.849231551964771e-06,
+      "logits/chosen": -0.39320772886276245,
+      "logits/rejected": -0.48544201254844666,
+      "logps/chosen": -0.3357451558113098,
+      "logps/rejected": -0.9199529886245728,
+      "loss": 0.3774,
+      "odds_ratio_loss": 3.414803981781006,
+      "rewards/accuracies": 0.8187500238418579,
+      "rewards/chosen": -0.03357451409101486,
+      "rewards/margins": 0.05842079594731331,
+      "rewards/rejected": -0.09199531376361847,
+      "sft_loss": 0.03593815118074417,
+      "step": 90
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "grad_norm": 1.9536067247390747,
+      "learning_rate": 4.775907352415367e-06,
+      "logits/chosen": -0.3746485710144043,
+      "logits/rejected": -0.46515822410583496,
+      "logps/chosen": -0.22609436511993408,
+      "logps/rejected": -0.7698042392730713,
+      "loss": 0.2666,
+      "odds_ratio_loss": 2.4158308506011963,
+      "rewards/accuracies": 0.8062499761581421,
+      "rewards/chosen": -0.022609438747167587,
+      "rewards/margins": 0.05437099188566208,
+      "rewards/rejected": -0.07698042690753937,
+      "sft_loss": 0.02498885616660118,
+      "step": 100
+    },
+    {
+      "epoch": 2.2222222222222223,
+      "eval_logits/chosen": -0.36658304929733276,
+      "eval_logits/rejected": -0.4319043755531311,
+      "eval_logps/chosen": -0.2119404524564743,
+      "eval_logps/rejected": -0.7287805676460266,
+      "eval_loss": 0.24907442927360535,
+      "eval_odds_ratio_loss": 2.2483649253845215,
+      "eval_rewards/accuracies": 0.824999988079071,
+      "eval_rewards/chosen": -0.02119404636323452,
+      "eval_rewards/margins": 0.05168401449918747,
+      "eval_rewards/rejected": -0.07287804782390594,
+      "eval_runtime": 3.3336,
+      "eval_samples_per_second": 23.998,
+      "eval_sft_loss": 0.02423796057701111,
+      "eval_steps_per_second": 11.999,
+      "step": 100
+    },
+    {
+      "epoch": 2.4444444444444446,
+      "grad_norm": 1.7341715097427368,
+      "learning_rate": 4.688895578255228e-06,
+      "logits/chosen": -0.38399261236190796,
+      "logits/rejected": -0.45104750990867615,
+      "logps/chosen": -0.1595337688922882,
+      "logps/rejected": -0.6476808190345764,
+      "loss": 0.1984,
+      "odds_ratio_loss": 1.7913240194320679,
+      "rewards/accuracies": 0.8062499761581421,
+      "rewards/chosen": -0.01595337688922882,
+      "rewards/margins": 0.048814695328474045,
+      "rewards/rejected": -0.06476806849241257,
+      "sft_loss": 0.0192393995821476,
+      "step": 110
+    },
+    {
+      "epoch": 2.6666666666666665,
+      "grad_norm": 1.6577801704406738,
+      "learning_rate": 4.588719528532342e-06,
+      "logits/chosen": -0.42996135354042053,
+      "logits/rejected": -0.5020943880081177,
+      "logps/chosen": -0.11305193603038788,
+      "logps/rejected": -0.6292703747749329,
+      "loss": 0.1528,
+      "odds_ratio_loss": 1.378528356552124,
+      "rewards/accuracies": 0.8125,
+      "rewards/chosen": -0.011305193416774273,
+      "rewards/margins": 0.05162184312939644,
+      "rewards/rejected": -0.06292703002691269,
+      "sft_loss": 0.014913685619831085,
+      "step": 120
+    },
+    {
+      "epoch": 2.888888888888889,
+      "grad_norm": 0.804415762424469,
+      "learning_rate": 4.475981673796899e-06,
+      "logits/chosen": -0.4010156989097595,
+      "logits/rejected": -0.4532565474510193,
+      "logps/chosen": -0.11344721168279648,
+      "logps/rejected": -0.5929852724075317,
+      "loss": 0.1557,
+      "odds_ratio_loss": 1.430558204650879,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": -0.011344721540808678,
+      "rewards/margins": 0.047953806817531586,
+      "rewards/rejected": -0.059298526495695114,
+      "sft_loss": 0.012685336172580719,
+      "step": 130
+    },
+    {
+      "epoch": 3.111111111111111,
+      "grad_norm": 1.188767910003662,
+      "learning_rate": 4.351360032772512e-06,
+      "logits/chosen": -0.37222054600715637,
+      "logits/rejected": -0.4317198395729065,
+      "logps/chosen": -0.10829365253448486,
+      "logps/rejected": -0.6532029509544373,
+      "loss": 0.1518,
+      "odds_ratio_loss": 1.385451078414917,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": -0.010829364880919456,
+      "rewards/margins": 0.05449093505740166,
+      "rewards/rejected": -0.06532029807567596,
+      "sft_loss": 0.013293206691741943,
+      "step": 140
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "grad_norm": 1.1802352666854858,
+      "learning_rate": 4.215604094671835e-06,
+      "logits/chosen": -0.3585512936115265,
+      "logits/rejected": -0.4357479512691498,
+      "logps/chosen": -0.07403306663036346,
+      "logps/rejected": -0.7158663868904114,
+      "loss": 0.1014,
+      "odds_ratio_loss": 0.9041417837142944,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": -0.007403307594358921,
+      "rewards/margins": 0.06418333202600479,
+      "rewards/rejected": -0.07158664613962173,
+      "sft_loss": 0.010936584323644638,
+      "step": 150
+    },
+    {
+      "epoch": 3.3333333333333335,
+      "eval_logits/chosen": -0.3283754289150238,
+      "eval_logits/rejected": -0.3820287585258484,
+      "eval_logps/chosen": -0.1292387694120407,
+      "eval_logps/rejected": -0.6057685017585754,
+      "eval_loss": 0.16315264999866486,
+      "eval_odds_ratio_loss": 1.4634513854980469,
+      "eval_rewards/accuracies": 0.8125,
+      "eval_rewards/chosen": -0.012923876754939556,
+      "eval_rewards/margins": 0.047652970999479294,
+      "eval_rewards/rejected": -0.060576848685741425,
+      "eval_runtime": 3.3355,
+      "eval_samples_per_second": 23.984,
+      "eval_sft_loss": 0.016807515174150467,
+      "eval_steps_per_second": 11.992,
+      "step": 150
+    },
+    {
+      "epoch": 3.5555555555555554,
+      "grad_norm": 2.5364763736724854,
+      "learning_rate": 4.069530311680247e-06,
+      "logits/chosen": -0.37899985909461975,
+      "logits/rejected": -0.4429406225681305,
+      "logps/chosen": -0.09646956622600555,
+      "logps/rejected": -0.5016245245933533,
+      "loss": 0.1441,
+      "odds_ratio_loss": 1.3272716999053955,
+      "rewards/accuracies": 0.793749988079071,
+      "rewards/chosen": -0.00964695680886507,
+      "rewards/margins": 0.04051550105214119,
+      "rewards/rejected": -0.050162456929683685,
+      "sft_loss": 0.011343193240463734,
+      "step": 160
+    },
+    {
+      "epoch": 3.7777777777777777,
+      "grad_norm": 1.1225122213363647,
+      "learning_rate": 3.914017188716347e-06,
+      "logits/chosen": -0.32853808999061584,
+      "logits/rejected": -0.3947625160217285,
+      "logps/chosen": -0.0880068689584732,
+      "logps/rejected": -0.5245143175125122,
+      "loss": 0.1251,
+      "odds_ratio_loss": 1.1624243259429932,
+      "rewards/accuracies": 0.824999988079071,
+      "rewards/chosen": -0.00880068726837635,
+      "rewards/margins": 0.04365074634552002,
+      "rewards/rejected": -0.05245143175125122,
+      "sft_loss": 0.00884676817804575,
+      "step": 170
+    },
+    {
+      "epoch": 4.0,
+      "grad_norm": 2.3566975593566895,
+      "learning_rate": 3.7500000000000005e-06,
+      "logits/chosen": -0.394605427980423,
+      "logits/rejected": -0.4463408589363098,
+      "logps/chosen": -0.09484803676605225,
+      "logps/rejected": -0.5903218984603882,
+      "loss": 0.1306,
+      "odds_ratio_loss": 1.1676770448684692,
+      "rewards/accuracies": 0.7875000238418579,
+      "rewards/chosen": -0.00948480237275362,
+      "rewards/margins": 0.049547381699085236,
+      "rewards/rejected": -0.05903219059109688,
+      "sft_loss": 0.013791071251034737,
+      "step": 180
+    },
+    {
+      "epoch": 4.222222222222222,
+      "grad_norm": 1.111855387687683,
+      "learning_rate": 3.578465164203134e-06,
+      "logits/chosen": -0.3551129698753357,
+      "logits/rejected": -0.43426722288131714,
+      "logps/chosen": -0.06811969727277756,
+      "logps/rejected": -0.6758677959442139,
+      "loss": 0.0949,
+      "odds_ratio_loss": 0.8656018376350403,
+      "rewards/accuracies": 0.887499988079071,
+      "rewards/chosen": -0.006811970379203558,
+      "rewards/margins": 0.06077481061220169,
+      "rewards/rejected": -0.06758677959442139,
+      "sft_loss": 0.008338207378983498,
+      "step": 190
+    },
+    {
+      "epoch": 4.444444444444445,
+      "grad_norm": 1.3708534240722656,
+      "learning_rate": 3.400444312011776e-06,
+      "logits/chosen": -0.345875084400177,
+      "logits/rejected": -0.3952234089374542,
+      "logps/chosen": -0.10109534114599228,
+      "logps/rejected": -0.5074071288108826,
+      "loss": 0.1429,
+      "odds_ratio_loss": 1.325073003768921,
+      "rewards/accuracies": 0.8187500238418579,
+      "rewards/chosen": -0.010109534487128258,
+      "rewards/margins": 0.04063117876648903,
+      "rewards/rejected": -0.05074070766568184,
+      "sft_loss": 0.010358814150094986,
+      "step": 200
+    },
+    {
+      "epoch": 4.444444444444445,
+      "eval_logits/chosen": -0.32978400588035583,
+      "eval_logits/rejected": -0.38182884454727173,
+      "eval_logps/chosen": -0.121058389544487,
+      "eval_logps/rejected": -0.5841361284255981,
+      "eval_loss": 0.1533508002758026,
+      "eval_odds_ratio_loss": 1.3751581907272339,
+      "eval_rewards/accuracies": 0.8125,
+      "eval_rewards/chosen": -0.01210583932697773,
+      "eval_rewards/margins": 0.04630777984857559,
+      "eval_rewards/rejected": -0.058413613587617874,
+      "eval_runtime": 3.3378,
+      "eval_samples_per_second": 23.968,
+      "eval_sft_loss": 0.015834979712963104,
+      "eval_steps_per_second": 11.984,
+      "step": 200
+    },
+    {
+      "epoch": 4.666666666666667,
+      "grad_norm": 1.4620966911315918,
+      "learning_rate": 3.217008081777726e-06,
+      "logits/chosen": -0.3843366503715515,
+      "logits/rejected": -0.4375368058681488,
+      "logps/chosen": -0.08334760367870331,
+      "logps/rejected": -0.5660797357559204,
+      "loss": 0.1171,
+      "odds_ratio_loss": 1.0661273002624512,
+      "rewards/accuracies": 0.84375,
+      "rewards/chosen": -0.00833476148545742,
+      "rewards/margins": 0.04827320575714111,
+      "rewards/rejected": -0.05660796910524368,
+      "sft_loss": 0.010473220609128475,
+      "step": 210
+    },
+    {
+      "epoch": 4.888888888888889,
+      "grad_norm": 1.2518880367279053,
+      "learning_rate": 3.0292596805735275e-06,
+      "logits/chosen": -0.3598805367946625,
+      "logits/rejected": -0.4199501872062683,
+      "logps/chosen": -0.07187429815530777,
+      "logps/rejected": -0.5510466694831848,
+      "loss": 0.1029,
+      "odds_ratio_loss": 0.9251865148544312,
+      "rewards/accuracies": 0.831250011920929,
+      "rewards/chosen": -0.007187430746853352,
+      "rewards/margins": 0.047917239367961884,
+      "rewards/rejected": -0.05510466545820236,
+      "sft_loss": 0.010374334640800953,
+      "step": 220
+    },
+    {
+      "epoch": 5.111111111111111,
+      "grad_norm": 1.3117250204086304,
+      "learning_rate": 2.8383282493753282e-06,
+      "logits/chosen": -0.3635903596878052,
+      "logits/rejected": -0.43442487716674805,
+      "logps/chosen": -0.06123008579015732,
+      "logps/rejected": -0.7030965089797974,
+      "loss": 0.0878,
+      "odds_ratio_loss": 0.7977678179740906,
+      "rewards/accuracies": 0.8374999761581421,
+      "rewards/chosen": -0.006123009137809277,
+      "rewards/margins": 0.06418663263320923,
+      "rewards/rejected": -0.07030965387821198,
+      "sft_loss": 0.007975312881171703,
+      "step": 230
+    },
+    {
+      "epoch": 5.333333333333333,
+      "grad_norm": 2.640739679336548,
+      "learning_rate": 2.6453620722761897e-06,
+      "logits/chosen": -0.350188285112381,
+      "logits/rejected": -0.4023573398590088,
+      "logps/chosen": -0.08658941090106964,
+      "logps/rejected": -0.6677496433258057,
+      "loss": 0.1183,
+      "odds_ratio_loss": 1.068228006362915,
+      "rewards/accuracies": 0.8687499761581421,
+      "rewards/chosen": -0.00865894090384245,
+      "rewards/margins": 0.05811602994799614,
+      "rewards/rejected": -0.06677497178316116,
+      "sft_loss": 0.011527000926434994,
+      "step": 240
+    },
+    {
+      "epoch": 5.555555555555555,
+      "grad_norm": 1.3426445722579956,
+      "learning_rate": 2.4515216705704396e-06,
+      "logits/chosen": -0.3609291911125183,
+      "logits/rejected": -0.4190797209739685,
+      "logps/chosen": -0.0702720507979393,
+      "logps/rejected": -0.596028745174408,
+      "loss": 0.1007,
+      "odds_ratio_loss": 0.925524115562439,
+      "rewards/accuracies": 0.8374999761581421,
+      "rewards/chosen": -0.00702720507979393,
+      "rewards/margins": 0.052575670182704926,
+      "rewards/rejected": -0.059602878987789154,
+      "sft_loss": 0.008168576285243034,
+      "step": 250
+    },
+    {
+      "epoch": 5.555555555555555,
+      "eval_logits/chosen": -0.32348352670669556,
+      "eval_logits/rejected": -0.37402915954589844,
+      "eval_logps/chosen": -0.12062982469797134,
+      "eval_logps/rejected": -0.6407424807548523,
+      "eval_loss": 0.15295907855033875,
+      "eval_odds_ratio_loss": 1.3704458475112915,
+      "eval_rewards/accuracies": 0.8125,
+      "eval_rewards/chosen": -0.012062981724739075,
+      "eval_rewards/margins": 0.05201127007603645,
+      "eval_rewards/rejected": -0.06407425552606583,
+      "eval_runtime": 3.3317,
+      "eval_samples_per_second": 24.012,
+      "eval_sft_loss": 0.01591448113322258,
+      "eval_steps_per_second": 12.006,
+      "step": 250
+    },
+    {
+      "epoch": 5.777777777777778,
+      "grad_norm": 1.4032063484191895,
+      "learning_rate": 2.2579728232420524e-06,
+      "logits/chosen": -0.40077710151672363,
+      "logits/rejected": -0.4527947008609772,
+      "logps/chosen": -0.0916273444890976,
+      "logps/rejected": -0.5904611349105835,
+      "loss": 0.127,
+      "odds_ratio_loss": 1.1558631658554077,
+      "rewards/accuracies": 0.8374999761581421,
+      "rewards/chosen": -0.009162734262645245,
+      "rewards/margins": 0.049883387982845306,
+      "rewards/rejected": -0.05904611945152283,
+      "sft_loss": 0.011430850252509117,
+      "step": 260
+    },
+    {
+      "epoch": 6.0,
+      "grad_norm": 1.6447800397872925,
+      "learning_rate": 2.0658795558326745e-06,
+      "logits/chosen": -0.36904090642929077,
+      "logits/rejected": -0.43377822637557983,
+      "logps/chosen": -0.05662701651453972,
+      "logps/rejected": -0.6641786098480225,
+      "loss": 0.0778,
+      "odds_ratio_loss": 0.7118644714355469,
+      "rewards/accuracies": 0.893750011920929,
+      "rewards/chosen": -0.005662702023983002,
+      "rewards/margins": 0.060755155980587006,
+      "rewards/rejected": -0.06641785800457001,
+      "sft_loss": 0.006575153209269047,
+      "step": 270
+    },
+    {
+      "epoch": 6.222222222222222,
+      "grad_norm": 2.223752021789551,
+      "learning_rate": 1.876397139855047e-06,
+      "logits/chosen": -0.3556548058986664,
+      "logits/rejected": -0.4135330319404602,
+      "logps/chosen": -0.0746704488992691,
+      "logps/rejected": -0.6784394979476929,
+      "loss": 0.1018,
+      "odds_ratio_loss": 0.9240021705627441,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": -0.007467044983059168,
+      "rewards/margins": 0.06037690117955208,
+      "rewards/rejected": -0.0678439512848854,
+      "sft_loss": 0.009404648095369339,
+      "step": 280
+    },
+    {
+      "epoch": 6.444444444444445,
+      "grad_norm": 1.75334894657135,
+      "learning_rate": 1.6906651448541977e-06,
+      "logits/chosen": -0.3690106272697449,
+      "logits/rejected": -0.4269256591796875,
+      "logps/chosen": -0.05620593950152397,
+      "logps/rejected": -0.6589001417160034,
+      "loss": 0.0805,
+      "odds_ratio_loss": 0.7411076426506042,
+      "rewards/accuracies": 0.887499988079071,
+      "rewards/chosen": -0.005620594136416912,
+      "rewards/margins": 0.060269422829151154,
+      "rewards/rejected": -0.06589001417160034,
+      "sft_loss": 0.006437716074287891,
+      "step": 290
+    },
+    {
+      "epoch": 6.666666666666667,
+      "grad_norm": 4.5506911277771,
+      "learning_rate": 1.509800584902108e-06,
+      "logits/chosen": -0.3519323468208313,
+      "logits/rejected": -0.41471752524375916,
+      "logps/chosen": -0.10020919144153595,
+      "logps/rejected": -0.6135331392288208,
+      "loss": 0.1385,
+      "odds_ratio_loss": 1.263484001159668,
+      "rewards/accuracies": 0.8062499761581421,
+      "rewards/chosen": -0.010020919144153595,
+      "rewards/margins": 0.051332395523786545,
+      "rewards/rejected": -0.06135330721735954,
+      "sft_loss": 0.012116280384361744,
+      "step": 300
+    },
+    {
+      "epoch": 6.666666666666667,
+      "eval_logits/chosen": -0.32135990262031555,
+      "eval_logits/rejected": -0.3725373148918152,
+      "eval_logps/chosen": -0.12169794738292694,
+      "eval_logps/rejected": -0.6881048083305359,
+      "eval_loss": 0.15343782305717468,
+      "eval_odds_ratio_loss": 1.3728852272033691,
+      "eval_rewards/accuracies": 0.800000011920929,
+      "eval_rewards/chosen": -0.012169795110821724,
+      "eval_rewards/margins": 0.05664069205522537,
+      "eval_rewards/rejected": -0.06881048530340195,
+      "eval_runtime": 3.3342,
+      "eval_samples_per_second": 23.994,
+      "eval_sft_loss": 0.016149301081895828,
+      "eval_steps_per_second": 11.997,
+      "step": 300
+    },
+    {
+      "epoch": 6.888888888888889,
+      "grad_norm": 1.4514155387878418,
+      "learning_rate": 1.3348912007436538e-06,
+      "logits/chosen": -0.33361396193504333,
+      "logits/rejected": -0.3964681923389435,
+      "logps/chosen": -0.05151065066456795,
+      "logps/rejected": -0.6935869455337524,
+      "loss": 0.075,
+      "odds_ratio_loss": 0.6938644051551819,
+      "rewards/accuracies": 0.90625,
+      "rewards/chosen": -0.005151065066456795,
+      "rewards/margins": 0.06420762091875076,
+      "rewards/rejected": -0.069358691573143,
+      "sft_loss": 0.005607123486697674,
+      "step": 310
+    },
+    {
+      "epoch": 7.111111111111111,
+      "grad_norm": 1.1120833158493042,
+      "learning_rate": 1.1669889179957725e-06,
+      "logits/chosen": -0.34897491335868835,
+      "logits/rejected": -0.4280971884727478,
+      "logps/chosen": -0.06502507627010345,
+      "logps/rejected": -0.724215030670166,
+      "loss": 0.0902,
+      "odds_ratio_loss": 0.8162932395935059,
+      "rewards/accuracies": 0.856249988079071,
+      "rewards/chosen": -0.006502507720142603,
+      "rewards/margins": 0.06591899693012238,
+      "rewards/rejected": -0.07242151349782944,
+      "sft_loss": 0.008545474149286747,
+      "step": 320
+    },
+    {
+      "epoch": 7.333333333333333,
+      "grad_norm": 0.9275766611099243,
+      "learning_rate": 1.0071035207430352e-06,
+      "logits/chosen": -0.34552061557769775,
+      "logits/rejected": -0.4186931550502777,
+      "logps/chosen": -0.057514738291502,
+      "logps/rejected": -0.7304258942604065,
+      "loss": 0.0812,
+      "odds_ratio_loss": 0.7468503713607788,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": -0.005751474294811487,
+      "rewards/margins": 0.06729111820459366,
+      "rewards/rejected": -0.07304258644580841,
+      "sft_loss": 0.006484520621597767,
+      "step": 330
+    },
+    {
+      "epoch": 7.555555555555555,
+      "grad_norm": 1.7526174783706665,
+      "learning_rate": 8.561965785773413e-07,
+      "logits/chosen": -0.35446950793266296,
+      "logits/rejected": -0.41398343443870544,
+      "logps/chosen": -0.04472974315285683,
+      "logps/rejected": -0.6881515979766846,
+      "loss": 0.0655,
+      "odds_ratio_loss": 0.5992408394813538,
+      "rewards/accuracies": 0.9125000238418579,
+      "rewards/chosen": -0.004472973756492138,
+      "rewards/margins": 0.0643421858549118,
+      "rewards/rejected": -0.06881515681743622,
+      "sft_loss": 0.0055578285828232765,
+      "step": 340
+    },
+    {
+      "epoch": 7.777777777777778,
+      "grad_norm": 1.4603219032287598,
+      "learning_rate": 7.151756636052529e-07,
+      "logits/chosen": -0.35244011878967285,
+      "logits/rejected": -0.4057006239891052,
+      "logps/chosen": -0.06629346311092377,
+      "logps/rejected": -0.6709402203559875,
+      "loss": 0.0918,
+      "odds_ratio_loss": 0.841974139213562,
+      "rewards/accuracies": 0.856249988079071,
+      "rewards/chosen": -0.0066293468698859215,
+      "rewards/margins": 0.060464680194854736,
+      "rewards/rejected": -0.06709402799606323,
+      "sft_loss": 0.0076185790821909904,
+      "step": 350
+    },
+    {
+      "epoch": 7.777777777777778,
+      "eval_logits/chosen": -0.31914329528808594,
+      "eval_logits/rejected": -0.3697816729545593,
+      "eval_logps/chosen": -0.12174425274133682,
+      "eval_logps/rejected": -0.6888757944107056,
+      "eval_loss": 0.1536635458469391,
+      "eval_odds_ratio_loss": 1.3741592168807983,
+      "eval_rewards/accuracies": 0.8125,
+      "eval_rewards/chosen": -0.012174425646662712,
+      "eval_rewards/margins": 0.05671315640211105,
+      "eval_rewards/rejected": -0.06888757646083832,
+      "eval_runtime": 3.3381,
+      "eval_samples_per_second": 23.966,
+      "eval_sft_loss": 0.016247618943452835,
+      "eval_steps_per_second": 11.983,
+      "step": 350
+    },
+    {
+      "epoch": 8.0,
+      "grad_norm": 0.9194740056991577,
+      "learning_rate": 5.848888922025553e-07,
+      "logits/chosen": -0.3591064214706421,
+      "logits/rejected": -0.4297390878200531,
+      "logps/chosen": -0.08391134440898895,
+      "logps/rejected": -0.7468418478965759,
+      "loss": 0.118,
+      "odds_ratio_loss": 1.0900821685791016,
+      "rewards/accuracies": 0.84375,
+      "rewards/chosen": -0.00839113537222147,
+      "rewards/margins": 0.06629304587841034,
+      "rewards/rejected": -0.07468418776988983,
+      "sft_loss": 0.008979488164186478,
+      "step": 360
+    },
+    {
+      "epoch": 8.222222222222221,
+      "grad_norm": 1.1211438179016113,
+      "learning_rate": 4.661198243425813e-07,
+      "logits/chosen": -0.3469446301460266,
+      "logits/rejected": -0.416546493768692,
+      "logps/chosen": -0.06295167654752731,
+      "logps/rejected": -0.6733905076980591,
+      "loss": 0.0877,
+      "odds_ratio_loss": 0.7965422868728638,
+      "rewards/accuracies": 0.887499988079071,
+      "rewards/chosen": -0.006295167841017246,
+      "rewards/margins": 0.0610438771545887,
+      "rewards/rejected": -0.06733904778957367,
+      "sft_loss": 0.008044925518333912,
+      "step": 370
+    },
+    {
+      "epoch": 8.444444444444445,
+      "grad_norm": 2.9023804664611816,
+      "learning_rate": 3.595827511743341e-07,
+      "logits/chosen": -0.36501026153564453,
+      "logits/rejected": -0.4163384437561035,
+      "logps/chosen": -0.0608031265437603,
+      "logps/rejected": -0.7410011291503906,
+      "loss": 0.0857,
+      "odds_ratio_loss": 0.7958894371986389,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": -0.006080311723053455,
+      "rewards/margins": 0.068019799888134,
+      "rewards/rejected": -0.07410011440515518,
+      "sft_loss": 0.006128143519163132,
+      "step": 380
+    },
+    {
+      "epoch": 8.666666666666666,
+      "grad_norm": 1.2382421493530273,
+      "learning_rate": 2.6591839919146963e-07,
+      "logits/chosen": -0.3580993711948395,
+      "logits/rejected": -0.4169589579105377,
+      "logps/chosen": -0.07410109043121338,
+      "logps/rejected": -0.7789124250411987,
+      "loss": 0.1054,
+      "odds_ratio_loss": 0.9693442583084106,
+      "rewards/accuracies": 0.862500011920929,
+      "rewards/chosen": -0.0074101099744439125,
+      "rewards/margins": 0.07048113644123077,
+      "rewards/rejected": -0.07789124548435211,
+      "sft_loss": 0.008436056785285473,
+      "step": 390
+    },
+    {
+      "epoch": 8.88888888888889,
+      "grad_norm": 1.7495219707489014,
+      "learning_rate": 1.8569007682777417e-07,
+      "logits/chosen": -0.37370580434799194,
+      "logits/rejected": -0.43429917097091675,
+      "logps/chosen": -0.05244039371609688,
+      "logps/rejected": -0.7023984789848328,
+      "loss": 0.0752,
+      "odds_ratio_loss": 0.6945605874061584,
+      "rewards/accuracies": 0.887499988079071,
+      "rewards/chosen": -0.005244039464741945,
+      "rewards/margins": 0.06499581038951874,
+      "rewards/rejected": -0.0702398419380188,
+      "sft_loss": 0.005755658261477947,
+      "step": 400
+    },
+    {
+      "epoch": 8.88888888888889,
+      "eval_logits/chosen": -0.3194546103477478,
+      "eval_logits/rejected": -0.3705773651599884,
+      "eval_logps/chosen": -0.1213468462228775,
+      "eval_logps/rejected": -0.689638614654541,
+      "eval_loss": 0.15339744091033936,
+      "eval_odds_ratio_loss": 1.3723235130310059,
+      "eval_rewards/accuracies": 0.800000011920929,
+      "eval_rewards/chosen": -0.01213468424975872,
+      "eval_rewards/margins": 0.056829191744327545,
+      "eval_rewards/rejected": -0.06896387040615082,
+      "eval_runtime": 3.3351,
+      "eval_samples_per_second": 23.987,
+      "eval_sft_loss": 0.016165101900696754,
+      "eval_steps_per_second": 11.994,
+      "step": 400
+    },
+    {
+      "epoch": 9.11111111111111,
+      "grad_norm": 2.9914894104003906,
+      "learning_rate": 1.1938028665396172e-07,
+      "logits/chosen": -0.3595274090766907,
+      "logits/rejected": -0.41625022888183594,
+      "logps/chosen": -0.07939668744802475,
+      "logps/rejected": -0.6182006597518921,
+      "loss": 0.1064,
+      "odds_ratio_loss": 0.9655207395553589,
+      "rewards/accuracies": 0.856249988079071,
+      "rewards/chosen": -0.007939668372273445,
+      "rewards/margins": 0.05388040468096733,
+      "rewards/rejected": -0.06182006746530533,
+      "sft_loss": 0.009880236349999905,
+      "step": 410
+    },
+    {
+      "epoch": 9.333333333333334,
+      "grad_norm": 1.7583844661712646,
+      "learning_rate": 6.738782355044048e-08,
+      "logits/chosen": -0.35592731833457947,
+      "logits/rejected": -0.41578736901283264,
+      "logps/chosen": -0.06436184048652649,
+      "logps/rejected": -0.6822584867477417,
+      "loss": 0.0919,
+      "odds_ratio_loss": 0.8448025584220886,
+      "rewards/accuracies": 0.8687499761581421,
+      "rewards/chosen": -0.006436183117330074,
+      "rewards/margins": 0.06178967282176018,
+      "rewards/rejected": -0.06822585314512253,
+      "sft_loss": 0.007466461509466171,
+      "step": 420
+    },
+    {
+      "epoch": 9.555555555555555,
+      "grad_norm": 0.7222520112991333,
+      "learning_rate": 3.0025376307977474e-08,
+      "logits/chosen": -0.3527710437774658,
+      "logits/rejected": -0.41858869791030884,
+      "logps/chosen": -0.042861782014369965,
+      "logps/rejected": -0.7103925943374634,
+      "loss": 0.0601,
+      "odds_ratio_loss": 0.5496730804443359,
+      "rewards/accuracies": 0.9375,
+      "rewards/chosen": -0.004286178387701511,
+      "rewards/margins": 0.0667530745267868,
+      "rewards/rejected": -0.07103925198316574,
+      "sft_loss": 0.005130757577717304,
+      "step": 430
+    },
+    {
+      "epoch": 9.777777777777779,
+      "grad_norm": 1.2489662170410156,
+      "learning_rate": 7.517647080519941e-09,
+      "logits/chosen": -0.34115296602249146,
+      "logits/rejected": -0.39803385734558105,
+      "logps/chosen": -0.05217872932553291,
+      "logps/rejected": -0.8210271000862122,
+      "loss": 0.0779,
+      "odds_ratio_loss": 0.7251914739608765,
+      "rewards/accuracies": 0.8687499761581421,
+      "rewards/chosen": -0.0052178725600242615,
+      "rewards/margins": 0.07688483595848083,
+      "rewards/rejected": -0.0821027159690857,
+      "sft_loss": 0.0053517939522862434,
+      "step": 440
+    },
+    {
+      "epoch": 10.0,
+      "grad_norm": 1.9128329753875732,
+      "learning_rate": 0.0,
+      "logits/chosen": -0.3783785402774811,
+      "logits/rejected": -0.4389713406562805,
+      "logps/chosen": -0.07910243421792984,
+      "logps/rejected": -0.6643815040588379,
+      "loss": 0.1052,
+      "odds_ratio_loss": 0.9564045667648315,
+      "rewards/accuracies": 0.875,
+      "rewards/chosen": -0.007910243235528469,
+      "rewards/margins": 0.05852789804339409,
+      "rewards/rejected": -0.06643815338611603,
+      "sft_loss": 0.009510872885584831,
+      "step": 450
+    },
+    {
+      "epoch": 10.0,
+      "eval_logits/chosen": -0.3184443712234497,
+      "eval_logits/rejected": -0.36903008818626404,
+      "eval_logps/chosen": -0.1208958849310875,
+      "eval_logps/rejected": -0.6922038793563843,
+      "eval_loss": 0.1529119908809662,
+      "eval_odds_ratio_loss": 1.3675727844238281,
+      "eval_rewards/accuracies": 0.8125,
+      "eval_rewards/chosen": -0.012089588679373264,
+      "eval_rewards/margins": 0.05713079497218132,
+      "eval_rewards/rejected": -0.06922038644552231,
+      "eval_runtime": 3.3378,
+      "eval_samples_per_second": 23.968,
+      "eval_sft_loss": 0.016154715791344643,
+      "eval_steps_per_second": 11.984,
+      "step": 450
+    },
+    {
+      "epoch": 10.0,
+      "step": 450,
+      "total_flos": 8.074648418018918e+16,
+      "train_loss": 0.3428536836306254,
+      "train_runtime": 994.0751,
+      "train_samples_per_second": 7.243,
+      "train_steps_per_second": 0.453
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 450,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 50,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 8.074648418018918e+16,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

chchen__Llama-3.1-8B-Instruct-SAA-800/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d2135b35ec3a6c24b735c596abdf5ef75f5129a3177e9d6ef3cf7f021928112a
+size 5432

chchen__Llama-3.1-8B-Instruct-SAA-800/training_eval_loss.png ADDED Viewed

chchen__Llama-3.1-8B-Instruct-SAA-800/training_loss.png ADDED Viewed

chchen__Llama-3.1-8B-Instruct-SAA-800/training_rewards_accuracies.png ADDED Viewed