Initial release

README.md

@@ -1,3 +1,113 @@
 ---
-license: apache-2.0
+language: 
+- ja
+license: cc-by-sa-4.0
+tags:
+- zero-shot-classification
+- text-classification
+- nli
+- pytorch
+metrics:
+- accuracy
+datasets:
+- JSNLI
+pipeline_tag: text-classification
+widget:
+- text: "あなた が 好きです 。 　 あなた を 愛して い ます 。"
+model-index:
+- name: roberta-base-japanese-jsnli
+  results:
+  - task:
+      type: text-classification
+      name: Natural Language Inference
+    dataset:
+      type: snli
+      name: JSNLI 
+      split: dev
+    metrics:
+      - type: accuracy
+        value: 0.9328
+        verified: false
 ---
+# roberta-base-japanese-jsnli
+
+This model is a fine-tuned version of [nlp-waseda/roberta-base-japanese](https://huggingface.co/nlp-waseda/roberta-base-japanese) on the [JSNLI](https://nlp.ist.i.kyoto-u.ac.jp/?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88) dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2039
+- Accuracy: 0.9328
+
+### How to use the model
+
+The input text should be segmented into words by [Juman++](https://github.com/ku-nlp/jumanpp) in advance.
+
+#### Simple zero-shot classification pipeline
+```python
+from transformers import pipeline
+from pyknp import Juman
+
+juman = Juman()
+
+classifier = pipeline("zero-shot-classification", model="Formzu/roberta-base-japanese-jsnli")
+
+sequence_to_classify = " ".join([tok.midasi for tok in juman.analysis("いつか世界を見る。").mrph_list()])
+
+candidate_labels = ['旅行', '料理', '踊り']
+out = classifier(sequence_to_classify, candidate_labels, hypothesis_template="この 例 は {} です 。")
+print(out)
+#{'sequence': 'いつか 世界 を 見る 。', 
+# 'labels': ['旅行', '踊り', '料理'], 
+# 'scores': [0.8998081684112549, 0.06059670448303223, 0.03959512338042259]}
+```
+#### NLI use-case
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
+from pyknp import Juman
+
+juman = Juman()
+
+device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
+
+model_name = "Formzu/roberta-base-japanese-jsnli"
+model = AutoModelForSequenceClassification.from_pretrained(model_name).to(device)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+premise = " ".join([tok.midasi for tok in juman.analysis("いつか世界を見る。").mrph_list()])
+label = '旅行'
+hypothesis = f'この 例 は {label} です 。'
+
+input = tokenizer.encode(premise, hypothesis, return_tensors='pt').to(device)
+with torch.no_grad():
+    logits = model(input)["logits"][0]
+    probs = logits.softmax(dim=-1)
+    print(probs.cpu().numpy(), logits.cpu().numpy())
+#[0.82168734 0.1744363  0.00387629] [ 2.3362164   0.78641605 -3.0202653 ]
+```
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 3.0
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | Accuracy |
+|:-------------:|:-----:|:-----:|:---------------:|:--------:|
+| 0.4067        | 1.0   | 16657 | 0.2224          | 0.9201   |
+| 0.3397        | 2.0   | 33314 | 0.2152          | 0.9208   |
+| 0.2775        | 3.0   | 49971 | 0.2039          | 0.9328   |
+
+
+### Framework versions
+
+- Transformers 4.21.2
+- Pytorch 1.12.1+cu116
+- Datasets 2.4.0
+- Tokenizers 0.12.1

all_results.json

@@ -0,0 +1,14 @@
+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.9328396320343018,
+    "eval_loss": 0.2038867175579071,
+    "eval_runtime": 27.4426,
+    "eval_samples": 3916,
+    "eval_samples_per_second": 142.698,
+    "eval_steps_per_second": 4.482,
+    "train_loss": 0.36741465170648585,
+    "train_runtime": 31812.3263,
+    "train_samples": 533005,
+    "train_samples_per_second": 50.264,
+    "train_steps_per_second": 1.571
+}

config.json

@@ -0,0 +1,38 @@
+{
+  "_name_or_path": "nlp-waseda/roberta-base-japanese",
+  "architectures": [
+    "RobertaForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 2,
+  "classifier_dropout": null,
+  "eos_token_id": 3,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "entailment",
+    "1": "neutral",
+    "2": "contradiction"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "entailment": 0,
+    "neutral": 1,
+    "contradiction": 2
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.21.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 32000
+}

eval_results.json

@@ -0,0 +1,9 @@
+{
+    "epoch": 3.0,
+    "eval_accuracy": 0.9328396320343018,
+    "eval_loss": 0.2038867175579071,
+    "eval_runtime": 27.4426,
+    "eval_samples": 3916,
+    "eval_samples_per_second": 142.698,
+    "eval_steps_per_second": 4.482
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4c0dc9f05170a5f332bc21660acb751993db79533b7c311ffe255852d666e5a
+size 442556397

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a7f87f538d8c73fb0a6a34efb7ba6e3488f920341119c02c208bce7965cf248e
+size 810161

tokenizer_config.json

@@ -0,0 +1,23 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "keep_accents": true,
+  "mask_token": {
+    "__type": "AddedToken",
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "name_or_path": "nlp-waseda/roberta-base-japanese",
+  "pad_token": "[PAD]",
+  "remove_space": true,
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "special_tokens_map_file": null,
+  "tokenizer_class": "AlbertTokenizer",
+  "unk_token": "[UNK]"
+}

train_results.json

@@ -0,0 +1,8 @@
+{
+    "epoch": 3.0,
+    "train_loss": 0.36741465170648585,
+    "train_runtime": 31812.3263,
+    "train_samples": 533005,
+    "train_samples_per_second": 50.264,
+    "train_steps_per_second": 1.571
+}

trainer_state.json

@@ -0,0 +1,646 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "global_step": 49971,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.03,
+      "learning_rate": 1.9799883932680958e-05,
+      "loss": 0.7935,
+      "step": 500
+    },
+    {
+      "epoch": 0.06,
+      "learning_rate": 1.959976786536191e-05,
+      "loss": 0.6149,
+      "step": 1000
+    },
+    {
+      "epoch": 0.09,
+      "learning_rate": 1.9399651798042868e-05,
+      "loss": 0.5758,
+      "step": 1500
+    },
+    {
+      "epoch": 0.12,
+      "learning_rate": 1.919953573072382e-05,
+      "loss": 0.5473,
+      "step": 2000
+    },
+    {
+      "epoch": 0.15,
+      "learning_rate": 1.8999419663404778e-05,
+      "loss": 0.5186,
+      "step": 2500
+    },
+    {
+      "epoch": 0.18,
+      "learning_rate": 1.879930359608573e-05,
+      "loss": 0.5147,
+      "step": 3000
+    },
+    {
+      "epoch": 0.21,
+      "learning_rate": 1.8599187528766688e-05,
+      "loss": 0.5012,
+      "step": 3500
+    },
+    {
+      "epoch": 0.24,
+      "learning_rate": 1.839907146144764e-05,
+      "loss": 0.49,
+      "step": 4000
+    },
+    {
+      "epoch": 0.27,
+      "learning_rate": 1.8198955394128598e-05,
+      "loss": 0.4918,
+      "step": 4500
+    },
+    {
+      "epoch": 0.3,
+      "learning_rate": 1.799883932680955e-05,
+      "loss": 0.4766,
+      "step": 5000
+    },
+    {
+      "epoch": 0.33,
+      "learning_rate": 1.7798723259490508e-05,
+      "loss": 0.4886,
+      "step": 5500
+    },
+    {
+      "epoch": 0.36,
+      "learning_rate": 1.759860719217146e-05,
+      "loss": 0.4762,
+      "step": 6000
+    },
+    {
+      "epoch": 0.39,
+      "learning_rate": 1.7398491124852417e-05,
+      "loss": 0.4621,
+      "step": 6500
+    },
+    {
+      "epoch": 0.42,
+      "learning_rate": 1.719837505753337e-05,
+      "loss": 0.46,
+      "step": 7000
+    },
+    {
+      "epoch": 0.45,
+      "learning_rate": 1.6998258990214324e-05,
+      "loss": 0.457,
+      "step": 7500
+    },
+    {
+      "epoch": 0.48,
+      "learning_rate": 1.679814292289528e-05,
+      "loss": 0.4413,
+      "step": 8000
+    },
+    {
+      "epoch": 0.51,
+      "learning_rate": 1.6598026855576234e-05,
+      "loss": 0.4482,
+      "step": 8500
+    },
+    {
+      "epoch": 0.54,
+      "learning_rate": 1.639791078825719e-05,
+      "loss": 0.4495,
+      "step": 9000
+    },
+    {
+      "epoch": 0.57,
+      "learning_rate": 1.6197794720938144e-05,
+      "loss": 0.4391,
+      "step": 9500
+    },
+    {
+      "epoch": 0.6,
+      "learning_rate": 1.59976786536191e-05,
+      "loss": 0.4369,
+      "step": 10000
+    },
+    {
+      "epoch": 0.63,
+      "learning_rate": 1.5797562586300054e-05,
+      "loss": 0.4413,
+      "step": 10500
+    },
+    {
+      "epoch": 0.66,
+      "learning_rate": 1.559744651898101e-05,
+      "loss": 0.4354,
+      "step": 11000
+    },
+    {
+      "epoch": 0.69,
+      "learning_rate": 1.5397330451661963e-05,
+      "loss": 0.4218,
+      "step": 11500
+    },
+    {
+      "epoch": 0.72,
+      "learning_rate": 1.519721438434292e-05,
+      "loss": 0.4267,
+      "step": 12000
+    },
+    {
+      "epoch": 0.75,
+      "learning_rate": 1.4997098317023875e-05,
+      "loss": 0.424,
+      "step": 12500
+    },
+    {
+      "epoch": 0.78,
+      "learning_rate": 1.479698224970483e-05,
+      "loss": 0.4234,
+      "step": 13000
+    },
+    {
+      "epoch": 0.81,
+      "learning_rate": 1.4596866182385785e-05,
+      "loss": 0.423,
+      "step": 13500
+    },
+    {
+      "epoch": 0.84,
+      "learning_rate": 1.439675011506674e-05,
+      "loss": 0.4188,
+      "step": 14000
+    },
+    {
+      "epoch": 0.87,
+      "learning_rate": 1.4196634047747695e-05,
+      "loss": 0.4145,
+      "step": 14500
+    },
+    {
+      "epoch": 0.9,
+      "learning_rate": 1.399651798042865e-05,
+      "loss": 0.4213,
+      "step": 15000
+    },
+    {
+      "epoch": 0.93,
+      "learning_rate": 1.3796401913109605e-05,
+      "loss": 0.4177,
+      "step": 15500
+    },
+    {
+      "epoch": 0.96,
+      "learning_rate": 1.359628584579056e-05,
+      "loss": 0.4071,
+      "step": 16000
+    },
+    {
+      "epoch": 0.99,
+      "learning_rate": 1.3396169778471514e-05,
+      "loss": 0.4067,
+      "step": 16500
+    },
+    {
+      "epoch": 1.0,
+      "eval_accuracy": 0.9200714826583862,
+      "eval_loss": 0.22239993512630463,
+      "eval_runtime": 27.5138,
+      "eval_samples_per_second": 142.329,
+      "eval_steps_per_second": 4.47,
+      "step": 16657
+    },
+    {
+      "epoch": 1.02,
+      "learning_rate": 1.319605371115247e-05,
+      "loss": 0.3698,
+      "step": 17000
+    },
+    {
+      "epoch": 1.05,
+      "learning_rate": 1.2995937643833424e-05,
+      "loss": 0.3628,
+      "step": 17500
+    },
+    {
+      "epoch": 1.08,
+      "learning_rate": 1.279582157651438e-05,
+      "loss": 0.3584,
+      "step": 18000
+    },
+    {
+      "epoch": 1.11,
+      "learning_rate": 1.2595705509195336e-05,
+      "loss": 0.3546,
+      "step": 18500
+    },
+    {
+      "epoch": 1.14,
+      "learning_rate": 1.239558944187629e-05,
+      "loss": 0.359,
+      "step": 19000
+    },
+    {
+      "epoch": 1.17,
+      "learning_rate": 1.2195473374557246e-05,
+      "loss": 0.3564,
+      "step": 19500
+    },
+    {
+      "epoch": 1.2,
+      "learning_rate": 1.19953573072382e-05,
+      "loss": 0.3621,
+      "step": 20000
+    },
+    {
+      "epoch": 1.23,
+      "learning_rate": 1.1795241239919156e-05,
+      "loss": 0.353,
+      "step": 20500
+    },
+    {
+      "epoch": 1.26,
+      "learning_rate": 1.1595125172600107e-05,
+      "loss": 0.3525,
+      "step": 21000
+    },
+    {
+      "epoch": 1.29,
+      "learning_rate": 1.1395009105281064e-05,
+      "loss": 0.3564,
+      "step": 21500
+    },
+    {
+      "epoch": 1.32,
+      "learning_rate": 1.1194893037962019e-05,
+      "loss": 0.3506,
+      "step": 22000
+    },
+    {
+      "epoch": 1.35,
+      "learning_rate": 1.0994776970642974e-05,
+      "loss": 0.3454,
+      "step": 22500
+    },
+    {
+      "epoch": 1.38,
+      "learning_rate": 1.0794660903323929e-05,
+      "loss": 0.3479,
+      "step": 23000
+    },
+    {
+      "epoch": 1.41,
+      "learning_rate": 1.0594544836004884e-05,
+      "loss": 0.3604,
+      "step": 23500
+    },
+    {
+      "epoch": 1.44,
+      "learning_rate": 1.0394428768685838e-05,
+      "loss": 0.36,
+      "step": 24000
+    },
+    {
+      "epoch": 1.47,
+      "learning_rate": 1.0194312701366793e-05,
+      "loss": 0.356,
+      "step": 24500
+    },
+    {
+      "epoch": 1.5,
+      "learning_rate": 9.994196634047748e-06,
+      "loss": 0.3515,
+      "step": 25000
+    },
+    {
+      "epoch": 1.53,
+      "learning_rate": 9.794080566728703e-06,
+      "loss": 0.3447,
+      "step": 25500
+    },
+    {
+      "epoch": 1.56,
+      "learning_rate": 9.593964499409658e-06,
+      "loss": 0.3548,
+      "step": 26000
+    },
+    {
+      "epoch": 1.59,
+      "learning_rate": 9.393848432090613e-06,
+      "loss": 0.3558,
+      "step": 26500
+    },
+    {
+      "epoch": 1.62,
+      "learning_rate": 9.193732364771568e-06,
+      "loss": 0.349,
+      "step": 27000
+    },
+    {
+      "epoch": 1.65,
+      "learning_rate": 8.993616297452523e-06,
+      "loss": 0.3352,
+      "step": 27500
+    },
+    {
+      "epoch": 1.68,
+      "learning_rate": 8.793500230133478e-06,
+      "loss": 0.3475,
+      "step": 28000
+    },
+    {
+      "epoch": 1.71,
+      "learning_rate": 8.593384162814433e-06,
+      "loss": 0.3482,
+      "step": 28500
+    },
+    {
+      "epoch": 1.74,
+      "learning_rate": 8.393268095495388e-06,
+      "loss": 0.3389,
+      "step": 29000
+    },
+    {
+      "epoch": 1.77,
+      "learning_rate": 8.193152028176343e-06,
+      "loss": 0.3485,
+      "step": 29500
+    },
+    {
+      "epoch": 1.8,
+      "learning_rate": 7.993035960857298e-06,
+      "loss": 0.3454,
+      "step": 30000
+    },
+    {
+      "epoch": 1.83,
+      "learning_rate": 7.792919893538253e-06,
+      "loss": 0.3354,
+      "step": 30500
+    },
+    {
+      "epoch": 1.86,
+      "learning_rate": 7.592803826219208e-06,
+      "loss": 0.3284,
+      "step": 31000
+    },
+    {
+      "epoch": 1.89,
+      "learning_rate": 7.3926877589001625e-06,
+      "loss": 0.3361,
+      "step": 31500
+    },
+    {
+      "epoch": 1.92,
+      "learning_rate": 7.192571691581117e-06,
+      "loss": 0.3418,
+      "step": 32000
+    },
+    {
+      "epoch": 1.95,
+      "learning_rate": 6.992455624262072e-06,
+      "loss": 0.3373,
+      "step": 32500
+    },
+    {
+      "epoch": 1.98,
+      "learning_rate": 6.792339556943027e-06,
+      "loss": 0.3397,
+      "step": 33000
+    },
+    {
+      "epoch": 2.0,
+      "eval_accuracy": 0.9208375811576843,
+      "eval_loss": 0.21520225703716278,
+      "eval_runtime": 27.5064,
+      "eval_samples_per_second": 142.367,
+      "eval_steps_per_second": 4.472,
+      "step": 33314
+    },
+    {
+      "epoch": 2.01,
+      "learning_rate": 6.592223489623982e-06,
+      "loss": 0.32,
+      "step": 33500
+    },
+    {
+      "epoch": 2.04,
+      "learning_rate": 6.392107422304937e-06,
+      "loss": 0.2961,
+      "step": 34000
+    },
+    {
+      "epoch": 2.07,
+      "learning_rate": 6.191991354985892e-06,
+      "loss": 0.2895,
+      "step": 34500
+    },
+    {
+      "epoch": 2.1,
+      "learning_rate": 5.991875287666847e-06,
+      "loss": 0.2965,
+      "step": 35000
+    },
+    {
+      "epoch": 2.13,
+      "learning_rate": 5.791759220347802e-06,
+      "loss": 0.2922,
+      "step": 35500
+    },
+    {
+      "epoch": 2.16,
+      "learning_rate": 5.591643153028758e-06,
+      "loss": 0.2837,
+      "step": 36000
+    },
+    {
+      "epoch": 2.19,
+      "learning_rate": 5.391527085709713e-06,
+      "loss": 0.2812,
+      "step": 36500
+    },
+    {
+      "epoch": 2.22,
+      "learning_rate": 5.191411018390668e-06,
+      "loss": 0.2819,
+      "step": 37000
+    },
+    {
+      "epoch": 2.25,
+      "learning_rate": 4.991294951071622e-06,
+      "loss": 0.2874,
+      "step": 37500
+    },
+    {
+      "epoch": 2.28,
+      "learning_rate": 4.791178883752577e-06,
+      "loss": 0.2727,
+      "step": 38000
+    },
+    {
+      "epoch": 2.31,
+      "learning_rate": 4.5910628164335316e-06,
+      "loss": 0.2733,
+      "step": 38500
+    },
+    {
+      "epoch": 2.34,
+      "learning_rate": 4.390946749114487e-06,
+      "loss": 0.2865,
+      "step": 39000
+    },
+    {
+      "epoch": 2.37,
+      "learning_rate": 4.190830681795442e-06,
+      "loss": 0.2799,
+      "step": 39500
+    },
+    {
+      "epoch": 2.4,
+      "learning_rate": 3.990714614476396e-06,
+      "loss": 0.2833,
+      "step": 40000
+    },
+    {
+      "epoch": 2.43,
+      "learning_rate": 3.7905985471573513e-06,
+      "loss": 0.2874,
+      "step": 40500
+    },
+    {
+      "epoch": 2.46,
+      "learning_rate": 3.5904824798383066e-06,
+      "loss": 0.2853,
+      "step": 41000
+    },
+    {
+      "epoch": 2.49,
+      "learning_rate": 3.3903664125192616e-06,
+      "loss": 0.2862,
+      "step": 41500
+    },
+    {
+      "epoch": 2.52,
+      "learning_rate": 3.1902503452002165e-06,
+      "loss": 0.2851,
+      "step": 42000
+    },
+    {
+      "epoch": 2.55,
+      "learning_rate": 2.9901342778811715e-06,
+      "loss": 0.2744,
+      "step": 42500
+    },
+    {
+      "epoch": 2.58,
+      "learning_rate": 2.7900182105621264e-06,
+      "loss": 0.2729,
+      "step": 43000
+    },
+    {
+      "epoch": 2.61,
+      "learning_rate": 2.589902143243081e-06,
+      "loss": 0.2792,
+      "step": 43500
+    },
+    {
+      "epoch": 2.64,
+      "learning_rate": 2.3897860759240363e-06,
+      "loss": 0.2828,
+      "step": 44000
+    },
+    {
+      "epoch": 2.67,
+      "learning_rate": 2.189670008604991e-06,
+      "loss": 0.277,
+      "step": 44500
+    },
+    {
+      "epoch": 2.7,
+      "learning_rate": 1.989553941285946e-06,
+      "loss": 0.2812,
+      "step": 45000
+    },
+    {
+      "epoch": 2.73,
+      "learning_rate": 1.789437873966901e-06,
+      "loss": 0.2807,
+      "step": 45500
+    },
+    {
+      "epoch": 2.76,
+      "learning_rate": 1.5893218066478558e-06,
+      "loss": 0.2852,
+      "step": 46000
+    },
+    {
+      "epoch": 2.79,
+      "learning_rate": 1.3892057393288107e-06,
+      "loss": 0.2824,
+      "step": 46500
+    },
+    {
+      "epoch": 2.82,
+      "learning_rate": 1.1890896720097659e-06,
+      "loss": 0.2784,
+      "step": 47000
+    },
+    {
+      "epoch": 2.85,
+      "learning_rate": 9.889736046907208e-07,
+      "loss": 0.2736,
+      "step": 47500
+    },
+    {
+      "epoch": 2.88,
+      "learning_rate": 7.888575373716756e-07,
+      "loss": 0.281,
+      "step": 48000
+    },
+    {
+      "epoch": 2.91,
+      "learning_rate": 5.887414700526306e-07,
+      "loss": 0.2781,
+      "step": 48500
+    },
+    {
+      "epoch": 2.94,
+      "learning_rate": 3.886254027335855e-07,
+      "loss": 0.2833,
+      "step": 49000
+    },
+    {
+      "epoch": 2.97,
+      "learning_rate": 1.8850933541454047e-07,
+      "loss": 0.2775,
+      "step": 49500
+    },
+    {
+      "epoch": 3.0,
+      "eval_accuracy": 0.9328396320343018,
+      "eval_loss": 0.2038867175579071,
+      "eval_runtime": 27.4892,
+      "eval_samples_per_second": 142.456,
+      "eval_steps_per_second": 4.474,
+      "step": 49971
+    },
+    {
+      "epoch": 3.0,
+      "step": 49971,
+      "total_flos": 4.207223016483379e+17,
+      "train_loss": 0.36741465170648585,
+      "train_runtime": 31812.3263,
+      "train_samples_per_second": 50.264,
+      "train_steps_per_second": 1.571
+    }
+  ],
+  "max_steps": 49971,
+  "num_train_epochs": 3,
+  "total_flos": 4.207223016483379e+17,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60ec9e05d62905148e152cb8ea6003042f18e94ab4eba950a4b1eb7c86a7ecaa
+size 3375