GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
Downloading and preparing dataset json/default to /home/ace14459tv/t5maru/cache/json/default-76e405bf2a5f1b35/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4...

Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]
Downloading data files: 100%|██████████| 1/1 [00:00<00:00, 7410.43it/s]

Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]
Extracting data files: 100%|██████████| 1/1 [00:00<00:00, 374.29it/s]

Generating train split: 0 examples [00:00, ? examples/s]
                                                        
Dataset json downloaded and prepared to /home/ace14459tv/t5maru/cache/json/default-76e405bf2a5f1b35/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4. Subsequent calls will reuse this data.

Map (num_proc=4):   0%|          | 0/2880 [00:00<?, ? examples/s]
Map (num_proc=4):   3%|▎         | 94/2880 [00:00<00:15, 175.45 examples/s]
Map (num_proc=4):  13%|█▎        | 384/2880 [00:00<00:03, 655.72 examples/s]
Map (num_proc=4):  32%|███▏      | 913/2880 [00:00<00:01, 1620.08 examples/s]
Map (num_proc=4):  58%|█████▊    | 1672/2880 [00:00<00:00, 2983.67 examples/s]
Map (num_proc=4):  86%|████████▌ | 2467/2880 [00:01<00:00, 4206.04 examples/s]
                                                                              
Downloading and preparing dataset json/default to /home/ace14459tv/t5maru/cache/json/default-8eccec914afb5393/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4...

Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]
Downloading data files: 100%|██████████| 1/1 [00:00<00:00, 8305.55it/s]

Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]
Extracting data files: 100%|██████████| 1/1 [00:00<00:00, 426.86it/s]

Generating train split: 0 examples [00:00, ? examples/s]
                                                        
Dataset json downloaded and prepared to /home/ace14459tv/t5maru/cache/json/default-8eccec914afb5393/0.0.0/e347ab1c932092252e717ff3f949105a4dd28b27e842dd53157d2f72e276c2e4. Subsequent calls will reuse this data.

Map (num_proc=4):   0%|          | 0/618 [00:00<?, ? examples/s]
Map (num_proc=4):  16%|█▌        | 96/618 [00:01<00:05, 94.84 examples/s]
                                                                         
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

  | Name  | Type            | Params
------------------------------------------
0 | model | OptimizedModule | 300 M 
------------------------------------------
300 M     Trainable params
0         Non-trainable params
300 M     Total params
1,200.707 Total estimated model params size (MB)
[2023-06-30 20:54:37,264] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
Metric val_loss improved. New best score: 1.139
Metric val_loss improved by 0.743 >= min_delta = 0.0. New best score: 0.397
Metric val_loss improved by 0.178 >= min_delta = 0.0. New best score: 0.219
Metric val_loss improved by 0.058 >= min_delta = 0.0. New best score: 0.161
Metric val_loss improved by 0.027 >= min_delta = 0.0. New best score: 0.134
Metric val_loss improved by 0.020 >= min_delta = 0.0. New best score: 0.115
Metric val_loss improved by 0.012 >= min_delta = 0.0. New best score: 0.103
Metric val_loss improved by 0.005 >= min_delta = 0.0. New best score: 0.098
Metric val_loss improved by 0.011 >= min_delta = 0.0. New best score: 0.087
Metric val_loss improved by 0.000 >= min_delta = 0.0. New best score: 0.086
Metric val_loss improved by 0.004 >= min_delta = 0.0. New best score: 0.083
Metric val_loss improved by 0.006 >= min_delta = 0.0. New best score: 0.077
Metric val_loss improved by 0.002 >= min_delta = 0.0. New best score: 0.075
Monitored metric val_loss did not improve in the last 3 records. Best score: 0.075. Signaling Trainer to stop.
{"log": "trained", "date": "2023-06-30T20:53:45", "elapsed": "00:05:22", "model": "google/mt5-small", "max_length": 128, "target_max_length": 128, "batch_size": 32, "gradient_accumulation_steps": 1, "train_steps": 2700, "accelerator": "gpu", "devices": "auto", "precision": 32, "strategy": "auto", "gradient_clip_val": 1.0, "compile": true, "solver": "adamw", "lr": 0.0003, "warmup_steps": 1, "training_steps": 100000, "adam_epsilon": 1e-08, "weight_decay": 0.0, "epoch": 17, "step": 1530, "saved": "0630_mT5"}
😊 testing /home/ace14459tv/t5maru/error_data/0630/error_3_0630_test.jsonl on cuda
Downloading and preparing dataset generator/default to /home/ace14459tv/t5maru/cache/generator/default-f9a3d4be341e4e78/0.0.0...

Generating train split: 0 examples [00:00, ? examples/s]
                                                        
Dataset generator downloaded and prepared to /home/ace14459tv/t5maru/cache/generator/default-f9a3d4be341e4e78/0.0.0. Subsequent calls will reuse this data.

Map (num_proc=4):   0%|          | 0/617 [00:00<?, ? examples/s]
Map (num_proc=4):  25%|██▌       | 155/617 [00:00<00:01, 267.27 examples/s]
                                                                           
😊 Tested 617 items. See 0630_mT5/error_3_0630_tested.jsonl