Error while running run_clm.py
Author: dpronyaev
Pull request: no

### System Info

- Windows 11
- Python 3.10.12
- accelerate==0.27.2
- aiohttp==3.9.3
- aiosignal==1.3.1
- async-timeout==4.0.3
- attrs==23.2.0
- blinker==1.4
- certifi==2024.2.2
- charset-normalizer==3.3.2
- command-not-found==0.3
- cryptography==3.4.8
- datasets==2.17.1
- dbus-python==1.2.18
- dill==0.3.8
- distro==1.7.0
- distro-info==1.1+ubuntu0.1
- evaluate==0.4.1
- filelock==3.13.1
- frozenlist==1.4.1
- fsspec==2023.10.0
- httplib2==0.20.2
- huggingface-hub==0.20.3
- idna==3.6
- importlib-metadata==4.6.4
- jeepney==0.7.1
- Jinja2==3.1.3
- joblib==1.3.2
- keyring==23.5.0
- launchpadlib==1.10.16
- lazr.restfulclient==0.14.4
- lazr.uri==1.0.6
- MarkupSafe==2.1.5
- more-itertools==8.10.0
- mpmath==1.3.0
- multidict==6.0.5
- multiprocess==0.70.16
- netifaces==0.11.0
- networkx==3.2.1
- numpy==1.26.4
- nvidia-cublas-cu12==12.1.3.1
- nvidia-cuda-cupti-cu12==12.1.105
- nvidia-cuda-nvrtc-cu12==12.1.105
- nvidia-cuda-runtime-cu12==12.1.105
- nvidia-cudnn-cu12==8.9.2.26
- nvidia-cufft-cu12==11.0.2.54
- nvidia-curand-cu12==10.3.2.106
- nvidia-cusolver-cu12==11.4.5.107
- nvidia-cusparse-cu12==12.1.0.106
- nvidia-nccl-cu12==2.19.3
- nvidia-nvjitlink-cu12==12.3.101
- nvidia-nvtx-cu12==12.1.105
- oauthlib==3.2.0
- packaging==23.2
- pandas==2.2.0
- psutil==5.9.8
- pyarrow==15.0.0
- pyarrow-hotfix==0.6
- PyGObject==3.42.1
- PyJWT==2.3.0
- pyparsing==2.4.7
- python-apt==2.4.0+ubuntu2
- python-dateutil==2.8.2
- pytz==2024.1
- PyYAML==5.4.1
- regex==2023.12.25
- requests==2.31.0
- responses==0.18.0
- safetensors==0.4.2
- scikit-learn==1.4.1.post1
- scipy==1.12.0
- SecretStorage==3.3.1
- six==1.16.0
- sympy==1.12
- systemd-python==234
- threadpoolctl==3.3.0
- tokenizers==0.15.2
- torch==2.2.0
- tqdm==4.66.2
- transformers @ file:///mnt/d/NEURO/distrib/transformers
- triton==2.2.0
- typing_extensions==4.9.0
- tzdata==2024.1
- ubuntu-advantage-tools==8001
- ufw==0.36.1
- unattended-upgrades==0.1
- urllib3==2.2.1
- wadllib==1.3.6
- xxhash==3.4.1
- yarl==1.9.4
- zipp==1.0.0




### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

While running
```
python3 run_clm.py \
   --model_name_or_path sberbank-ai/rugpt3small_based_on_gpt2 \
   --train_file corpus.txt \
   --per_device_train_batch_size 8 \
   --block_size 2048 \
   --dataset_config_name plain_text \
   --do_train \
   --gradient_accumulation_steps 4 \
   --gradient_checkpointing True \
   --fp16 True \
   --optim adafactor \
   --num_train_epochs 7 \
   --output_dir models/essays \
   --overwrite_output_dir
```

i got error 

```
03/02/2024 09:31:53 - WARNING - __main__ - Process rank: 0, device: cuda:0, n_gpu: 1, distributed training: False, 16-bits training: True
03/02/2024 09:31:53 - INFO - __main__ - Training/evaluation parameters TrainingArguments(
_n_gpu=1,
accelerator_config={'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True},
adafactor=False,
adam_beta1=0.9,
adam_beta2=0.999,
adam_epsilon=1e-08,
auto_find_batch_size=False,
bf16=False,
bf16_full_eval=False,
data_seed=None,
dataloader_drop_last=False,
dataloader_num_workers=0,
dataloader_persistent_workers=False,
dataloader_pin_memory=True,
dataloader_prefetch_factor=None,
ddp_backend=None,
ddp_broadcast_buffers=None,
ddp_bucket_cap_mb=None,
ddp_find_unused_parameters=None,
ddp_timeout=1800,
debug=[],
deepspeed=None,
disable_tqdm=False,
dispatch_batches=None,
do_eval=False,
do_predict=False,
do_train=True,
eval_accumulation_steps=None,
eval_delay=0,
eval_steps=None,
evaluation_strategy=no,
fp16=True,
fp16_backend=auto,
fp16_full_eval=False,
fp16_opt_level=O1,
fsdp=[],
fsdp_config={'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False},
fsdp_min_num_params=0,
fsdp_transformer_layer_cls_to_wrap=None,
full_determinism=False,
gradient_accumulation_steps=4,
gradient_checkpointing=True,
gradient_checkpointing_kwargs=None,
greater_is_better=None,
group_by_length=False,
half_precision_backend=auto,
hub_always_push=False,
hub_model_id=None,
hub_private_repo=False,
hub_strategy=every_save,
hub_token=<HUB_TOKEN>,
ignore_data_skip=False,
include_inputs_for_metrics=False,
include_num_input_tokens_seen=False,
include_tokens_per_second=False,
jit_mode_eval=False,
label_names=None,
label_smoothing_factor=0.0,
learning_rate=5e-05,
length_column_name=length,
load_best_model_at_end=False,
local_rank=0,
log_level=passive,
log_level_replica=warning,
log_on_each_node=True,
logging_dir=models/essays/runs/Mar02_09-31-53_DESKTOP-HFR1UK4,
logging_first_step=False,
logging_nan_inf_filter=True,
logging_steps=500,
logging_strategy=steps,
lr_scheduler_kwargs={},
lr_scheduler_type=linear,
max_grad_norm=1.0,
max_steps=-1,
metric_for_best_model=None,
mp_parameters=,
neftune_noise_alpha=None,
no_cuda=False,
num_train_epochs=7.0,
optim=adafactor,
optim_args=None,
output_dir=models/essays,
overwrite_output_dir=True,
past_index=-1,
per_device_eval_batch_size=8,
per_device_train_batch_size=8,
prediction_loss_only=False,
push_to_hub=False,
push_to_hub_model_id=None,
push_to_hub_organization=None,
push_to_hub_token=<PUSH_TO_HUB_TOKEN>,
ray_scope=last,
remove_unused_columns=True,
report_to=[],
resume_from_checkpoint=None,
run_name=models/essays,
save_on_each_node=False,
save_only_model=False,
save_safetensors=True,
save_steps=500,
save_strategy=steps,
save_total_limit=None,
seed=42,
skip_memory_metrics=True,
split_batches=None,
tf32=None,
torch_compile=False,
torch_compile_backend=None,
torch_compile_mode=None,
torchdynamo=None,
tpu_metrics_debug=False,
tpu_num_cores=None,
use_cpu=False,
use_ipex=False,
use_legacy_prediction_loop=False,
use_mps_device=False,
warmup_ratio=0.0,
warmup_steps=0,
weight_decay=0.0,
)
Using custom data configuration default-e57e946ad9254af3
03/02/2024 09:31:54 - INFO - datasets.builder - Using custom data configuration default-e57e946ad9254af3
Loading Dataset Infos from /home/pdv/.local/lib/python3.10/site-packages/datasets/packaged_modules/text
03/02/2024 09:31:54 - INFO - datasets.info - Loading Dataset Infos from /home/pdv/.local/lib/python3.10/site-packages/datasets/packaged_modules/text
Overwrite dataset info from restored data version if exists.
03/02/2024 09:31:54 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
03/02/2024 09:31:54 - INFO - datasets.info - Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
Found cached dataset text (/home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)
03/02/2024 09:31:54 - INFO - datasets.builder - Found cached dataset text (/home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)
Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
03/02/2024 09:31:54 - INFO - datasets.info - Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
Using custom data configuration default-e57e946ad9254af3
03/02/2024 09:31:55 - INFO - datasets.builder - Using custom data configuration default-e57e946ad9254af3
Loading Dataset Infos from /home/pdv/.local/lib/python3.10/site-packages/datasets/packaged_modules/text
03/02/2024 09:31:55 - INFO - datasets.info - Loading Dataset Infos from /home/pdv/.local/lib/python3.10/site-packages/datasets/packaged_modules/text
Overwrite dataset info from restored data version if exists.
03/02/2024 09:31:55 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.
Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
03/02/2024 09:31:55 - INFO - datasets.info - Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
Found cached dataset text (/home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)
03/02/2024 09:31:55 - INFO - datasets.builder - Found cached dataset text (/home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)
Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
03/02/2024 09:31:55 - INFO - datasets.info - Loading Dataset info from /home/pdv/.cache/huggingface/datasets/text/default-e57e946ad9254af3/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34
Traceback (most recent call last):
  File "/mnt/d/NEURO/distrib/transformers/run_clm.py", line 670, in <module>
    main()
  File "/mnt/d/NEURO/distrib/transformers/run_clm.py", line 373, in main
    raw_datasets["validation"] = load_dataset(
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/load.py", line 2587, in load_dataset
    ds = builder_instance.as_dataset(split=split, verification_mode=verification_mode, in_memory=keep_in_memory)
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/builder.py", line 1244, in as_dataset
    datasets = map_nested(
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 459, in map_nested
    return function(data_struct)
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/builder.py", line 1274, in _build_single_dataset
    ds = self._as_dataset(
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/builder.py", line 1348, in _as_dataset
    dataset_kwargs = ArrowReader(cache_dir, self.info).read(
  File "/home/pdv/.local/lib/python3.10/site-packages/datasets/arrow_reader.py", line 254, in read
    raise ValueError(msg)
ValueError: Instruction "train[:5%]" corresponds to no data!


```

here is corpus.txt used to learn model: https://gitlab.com/z00logist/artificial-dostoevsky/-/raw/main/data/corpus.txt (it's language is russian)

### Expected behavior

I expect "training complete" message.

Comments:
Hey! Example scripts should be adapted to each' s usecase, here you seem to have a problem with the data

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.