Error occurs when i try to pack my training.jsonl
State: closed | Pull request: no
Labels: name: bug

**Describe the bug**
When i follow this tutorial [https://docs.nvidia.com/nemo-framework/user-guide/latest/nemotoolkit/features/optimizations/sequence_packing.html#](url) with the training.jsonl generated by [https://docs.nvidia.com/nemo-framework/user-guide/24.07/playbooks/llama2sft.html](url) in Step 3.

A clear and concise description of what the bug is.
```shell
root@604d29b05e86:/opt/NeMo# export HYDRA_FULL_ERROR=1
root@604d29b05e86:/opt/NeMo# python scripts/nlp_language_modeling/prepare_packed_ft_dataset.py model.da
ta.train_ds.file_names=[/workspace/data/training.jsonl] model.data.train_ds.max_seq_length=2048 +tokeni
zer_path=/workspace/models/llama2-7b/tokenizer.model +output_dir=/workspace/data +pack_sizes=[2024,4096
,8192] +packing_algorithm=first_fit_shuffle +seed=0
[NeMo W 2025-01-09 11:02:07 nemo_logging:349] /usr/local/lib/python3.10/dist-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
    See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
      ret = run_job(
    
[NeMo I 2025-01-09 11:02:07 prepare_packed_ft_dataset:82] Tokenizing dataset...
[NeMo I 2025-01-09 11:02:07 tokenizer_utils:196] Getting SentencePiece with model: /workspace/models/llama2-7b/tokenizer.model
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:116] Building data files
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:525] Processing 1 data files using 2 workers
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:535] Time building 0 / 1 mem-mapped files: 0:00:00.121457
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:158] Loading data files
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:249] Loading /workspace/data/training.jsonl
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:161] Time loading 1 mem-mapped files: 0:00:00.003980
[NeMo I 2025-01-09 11:02:07 text_memmap_dataset:165] Computing global indices
[NeMo W 2025-01-09 11:02:08 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo W 2025-01-09 11:02:09 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo W 2025-01-09 11:02:09 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo W 2025-01-09 11:02:10 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo W 2025-01-09 11:02:12 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo W 2025-01-09 11:02:13 gpt_sft_dataset:324] input is not long enough to truncate.
[NeMo I 2025-01-09 11:02:14 sequence_packing_utils:115] Creating histogram from tokenized dataset...
[NeMo I 2025-01-09 11:02:14 sequence_packing_utils:155] Packing sequences to length 2024...
[NeMo I 2025-01-09 11:02:15 sequence_packing_utils:168] Packing is 99.79% efficient
[NeMo I 2025-01-09 11:02:15 sequence_packing_utils:169] >>>>> For pack size 2024, average number of sequences per pack is n = 10.168 <<<<<
100%|████████████████████████████████████████████████████████████| 2025/2025 [00:00<00:00, 3751.49it/s]
  6%|███▍                                                         | 67/1181 [00:00<00:00, 11882.38it/s]
Error executing job with overrides: ['model.data.train_ds.file_names=[/workspace/data/training.jsonl]', 'model.data.train_ds.max_seq_length=2048', '+tokenizer_path=/workspace/models/llama2-7b/tokenizer.model', '+output_dir=/workspace/data', '+pack_sizes=[2024,4096,8192]', '+packing_algorithm=first_fit_shuffle', '+seed=0']
Traceback (most recent call last):
  File "/opt/NeMo/scripts/nlp_language_modeling/prepare_packed_ft_dataset.py", line 174, in <module>
    main()
  File "/opt/NeMo/nemo/core/config/hydra_runner.py", line 129, in wrapper
    _run_hydra(
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 394, in _run_hydra
    _run_app(
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 457, in _run_app
    run_and_report(
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 223, in run_and_report
    raise ex
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 220, in run_and_report
    return func()
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/utils.py", line 458, in <lambda>
    lambda: hydra.run(
  File "/usr/local/lib/python3.10/dist-packages/hydra/_internal/hydra.py", line 132, in run
    _ = ret.return_value
  File "/usr/local/lib/python3.10/dist-packages/hydra/core/utils.py", line 260, in return_value
    raise self._return_value
  File "/usr/local/lib/python3.10/dist-packages/hydra/core/utils.py", line 186, in run_job
    ret.return_value = task_function(task_cfg)
  File "/opt/NeMo/scripts/nlp_language_modeling/prepare_packed_ft_dataset.py", line 142, in main
    output_data = fill_packing_strategy(assignments, sequences, pack_size)
  File "/opt/NeMo/nemo/utils/sequence_packing_utils.py", line 217, in fill_packing_strategy
    _input_ids.extend(ifile_handles[seq_length][0].pop())
KeyError: 2047
``` 
**Steps/Code to reproduce bug**
this is my scripts 
```shell
python scripts/nlp_language_modeling/prepare_packed_ft_
dataset.py model.data.train_ds.file_names=[/workspace/data/training.jsonl] model.data.train_ds.max_seq_
length=2048 +tokenizer_path=/workspace/models/llama2-7b/tokenizer.model +output_dir=/workspace/data +pa
ck_sizes=[2024,4096,8192] +packing_algorithm=first_fit_shuffle +seed=0
``` 
**Environment overview (please complete the following information)**
Docker from here [https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo/tags](url) version: nvcr.io/nvidia/nemo:24.05.llama3.1


Sorry, i make a mistake about the param "+pack_sizes [2024,...] -> [2048,...]"