Faster `Skipping the first batches` in Trainer
Author: SamuelLarkin
Pull request: no

### Feature request

Improve speed when skipping the first batches in `trainer.py`.

### Motivation

Skipping batches should be fast.
```
Skipping the first batches:  76%|███████▌  | 93017/122500 [6:35:47<2:26:12,  3.36it/s]
```
In this example, the 'Trainer` already spent ~6h30 simply skipping batches and it estimates another ~2h30 to complete the task.  This should not take that long to accomplish as those batches are not used and simply discarded.

Early investigation show that the `dataloader` is invoke which implies that it samples, fetches and collates the data where collating can be expensive and useless.
Wouldn't simply looping X times the `dataloader.sampler` be sufficient to resume the state of training?  This would be a light process that could be done prior to the training loop.
Perhaps, as an alternate solution, we could temporarily attach a noop collator to `train_dataloader` while skipping the first batches.

### Your contribution

I would be glad to provide a PR.  I can investigate the issue further but I would need advices on the matter as I don't have a setup to test all possible combinations of `dataloader`, distributed, `deepspeed` and so forth.

Comments:
This is already done in the main branch. You just need to have `Accelerate` installed as an extra dependency.

Thanks for the info but I do have `accelerate==0.16.0` installed as reported in my logs but I still get extremely slow `Skipping the first batches`

```yaml
CONDA: transformers-4.20.1
name: transformers-4.20.1
channels:
  - pytorch
  - huggingface
  - anaconda
  - conda-forge
  - defaults
dependencies:
  - _libgcc_mutex=0.1=conda_forge
  - _openmp_mutex=4.5=2_kmp_llvm
  - abseil-cpp=20211102.0=hd4dd3e8_0
  - aiohttp=3.8.1=py39h7f8727e_1
  - aiosignal=1.2.0=pyhd3eb1b0_0
  - arrow-cpp=8.0.0=py39h60b952e_0
  - async-timeout=4.0.1=pyhd3eb1b0_0
  - attrs=21.4.0=pyhd3eb1b0_0
  - aws-c-common=0.4.57=he6710b0_1
  - aws-c-event-stream=0.1.6=h2531618_5
  - aws-checksums=0.1.9=he6710b0_0
  - aws-sdk-cpp=1.8.185=hce553d0_0
  - blas=1.0=mkl
  - boost-cpp=1.73.0=h7f8727e_12
  - bottleneck=1.3.4=py39hce1f21e_0
  - brotli=1.0.9=he6710b0_2
  - brotlipy=0.7.0=py39h27cfd23_1003
  - bzip2=1.0.8=h7b6447c_0
  - c-ares=1.18.1=h7f8727e_0
  - ca-certificates=2023.01.10=h06a4308_0
  - certifi=2022.12.7=py39h06a4308_0
  - cffi=1.15.0=py39hd667e15_1
  - charset-normalizer=2.0.4=pyhd3eb1b0_0
  - click=8.0.4=py39h06a4308_0
  - cryptography=37.0.1=py39h9ce1e76_0
  - cudatoolkit=11.6.0=hecad31d_10
  - cycler=0.11.0=pyhd3eb1b0_0
  - dataclasses=0.8=pyh6d0b6a4_7
  - datasets=2.3.2=py_0
  - dbus=1.13.18=hb2f20db_0
  - dill=0.3.4=pyhd3eb1b0_0
  - expat=2.4.4=h295c915_0
  - filelock=3.6.0=pyhd3eb1b0_0
  - fontconfig=2.13.1=h6c09931_0
  - fonttools=4.25.0=pyhd3eb1b0_0
  - freetype=2.11.0=h70c0345_0
  - frozenlist=1.2.0=py39h7f8727e_0
  - fsspec=2022.3.0=py39h06a4308_0
  - gflags=2.2.2=he6710b0_0
  - giflib=5.2.1=h7b6447c_0
  - glib=2.69.1=h4ff587b_1
  - glog=0.5.0=h2531618_0
  - grpc-cpp=1.46.1=h33aed49_0
  - gst-plugins-base=1.14.0=h8213a91_2
  - gstreamer=1.14.0=h28cd5cc_2
  - h5py=3.7.0=py39h737f45e_0
  - hdf5=1.10.6=hb1b8bf9_0
  - huggingface_hub=0.8.1=py_0
  - icu=58.2=he6710b0_3
  - idna=3.3=pyhd3eb1b0_0
  - importlib-metadata=4.11.3=py39h06a4308_0
  - importlib_metadata=4.11.3=hd3eb1b0_0
  - intel-openmp=2021.4.0=h06a4308_3561
  - joblib=1.1.0=pyhd3eb1b0_0
  - jpeg=9e=h7f8727e_0
  - kiwisolver=1.4.2=py39h295c915_0
  - krb5=1.19.2=hac12032_0
  - lcms2=2.12=h3be6417_0
  - ld_impl_linux-64=2.38=h1181459_1
  - lerc=3.0=h295c915_0
  - libboost=1.73.0=h28710b8_12
  - libbrotlicommon=1.0.9=h166bdaf_7
  - libbrotlidec=1.0.9=h166bdaf_7
  - libbrotlienc=1.0.9=h166bdaf_7
  - libclang=10.0.1=default_hb85057a_2
  - libcurl=7.82.0=h0b77cf5_0
  - libdeflate=1.8=h7f8727e_5
  - libedit=3.1.20210910=h7f8727e_0
  - libev=4.33=h7f8727e_1
  - libevent=2.1.12=h8f2d780_0
  - libffi=3.3=he6710b0_2
  - libgcc=7.2.0=h69d50b8_2
  - libgcc-ng=12.1.0=h8d9b700_16
  - libgfortran-ng=7.5.0=ha8ba4b0_17
  - libgfortran4=7.5.0=ha8ba4b0_17
  - libllvm10=10.0.1=hbcb73fb_5
  - libnghttp2=1.46.0=hce63b2e_0
  - libpng=1.6.37=hbc83047_0
  - libpq=12.9=h16c4e8d_3
  - libprotobuf=3.20.1=h4ff587b_0
  - libssh2=1.10.0=h8f2d780_0
  - libstdcxx-ng=11.2.0=h1234567_1
  - libthrift=0.15.0=hcc01f38_0
  - libtiff=4.4.0=hecacb30_0
  - libutf8proc=2.6.1=h27cfd23_0
  - libuuid=1.0.3=h7f8727e_2
  - libwebp=1.2.2=h55f646e_0
  - libwebp-base=1.2.2=h7f8727e_0
  - libxcb=1.15=h7f8727e_0
  - libxkbcommon=1.0.1=hfa300c1_0
  - libxml2=2.9.14=h74e7548_0
  - libxslt=1.1.35=h4e12654_0
  - libzlib=1.2.12=h166bdaf_1
  - llvm-openmp=14.0.4=he0ac6c6_0
  - lz4-c=1.9.3=h295c915_1
  - matplotlib=3.5.1=py39h06a4308_1
  - matplotlib-base=3.5.1=py39ha18d171_1
  - mkl=2021.4.0=h06a4308_640
  - mkl-service=2.4.0=py39h7f8727e_0
  - mkl_fft=1.3.1=py39hd3c417c_0
  - mkl_random=1.2.2=py39h51133e4_0
  - multidict=5.2.0=py39h7f8727e_2
  - multiprocess=0.70.12.2=py39h7f8727e_0
  - munkres=1.1.4=py_0
  - ncurses=6.3=h7f8727e_2
  - nodejs=6.11.2=h3db8ef7_0
  - nspr=4.33=h295c915_0
  - nss=3.74=h0370c37_0
  - numexpr=2.8.1=py39h807cd23_2
  - numpy=1.22.3=py39he7a7128_0
  - numpy-base=1.22.3=py39hf524024_0
  - openssl=1.1.1t=h7f8727e_0
  - orc=1.7.4=h07ed6aa_0
  - packaging=21.3=pyhd3eb1b0_0
  - pandas=1.4.2=py39h295c915_0
  - parquet-cpp=1.5.1=h34088ae_4
  - pcre=8.45=h295c915_0
  - pillow=9.2.0=py39hace64e9_1
  - pip=21.2.4=py39h06a4308_0
  - ply=3.11=py39h06a4308_0
  - protobuf=3.20.1=py39h295c915_0
  - pyarrow=8.0.0=py39h992f0b0_0
  - pycparser=2.21=pyhd3eb1b0_0
  - pyopenssl=22.0.0=pyhd3eb1b0_0
  - pyparsing=3.0.4=pyhd3eb1b0_0
  - pyqt=5.15.7=py39h6a678d5_1
  - pyqt5-sip=12.11.0=py39h6a678d5_1
  - pysocks=1.7.1=py39h06a4308_0
  - python=3.9.12=h12debd9_1
  - python-dateutil=2.8.2=pyhd3eb1b0_0
  - python-xxhash=2.0.2=py39h7f8727e_0
  - python_abi=3.9=1_cp39
  - pytorch=1.12.0=py3.9_cuda11.6_cudnn8.3.2_0
  - pytorch-mutex=1.0=cuda
  - pytz=2022.1=py39h06a4308_0
  - pyyaml=6.0=py39h7f8727e_1
  - qt-main=5.15.2=h327a75a_6
  - qt-webengine=5.15.9=hd2b0992_4
  - qtwebkit=5.212=h4eab89a_4
  - re2=2022.04.01=h295c915_0
  - readline=8.1.2=h7f8727e_1
  - regex=2022.3.15=py39h7f8727e_0
  - requests=2.27.1=pyhd3eb1b0_0
  - s2n=1.3.0=h9b69904_0
  - sacremoses=master=py_0
  - scikit-learn=1.0.2=py39h51133e4_1
  - scipy=1.7.3=py39hc147768_0
  - setuptools=61.2.0=py39h06a4308_0
  - sip=6.6.2=py39h6a678d5_0
  - six=1.16.0=pyhd3eb1b0_1
  - snappy=1.1.9=h295c915_0
  - sqlite=3.38.5=hc218d9a_0
  - tensorboardx=2.2=pyhd3eb1b0_0
  - threadpoolctl=2.2.0=pyh0d69192_0
  - tk=8.6.12=h1ccaba5_0
  - tokenizers=0.12.1=py39_0
  - toml=0.10.2=pyhd3eb1b0_0
  - tornado=6.1=py39h27cfd23_0
  - tqdm=4.64.0=py39h06a4308_0
  - transformers=4.20.1=pyhd8ed1ab_0
  - typing-extensions=4.1.1=hd3eb1b0_0
  - typing_extensions=4.1.1=pyh06a4308_0
  - tzdata=2022a=hda174b7_0
  - urllib3=1.26.9=py39h06a4308_0
  - utf8proc=2.6.1=h27cfd23_0
  - wheel=0.37.1=pyhd3eb1b0_0
  - xxhash=0.8.0=h7f8727e_3
  - xz=5.2.5=h7f8727e_1
  - yaml=0.2.5=h7b6447c_0
  - yarl=1.6.3=py39h27cfd23_0
  - zipp=3.8.0=py39h06a4308_0
  - zlib=1.2.12=h166bdaf_1
  - zstd=1.5.2=ha4553b6_0
  - pip:
      - absl-py==1.3.0
      - accelerate==0.16.0
      - asttokens==2.2.1
      - backcall==0.2.0
      - cachetools==5.2.1
      - codetiming==1.4.0
      - decorator==5.1.1
      - executing==1.2.0
      - google-auth==2.15.0
      - google-auth-oauthlib==0.4.6
      - grpcio==1.51.1
      - ipython==8.7.0
      - jedi==0.18.2
      - levenshtein==0.20.9
      - markdown==3.4.1
      - markupsafe==2.1.1
      - matplotlib-inline==0.1.6
      - more-itertools==9.0.0
      - oauthlib==3.2.2
      - parso==0.8.3
      - pexpect==4.8.0
      - pickleshare==0.7.5
      - prompt-toolkit==3.0.36
      - psutil==5.9.4
      - ptyprocess==0.7.0
      - pudb==2022.1.3
      - pure-eval==0.2.2
      - py-spy==0.3.3+computecanada
      - pyasn1==0.4.8
      - pyasn1-modules==0.2.8
      - pygments==2.13.0
      - rapidfuzz==2.13.7
      - requests-oauthlib==1.3.1
      - rsa==4.9
      - stack-data==0.6.2
      - tensorboard==2.11.0
      - tensorboard-data-server==0.6.1
      - tensorboard-plugin-wit==1.8.1
      - traitlets==5.7.1
      - urwid==2.1.2
      - urwid-readline==0.13
      - wcwidth==0.2.5
      - werkzeug==2.2.2
prefix: /gpfs/projects/DT/mtp/WMT20/opt/miniconda3/envs/transformers-4.20.1
```

I should point out that I'm running `examples/pytorch/language-modeling/run_mlm.py`.

I said on the the main branch. You have Transformers 4.20 installed, you need a source install.

Thank you for the much needed speed improvement.
I'm now using `accelerate==0.16.0` with `transformers==4.27.0.dev0` and it no longer takes 11h to skip the first batches.

> Thank you for the much needed speed improvement. I'm now using `accelerate==0.16.0` with `transformers==4.27.0.dev0` and it no longer takes 11h to skip the first batches.

how long does it take now?
for me I've been waiting for 5 hrs and the training has not even started.