Sagemaker Estimator for fine tuning where all the transform code is in the train.py
Author: j2cunningham
Pull request: no

### Feature request

I work for a company that is a heavy user of AWS sagemaker.  I am on a professional services team where I build a lot of examples for our data scientists to follow.  I recently wanted to use the Sagemaker Huggingface estimator to fine tune a transformer and create a model for our custom NLP task.

I had csv data in S3.  I found several examples of fine tuning that involved pulling nicely curated datasets from HF hub down to the SM notebook and then transforming it into arrow with `save_to_disk` and pushing it to S3 as a dataset that could be read in the train.py file.  

I struggled mightily to find an example and never found a good example of how to start with just CSV files, use the HF existing tools load the data and then pass it to the estimator.  Furthermore, the examples I find have the user pulling the data over to the notebook and doing the conversion to arrow there.  That seems inefficient when the point of an estimator is to utilize a small instance to host your notebook and a large instance to do the work.  If I had a large amount of data to to convert to arrow and I followed the given examples, I would need a large notebook instance and a large estimator instance.  

I wrote an example that puts all the transform code in the train.py and only invokes it from the notebook.  In my train.py, I use load_dataset with the csv script to transform the data to arrow and do the save and load there.  I wanted to use the arrow format for efficiency.  

I propose that I update your documentation with this unique example.

### Motivation

I feel that the proposed documentation is unifies several previously documented concepts into a single, useful example.

### Your contribution

I would be happy to build the example and have you guys approve it.  I have never contributed to HF before, so I would need a bit of guidance to get started.

Comments:
WDYT @philschmid @sgugger ?

Hello @j2cunningham, 

Thank you for all of the information and it is super cool to hear that you are using SageMaker! We have over 20 examples for how to use transformers with SageMaker for inference and training: https://github.com/huggingface/notebooks/tree/main/sagemaker

In there should be examples of how to use a CSV file directly for [batch transform](https://github.com/huggingface/notebooks/blob/main/sagemaker/12_batch_transform_inference/sagemaker-notebook.ipynb), and we also ran a whole [workshop series last year](https://github.com/philschmid/huggingface-sagemaker-workshop-series),  where you have example of the doing the processing in the [train.py](https://github.com/philschmid/huggingface-sagemaker-workshop-series/blob/main/workshop_4_distillation_and_acceleration/scripts/train.py)

Regarding your struggle with loading CSV compared to regular datasets, this quite easy. Instead of providing the huggingface hub id you can use `csv` and then provide the path to your files. This will then created a dataset you can seamlessly use with the examples: [Documentation](https://huggingface.co/docs/datasets/v2.6.0/en/loading#csv)

```python
from datasets import load_dataset
dataset = load_dataset("csv", data_files="my_file.csv"
```

For more SageMaker related question or ideas please use the forum next time: https://discuss.huggingface.co/c/sagemaker/17


I totally get how to use load_dataset with csv data now. My observation is that there isn't an example that starts with just plain csv data, explains how to use load_dataset and why and then does some fine tuning.  The examples I found would all start with nicely curated huggingface datasets, do a save to disk (not explain why) and then do read from disk in the train.py.  I had to find snippets and documentation that used load_dataset for csv, snippets that explained what save_to_disk was doing and what arrow was, snippets that explained that you could do transformation in the notebook or in the train.py and then wrap it all up into working code.  I just feel like starting from raw csv data or image data and doing most of the work in the train.py and not the notebook is pretty common pattern.  There very well could be the perfect example from HF that I couldn't find or I could be off the mark when I think this is a common pattern outside of my company.  I got this all working and am just offering to share my notebook and train.py with the community.  I know I could do a medium article, but thought an example on the HF git would be most beneficial.  Thanks

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.