info/train_a_model.py

LLM_BENCHMARKS_TEXT = f"""
# 🧰 Train a Model

Intel offers a variety of platforms that can be used to train LLMs including data center and consumer grade CPUs, GPUs, and ASICs. 
Below, you can find documentation on how to access free and paid resources to train a model on Intel hardware and submit it to the Hugging Face Model Hub.

<hr> 

## Intel Developer Cloud - Quick Start
The Intel Developer Cloud is one of the best places to access free and paid compute instances for model training. Intel offers Jupyter Notebook instances supported by
224 Core 4th Generation Xeon Bare Metal nodes with 4x GPU Max Series 1100. To access these resources please follow the instructions below:
1. Visit the [Intel Developer Cloud](https://bit.ly/inteldevelopercloud) and sign up for the "Standard - Free" tier to get started.
2. Navigate to the "Training" module under the "Software" section in the left panel.
3. Under the GenAI Essentials section, select the [Gemma Model Fine-tuning using SFT and LoRA](https://console.idcservice.net/training/detail/99deeb99-b0c6-4d02-a1d5-a46d95344ff3) notebook and click "Launch".
4. Follow the instructions in the notebook to train your model using Intel® Data Center GPU Max 1100.
5. Upload your model to the Hugging Face Model Hub.
6. Go to the "Submit" tab on this Leaderboard and follow the instructions to submit your model.

<hr> 

## Training Code Samples
Below are some resources to get you started on training models on Intel platforms:
- Intel® Gaudi® Accelerators
    - [Parameter Efficient Fine-Tuning of Llama-2 70B](https://github.com/HabanaAI/Gaudi-tutorials/blob/main/PyTorch/llama2_fine_tuning_inference/llama2_fine_tuning_inference.ipynb)
- Intel® Xeon® Processors
    - [Distributed Training of GPT2 LLMs on AWS](https://github.com/intel/intel-cloud-optimizations-aws/tree/main/distributed-training)
    - [Fine-tuning Falcon 7B on Xeon Processors](https://medium.com/@eduand-alvarez/fine-tune-falcon-7-billion-on-xeon-cpus-with-hugging-face-and-oneapi-a25e10803a53)
- Intel® Data Center GPU Max Series
    - [Gemma Model Fine-tuning using SFT and LoRA](https://console.idcservice.net/training/detail/99deeb99-b0c6-4d02-a1d5-a46d95344ff3)
    - [LLM Fine-tuning with QLoRA on Max Series GPUs](https://console.idcservice.net/training/detail/159c24e4-5598-3155-a790-2qv973tlm172)

<hr> 

## Submitting your Model to the Hugging Face Model Hub
Once your model is trained, it is a straighforward process to upload and open source it on the Hugging Face Model Hub. The commands from a Jupyter notebook are given below:

```python
# Logging in to Hugging Face
from huggingface_hub import notebook_login, Repository

# Login to Hugging Face
notebook_login()

# Model and Tokenize Loading
from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Define the path to the checkpoint
checkpoint_path = ""  # Replace with your checkpoint folder

# Load the model
model = AutoModelForSequenceClassification.from_pretrained(checkpoint_path)

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("") #add name of your model's tokenizer on Hugging Face OR custom tokenizer

# Save the model and tokenizer
model_name_on_hub = "desired-model-name"
model.save_pretrained(model_name_on_hub)
tokenizer.save_pretrained(model_name_on_hub)

# Push to the hub
model.push_to_hub(model_name_on_hub)
tokenizer.push_to_hub(model_name_on_hub)

# Congratulations! Your fine-tuned model is now uploaded to the Hugging Face Model Hub. 
# You can view and share your model using its URL: https://huggingface.co/<your-username>/<your-model-name>

```
Once your model is uploaded, make sure to update your model card, specifying your use of Intel software and hardware. Hugging Face has a great description on [how to build model cards here](https://huggingface.co/docs/hub/en/model-cards).
"""

SUBMIT_TEXT = f"""
# Use the Resource Below to Start Training a Model Today

"""