README.md

---
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: NousResearch/Llama-2-7b-chat-hf
model-index:
- name: results
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

#dataset-used: codeparrot/xlcost-text-to-code

#github notebook: https://github.com/manishzed/LLM-Fine-tune/blob/main/Llama_2_7b_chat_fine_tune_text_to_python.ipynb

#code

```python
#testing and loading model

import torch, gc
gc.collect()
torch.cuda.empty_cache()

import numpy as np
import pandas as pd
import os
from tqdm import tqdm
import bitsandbytes as bnb
import torch
import torch.nn as nn
import transformers
from datasets import Dataset
from peft import LoraConfig, PeftConfig
from trl import SFTTrainer
from transformers import (AutoModelForCausalLM,
                          AutoTokenizer,
                          BitsAndBytesConfig,
                          TrainingArguments,
                          pipeline,
                          logging)
from sklearn.metrics import (accuracy_score,
                             classification_report,
                             confusion_matrix)
from sklearn.model_selection import train_test_split

from datasets import load_dataset

#testing----1


# Ruta del modelo guardado en el dataset de Kaggle
from peft import LoraConfig, PeftModel

device_map = {"": 0}
PEFT_MODEL = "kr-manish/Llama-2-7b-chat-fine-tune-text-to-python"
#model_name = "NousResearch/Llama-2-7b-hf"

# Cargar la configuración del modelo
config = PeftConfig.from_pretrained(PEFT_MODEL)

# Cargar el modelo
model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    low_cpu_mem_usage=True,
    return_dict=True,
    #quantization_config=bnb_config,
    device_map="auto",
    #trust_remote_code=True,
    torch_dtype=torch.float16,
)

# Cargar el tokenizador
tokenizer=AutoTokenizer.from_pretrained(config.base_model_name_or_path)
tokenizer.pad_token = tokenizer.eos_token

# Cargar el modelo PEFT
load_model = PeftModel.from_pretrained(model, PEFT_MODEL)

input_text ="Program to convert Centimeters to Pixels | Function to convert centimeters to pixels ; Driver Code"
prompt_test = input_text
pipe_test = pipeline(task="text-generation",
                model=load_model,
                tokenizer=tokenizer,
                max_length =200,
                #max_new_tokens =25,
                )
#result_test = pipe_test(prompt_test)
#answer_test = result_test[0]['generated_text']
#answer_test
#or
result = pipe_test(f"<s>[INST] {input_text} [/INST]")
print(result[0]['generated_text'])

#Program to convert Centimeters to Pixels | Function to convert centimeters to pixels ; Driver Code [/code] def convertCentimetersToPixels ( cm ) : NEW_LINE INDENT pixels =

```

#code

# results

This model is a fine-tuned version of [NousResearch/Llama-2-7b-chat-hf](https://huggingface.co/NousResearch/Llama-2-7b-chat-hf) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.7746

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 8
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 1
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 0.7836        | 1.0   | 463  | 0.7746          |


### Framework versions

- PEFT 0.10.0
- Transformers 4.38.2
- Pytorch 2.2.1+cu121
- Datasets 2.18.0
- Tokenizers 0.15.2