|
|
|
|
|
|
|
|
|
--- |
|
license: apache-2.0 |
|
datasets: |
|
- NickyNicky/oasst2_clusters |
|
- OpenAssistant/oasst2 |
|
model: |
|
- google/gemma-2b-it |
|
language: |
|
- bg |
|
- ca |
|
- cs |
|
- da |
|
- de |
|
- en |
|
- es |
|
- fr |
|
- hr |
|
- hu |
|
- it |
|
- nl |
|
- pl |
|
- pt |
|
- ro |
|
- ru |
|
- sl |
|
- sr |
|
- sv |
|
- uk |
|
|
|
library_name: transformers |
|
|
|
widget: |
|
- text: | |
|
<bos><start_of_turn>system\nYou are a helpful AI assistant.<end_of_turn>\n<start_of_turn>user\nescribe una historia de 100 palabras<end_of_turn>\n<start_of_turn>model\n |
|
|
|
--- |
|
|
|
|
|
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/YXqUXFjX8uIJT-mdOnM1h.png) |
|
|
|
``` |
|
reference data model: |
|
|
|
datasets: |
|
- lang: "bg,ca,cs,da,de,en,es,fr,hr,hu,it,nl,pl,pt,ro,ru,sl,sr,sv,uk" |
|
link: https://huggingface.co/datasets/NickyNicky/oasst2_clusters |
|
|
|
model: |
|
- google/gemma-2b-it |
|
Link: |
|
https://huggingface.co/google/gemma-2b-it |
|
|
|
base fine tune: google/gemma-2b-it |
|
|
|
Epoch: 3 |
|
|
|
future experts: 8 |
|
|
|
Eval model: |
|
- link: |
|
soon |
|
|
|
``` |
|
|
|
|
|
## train/loss: 0.4943 |
|
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/wRZtjbfYp2OcG16yeNQbG.png) |
|
|
|
|
|
## dataset tokens: |
|
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/xmf6SCGVtFpkJBrtIalj1.png) |
|
|
|
|
|
```Python |
|
!python -m pip install --upgrade pip |
|
!pip install "torch>=2.1.1" -U |
|
!pip install to |
|
![image/png](https://cdn-uploads.huggingface.co/production/uploads/641b435ba5f876fe30c5ae0a/atkOgaHsp1oRX1Ua1dFEA.png)ll -q datasets trl peft bitsandbytes sentencepiece wandb |
|
!pip install -q accelerate safetensors deepspeed |
|
!pip install -q scipy ninja -U |
|
!pip install -q -U transformers==4.38.0 |
|
!pip install flash-attn==2.5.5 --no-build-isolation |
|
``` |
|
|
|
|
|
## Version |
|
```py |
|
import torch |
|
torch.__version__ |
|
#OUTPUTS: ('2.2.0+cu121' ) |
|
``` |
|
|
|
## How to use |
|
```py |
|
|
|
from transformers import ( |
|
AutoModelForCausalLM, |
|
AutoTokenizer, |
|
BitsAndBytesConfig, |
|
HfArgumentParser, |
|
TrainingArguments, |
|
pipeline, |
|
logging, |
|
GenerationConfig, |
|
TextIteratorStreamer, |
|
) |
|
|
|
from transformers import StoppingCriteria, StoppingCriteriaList |
|
|
|
import torch |
|
|
|
model_id='NickyNicky/gemma-2b-it_oasst2_all_chatML_Unsloth_V1' |
|
|
|
model = AutoModelForCausalLM.from_pretrained(model_id, |
|
device_map="auto", |
|
trust_remote_code=True, |
|
torch_dtype=torch.bfloat16, |
|
attn_implementation="flash_attention_2", |
|
# load_in_4bit=True, |
|
# low_cpu_mem_usage= True, |
|
|
|
) |
|
|
|
max_length=2048 |
|
print("max_length",max_length) |
|
|
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id, |
|
# use_fast = False, |
|
max_length=max_length,) |
|
|
|
|
|
class ListOfTokensStoppingCriteria(StoppingCriteria): |
|
""" |
|
Clase para definir un criterio de parada basado en una lista de tokens específicos. |
|
""" |
|
def __init__(self, tokenizer, stop_tokens): |
|
self.tokenizer = tokenizer |
|
# Codifica cada token de parada y guarda sus IDs en una lista |
|
self.stop_token_ids_list = [tokenizer.encode(stop_token, add_special_tokens=False) for stop_token in stop_tokens] |
|
|
|
def __call__(self, input_ids, scores, **kwargs): |
|
# Verifica si los últimos tokens generados coinciden con alguno de los conjuntos de tokens de parada |
|
for stop_token_ids in self.stop_token_ids_list: |
|
len_stop_tokens = len(stop_token_ids) |
|
if len(input_ids[0]) >= len_stop_tokens: |
|
if input_ids[0, -len_stop_tokens:].tolist() == stop_token_ids: |
|
return True |
|
return False |
|
|
|
# Uso del criterio de parada personalizado |
|
stop_tokens = ["<end_of_turn>"] # Lista de tokens de parada |
|
|
|
# Inicializa tu criterio de parada con el tokenizer y la lista de tokens de parada |
|
stopping_criteria = ListOfTokensStoppingCriteria(tokenizer, stop_tokens) |
|
|
|
# Añade tu criterio de parada a una StoppingCriteriaList |
|
stopping_criteria_list = StoppingCriteriaList([stopping_criteria]) |
|
|
|
|
|
#EXAMPLE #1 |
|
input_text = f"""<bos><start_of_turn>system |
|
You are a helpful AI assistant.<end_of_turn> |
|
<start_of_turn>user |
|
**News:** |
|
he Texas Blockchain Council (TBC) and Bitcoin mining firm Riot Platforms have won a favorable ruling from a United States District Judge in a lawsuit against several United States energy officials. |
|
On February 22, Cointelegraph reported that the TBC and Riot alleged the U.S. Department of Energy, Energy Information Administration (EIA), Office of Management and Budget (OMB) and their leadership sought an “invasive” data collection from cryptocurrency miners. |
|
According to a February 23 filing in the United States District Court for the Western District of Texas, the TBC and Riot convinced the judge that irreversible harm would happen without a temporary restraining order (TRO) against further data collection. |
|
As a result, the court enforced a TRO which prohibits the EIA from requiring crypto miners to respond to the survey, as well as prohibiting the EIA from sharing any data that has already been received from the survey. |
|
“The Court finds that Plaintiffs have shown through a verified complaint and supporting evidence that immediate and irreparable injury, loss, or damage will result if a TRO is not issued.” |
|
|
|
|
|
Instruccion: |
|
- responde en español. |
|
- has un análisis sobre el contexto de la noticia y buscar información relevante para poder responder satisfactoriamente. |
|
- has 5 preguntas importantes y sus respuestas. |
|
|
|
en español responde solo en json: |
|
```json |
|
{ |
|
"analisis_noticia": "", |
|
"preguntas_respuestas": [ |
|
{ |
|
"pregunta": "", |
|
"respuesta": "" |
|
} |
|
] |
|
}```<end_of_turn> |
|
<start_of_turn>model |
|
""" |
|
|
|
inputs = tokenizer.encode(txt, |
|
return_tensors="pt", |
|
add_special_tokens=False).to("cuda:0") |
|
max_new_tokens=1500 |
|
generation_config = GenerationConfig( |
|
max_new_tokens=max_new_tokens, |
|
temperature=0.15, |
|
# top_p=0.55, |
|
top_k=50, |
|
repetition_penalty=1.1, |
|
do_sample=True, |
|
) |
|
outputs = base_model.generate(generation_config=generation_config, |
|
input_ids=inputs, |
|
stopping_criteria=stopping_criteria_list,) |
|
print(tokenizer.decode(outputs[0], skip_special_tokens=False) ) |
|
``` |
|
|
|
``` |
|
''' |
|
### OUTPUT EXAMPLE |
|
<start_of_turn>model |
|
|
|
''' |
|
``` |