Hugging Face's logo

sandbox-ai
/
Llama-3.1-Tango-8b-int4

Text Generation
Transformers
Safetensors
English
Spanish
llama
text-generation-inference
unsloth
trl
sft
tango
Inference Endpoints
4-bit precision
bitsandbytes
Model card Files Community
Llama-3.1-Tango-8b-int4 / README.md
tatakof's picture
tatakof
Update README.md
f6585d5 verified
preview code
|
raw
history blame contribute delete
18.8 kB
metadata
base_model: unsloth/meta-llama-3.1-8b-bnb-4bit
tags:
  - text-generation-inference
  - transformers
  - unsloth
  - llama
  - trl
  - sft
  - tango
license: llama3.1
language:
  - en
  - es
datasets:
  - spanish-ir/messirve
  - tatakof/messi_mod-v0.0.2
pipeline_tag: text-generation
library_name: transformers

Uploaded model

  • Developed by: sandbox-ai
  • License: apache-2.0
  • Finetuned from model : unsloth/meta-llama-3.1-8b-bnb-4bit

Evaluation Metrics

Task Name Description Language Metric Task type
AQuAS AQuAS Abstractive Question-Answering in Spanish ES sas_encoder Abstractive QA
ARC_ca ARC_ca Grade-school level science questions in Catalan CA acc Multi choice QA
BEC2016eu BEC2016eu Basque Election Campaign 2016 Opinion Dataset EU f1 Sentiment Analysis
Belebele Glg Belebele Glg Reading Comprehension in Galician GL acc Reading Comprehension
BertaQA BertaQA Trivia dataset with global and local questions about the Basque Country EU acc Multi choice QA
BHTCv2 BHTCv2 Topic Classification of News Headlines in Basque EU f1 Classification, Topic Classification
caBREU caBREU Article Summarization in Catalan CA bleu Summarization
CatalanQA CatalanQA Extractive QA in Catalan CA f1 Extractive QA
CatCoLA CatCoLA Linguistic Acceptability in Catalan CA mcc Linguistic Acceptability
ClinDiagnosES ClinDiagnosES Diagnosis of clinical cases in Spanish ES sas_encoder Open QA
ClinTreatES ClinTreatES Treatment for clinical cases in Spanish ES sas_encoder Open QA
COPA_ca COPA_ca Choice Of Plausible Alternatives in Catalan CA acc Reasoning
CoQCat CoQCat Conversational Question Answering in Catalan CA f1 Extractive QA
Crows Pairs Spanish Crows Pairs Spanish Bias evaluation using stereotypes ES pct_stereotype Bias Detection
EpecKorrefBin EpecKorrefBin Coreference resolution in Basque EU acc Coreference Resolution, Textual Entailment
EsCoLA EsCoLA Spanish Corpus of Linguistic Acceptability ES mcc Linguistic Acceptability
EusExams EusExams Public Service examinations questions in Basque EU acc Multi choice QA
EusProficiency EusProficiency C1-level proficiency questions in Basque EU acc Multi choice QA
EusReading EusReading EGA exams reading comprehension in Basque EU acc Multi choice QA
EusTrivia EusTrivia Trivia questions in Basque EU acc Multi choice QA
Fake News ES Fake News ES Fake News Detection in Spanish ES acc Classification
GalCoLA GalCoLA Galician Corpus of Linguistic Acceptability GL mcc Linguistic Acceptability
HumorQA HumorQA White humour joke classification ES acc Classification
MGSM_ca MGSM_ca Grade-school math problems in Catalan CA exact_match Math Reasoning
MGSM_es MGSM_es Grade-school math problems in Spanish ES exact_match Math Reasoning
MGSM_eu MGSM_eu Grade-school math problems in Basque EU exact_match Math Reasoning
MGSM_gl MGSM_gl Grade-school math problems in Galician GL exact_match Math Reasoning
NoticIA NoticIA A Clickbait Article Summarization Dataset in Spanish ES rouge1 Summarization
OffendES OffendES Clasificación de comentarios ofensivos en español ES acc Classification
OpenBookQA_ca OpenBookQA_ca Multi-step reasoning QA in Catalan CA acc Reasoning
OpenBookQA_gl OpenBookQA_gl Multi-step reasoning QA in Galician GL acc Reasoning
Parafraseja Parafraseja Paraphrase identification in Catalan CA acc Paraphrasing
ParafrasesGL ParafrasesGL Paraphrase identification in Galician GL acc Paraphrasing
PAWS_ca PAWS_ca Paraphrase Adversaries from Word Scrambling in Catalan CA acc Paraphrasing
PAWS-X_es PAWS-X_es Paraphrase Adversaries from Word Scrambling in Spanish ES acc Paraphrasing
PAWS_gl PAWS_gl Paraphrase Adversaries from Word Scrambling in Galician GL acc Paraphrasing
PIQA_ca PIQA_ca Physical Interaction QA in Catalan CA acc Reasoning
QNLIeu QNLIeu Textual Entailment in Basque EU acc NLI, Textual Entailment
RagQuAS RagQuAS Retrieval-Augmented-Generation and Question-Answering in Spanish ES sas_encoder Abstractive QA
SIQA_ca SIQA_ca Social Interaction QA in Catalan CA acc Reasoning
SpaLawEx SpaLawEx Spanish Law School Access Exams ES acc Multi choice QA
SummarizationGL SummarizationGL Abstractive Summarization in Galician GL bleu Summarization
TE-ca TE-ca Textual Entailment in Catalan CA acc Textual Entailment
TELEIA TELEIA Test de Español como Lengua Extranjera para Inteligencia Artificial ES acc Multi choice QA
VaxxStance VaxxStance Stance detection on the Antivaxxers movement EU f1 Sentiment Analysis, Stance Detection
WiCeu WiCeu Word sense disambiguation in Basque EU acc Textual Entailment
WNLI_ca WNLI_ca Winograd-schema-type dataset in Catalan CA acc NLI, Textual Entailment
WNLI ES WNLI ES Winograd-schema-type dataset in Spanish ES acc NLI, Textual Entailment
XCOPA_eu XCOPA_eu Choice Of Plausible Alternatives in Basque EU acc Reasoning
XNLI_ca XNLI_ca Cross-lingual Natural Language Inference in Catalan CA acc NLI, Textual Entailment
XNLI_es XNLI_es Cross-lingual Natural Language Inference in Spanish ES acc NLI
XNLI_eu XNLI_eu Cross-lingual Natural Language Inference in Basque EU acc NLI, Textual Entailment
XQuAD_ca XQuAD_ca Cross-lingual Question Answering Dataset in Catalan CA f1 Extractive QA
XQuAD_es XQuAD_es Cross-lingual Question Answering Dataset in Spanish ES f1 Extractive QA
xStoryCloze_ca xStoryCloze_ca Narrative completion in Catalan CA acc Reasoning
xStoryCloze_es xStoryCloze_es Narrative completion in Spanish ES acc Reasoning
xStoryCloze_eu xStoryCloze_eu Narrative completion in Basque EU acc Reasoning

Usage:

You can use the model using HuggingFace Transformers library with 2 or more 80GB GPUs (NVIDIA Ampere or newer) with at least 150GB of free disk space to accomodate the download.

This code has been tested on Transformers v4.44.0, torch v2.4.0 and 2 A100 80GB GPUs, but any setup that supports meta-llama/Llama-3.1-70B-Instruct should support this model as well. If you run into problems, you can consider doing pip install -U transformers.

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sandbox-ai/Llama-3.1-Tango-8b-int4")

References(s):

  • TODO

Model Architecture:

Architecture Type: Transformer
Network Architecture: Llama 3.1

Input:

Input Type(s): Text
Input Format: String
Input Parameters: One Dimensional (1D)
Other Properties Related to Input: Max of 128k tokens

Output:

Output Type(s): Text
Output Format: String
Output Parameters: One Dimensional (1D)
Other Properties Related to Output: Max of 4k tokens

Training & Evaluation:

  • TODO

Dataset:

MessIRve: A Large-Scale Spanish Information Retrieval Dataset

Citation 

@article{valentini2024messirve,
      title={MessIRve: A Large-Scale Spanish Information Retrieval Dataset}, 
      author={Francisco Valentini and Viviana Cotik and Damián Furman and Ivan Bercovich and Edgar Altszyler and Juan Manuel Pérez},
      year={2024},
      eprint={2409.05994},
      journal={arxiv:2409.05994},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2409.05994}, 
}

@misc{wang2024helpsteer2preferencecomplementingratingspreferences,
      title={HelpSteer2-Preference: Complementing Ratings with Preferences}, 
      author={Zhilin Wang and Alexander Bukharin and Olivier Delalleau and Daniel Egert and Gerald Shen and Jiaqi Zeng and Oleksii Kuchaiev and Yi Dong},
      year={2024},
      eprint={2410.01257},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2410.01257}, 
}

This llama model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)