Hugging Face's logo

tensorblock
/
Aira-2-1B1-GGUF

Text Generation
Transformers
GGUF
English
alignment
instruction tuned
text generation
conversation
assistant
TensorBlock
GGUF
Inference Endpoints
Model card Files Community
Aira-2-1B1-GGUF / README.md
morriszms's picture
morriszms
Upload folder using huggingface_hub
1894221 verified
preview code
|
raw
history blame contribute delete
5.21 kB
metadata
license: apache-2.0
datasets:
  - nicholasKluge/instruct-aira-dataset
language:
  - en
metrics:
  - accuracy
library_name: transformers
tags:
  - alignment
  - instruction tuned
  - text generation
  - conversation
  - assistant
  - TensorBlock
  - GGUF
pipeline_tag: text-generation
widget:
  - text: Can you explain what is Machine Learning?<|endofinstruction|>
    example_title: Machine Learning
  - text: Do you know anything about virtue ethics?<|endofinstruction|>
    example_title: Ethics
  - text: How can I make my girlfriend happy?<|endofinstruction|>
    example_title: Advise
inference:
  parameters:
    repetition_penalty: 1.2
    temperature: 0.2
    top_k: 30
    top_p: 0.3
    max_new_tokens: 200
    length_penalty: 0.3
    early_stopping: true
co2_eq_emissions:
  emissions: 1710
  source: CodeCarbon
  training_type: fine-tuning
  geographical_location: Singapore
  hardware_used: NVIDIA A100-SXM4-40GB
base_model: nicholasKluge/Aira-2-1B1
TensorBlock

Feedback and support: TensorBlock's Twitter/X, Telegram Group and Discord server

nicholasKluge/Aira-2-1B1 - GGUF

This repo contains GGUF format model files for nicholasKluge/Aira-2-1B1.

The files were quantized using machines provided by TensorBlock, and they are compatible with llama.cpp as of commit b4242.

Prompt template

Model file specification

Filename Quant type File Size Description
Aira-2-1B1-Q2_K.gguf Q2_K 0.432 GB smallest, significant quality loss - not recommended for most purposes
Aira-2-1B1-Q3_K_S.gguf Q3_K_S 0.499 GB very small, high quality loss
Aira-2-1B1-Q3_K_M.gguf Q3_K_M 0.548 GB very small, high quality loss
Aira-2-1B1-Q3_K_L.gguf Q3_K_L 0.592 GB small, substantial quality loss
Aira-2-1B1-Q4_0.gguf Q4_0 0.637 GB legacy; small, very high quality loss - prefer using Q3_K_M
Aira-2-1B1-Q4_K_S.gguf Q4_K_S 0.640 GB small, greater quality loss
Aira-2-1B1-Q4_K_M.gguf Q4_K_M 0.668 GB medium, balanced quality - recommended
Aira-2-1B1-Q5_0.gguf Q5_0 0.766 GB legacy; medium, balanced quality - prefer using Q4_K_M
Aira-2-1B1-Q5_K_S.gguf Q5_K_S 0.766 GB large, low quality loss - recommended
Aira-2-1B1-Q5_K_M.gguf Q5_K_M 0.782 GB large, very low quality loss - recommended
Aira-2-1B1-Q6_K.gguf Q6_K 0.903 GB very large, extremely low quality loss
Aira-2-1B1-Q8_0.gguf Q8_0 1.170 GB very large, extremely low quality loss - not recommended

Downloading instruction

Command line

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/Aira-2-1B1-GGUF --include "Aira-2-1B1-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/Aira-2-1B1-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'