metadata

base_model: BEE-spoke-data/smol_llama-220M-bees-internal
datasets:
  - BEE-spoke-data/bees-internal
inference: false
language:
  - en
license: apache-2.0
metrics:
  - accuracy
model_creator: BEE-spoke-data
model_name: smol_llama-220M-bees-internal
pipeline_tag: text-generation
quantized_by: afrideva
tags:
  - generated_from_trainer
  - gguf
  - ggml
  - quantized
  - q2_k
  - q3_k_m
  - q4_k_m
  - q5_k_m
  - q6_k
  - q8_0
widget:
  - example_title: Queen Excluder
    text: In beekeeping, the term "queen excluder" refers to
  - example_title: Increasing Honey Production
    text: One way to encourage a honey bee colony to produce more honey is by
  - example_title: Lifecycle of a Worker Bee
    text: The lifecycle of a worker bee consists of several stages, starting with
  - example_title: Varroa Destructor
    text: Varroa destructor is a type of mite that
  - example_title: Beekeeping PPE
    text: In the world of beekeeping, the acronym PPE stands for
  - example_title: Robbing in Beekeeping
    text: The term "robbing" in beekeeping refers to the act of
  - example_title: Role of Drone Bees
    text: |-
      Question: What's the primary function of drone bees in a hive?
      Answer:
  - example_title: Honey Harvesting Device
    text: To harvest honey from a hive, beekeepers often use a device known as a
  - example_title: Beekeeping Math Problem
    text: >-
      Problem: You have a hive that produces 60 pounds of honey per year. You
      decide to split the hive into two. Assuming each hive now produces at a
      70% rate compared to before, how much honey will you get from both hives
      next year?

      To calculate
  - example_title: Swarming
    text: In beekeeping, "swarming" is the process where

BEE-spoke-data/smol_llama-220M-bees-internal-GGUF

Quantized GGUF model files for smol_llama-220M-bees-internal from BEE-spoke-data

Name	Quant method	Size
smol_llama-220m-bees-internal.fp16.gguf	fp16	436.50 MB
smol_llama-220m-bees-internal.q2_k.gguf	q2_k	94.43 MB
smol_llama-220m-bees-internal.q3_k_m.gguf	q3_k_m	114.65 MB
smol_llama-220m-bees-internal.q4_k_m.gguf	q4_k_m	137.58 MB
smol_llama-220m-bees-internal.q5_k_m.gguf	q5_k_m	157.91 MB
smol_llama-220m-bees-internal.q6_k.gguf	q6_k	179.52 MB
smol_llama-220m-bees-internal.q8_0.gguf	q8_0	232.28 MB

Original Model Card:

smol_llama-220M-bees-internal

This model is a fine-tuned version of BEE-spoke-data/smol_llama-220M-GQA on the None dataset. It achieves the following results on the evaluation set:

Loss: 2.6892
Accuracy: 0.4610

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 4
eval_batch_size: 2
seed: 27634
gradient_accumulation_steps: 8
total_train_batch_size: 32
optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
3.0959	0.1	50	2.9671	0.4245
2.9975	0.19	100	2.8691	0.4371
2.8938	0.29	150	2.8271	0.4419
2.9027	0.39	200	2.7973	0.4457
2.8983	0.49	250	2.7719	0.4489
2.8789	0.58	300	2.7519	0.4515
2.8672	0.68	350	2.7366	0.4535
2.8369	0.78	400	2.7230	0.4558
2.8271	0.88	450	2.7118	0.4569
2.7775	0.97	500	2.7034	0.4587
2.671	1.07	550	2.6996	0.4592
2.695	1.17	600	2.6965	0.4598
2.6962	1.27	650	2.6934	0.4601
2.6034	1.36	700	2.6916	0.4605
2.716	1.46	750	2.6901	0.4609
2.6968	1.56	800	2.6896	0.4608
2.6626	1.66	850	2.6893	0.4609
2.6881	1.75	900	2.6891	0.4610
2.7339	1.85	950	2.6891	0.4610
2.6729	1.95	1000	2.6892	0.4610

Framework versions

Transformers 4.36.2
Pytorch 2.1.0
Datasets 2.16.1
Tokenizers 0.15.0