File size: 5,824 Bytes
92f94be |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 |
---
base_model: BEE-spoke-data/smol_llama-220M-bees-internal
datasets:
- BEE-spoke-data/bees-internal
inference: false
language:
- en
license: apache-2.0
metrics:
- accuracy
model_creator: BEE-spoke-data
model_name: smol_llama-220M-bees-internal
pipeline_tag: text-generation
quantized_by: afrideva
tags:
- generated_from_trainer
- gguf
- ggml
- quantized
- q2_k
- q3_k_m
- q4_k_m
- q5_k_m
- q6_k
- q8_0
widget:
- example_title: Queen Excluder
text: In beekeeping, the term "queen excluder" refers to
- example_title: Increasing Honey Production
text: One way to encourage a honey bee colony to produce more honey is by
- example_title: Lifecycle of a Worker Bee
text: The lifecycle of a worker bee consists of several stages, starting with
- example_title: Varroa Destructor
text: Varroa destructor is a type of mite that
- example_title: Beekeeping PPE
text: In the world of beekeeping, the acronym PPE stands for
- example_title: Robbing in Beekeeping
text: The term "robbing" in beekeeping refers to the act of
- example_title: Role of Drone Bees
text: 'Question: What''s the primary function of drone bees in a hive?
Answer:'
- example_title: Honey Harvesting Device
text: To harvest honey from a hive, beekeepers often use a device known as a
- example_title: Beekeeping Math Problem
text: 'Problem: You have a hive that produces 60 pounds of honey per year. You decide
to split the hive into two. Assuming each hive now produces at a 70% rate compared
to before, how much honey will you get from both hives next year?
To calculate'
- example_title: Swarming
text: In beekeeping, "swarming" is the process where
---
# BEE-spoke-data/smol_llama-220M-bees-internal-GGUF
Quantized GGUF model files for [smol_llama-220M-bees-internal](https://huggingface.co/BEE-spoke-data/smol_llama-220M-bees-internal) from [BEE-spoke-data](https://huggingface.co/BEE-spoke-data)
| Name | Quant method | Size |
| ---- | ---- | ---- |
| [smol_llama-220m-bees-internal.fp16.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.fp16.gguf) | fp16 | 436.50 MB |
| [smol_llama-220m-bees-internal.q2_k.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q2_k.gguf) | q2_k | 94.43 MB |
| [smol_llama-220m-bees-internal.q3_k_m.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q3_k_m.gguf) | q3_k_m | 114.65 MB |
| [smol_llama-220m-bees-internal.q4_k_m.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q4_k_m.gguf) | q4_k_m | 137.58 MB |
| [smol_llama-220m-bees-internal.q5_k_m.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q5_k_m.gguf) | q5_k_m | 157.91 MB |
| [smol_llama-220m-bees-internal.q6_k.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q6_k.gguf) | q6_k | 179.52 MB |
| [smol_llama-220m-bees-internal.q8_0.gguf](https://huggingface.co/afrideva/smol_llama-220M-bees-internal-GGUF/resolve/main/smol_llama-220m-bees-internal.q8_0.gguf) | q8_0 | 232.28 MB |
## Original Model Card:
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# smol_llama-220M-bees-internal
This model is a fine-tuned version of [BEE-spoke-data/smol_llama-220M-GQA](https://huggingface.co/BEE-spoke-data/smol_llama-220M-GQA) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 2.6892
- Accuracy: 0.4610
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 4
- eval_batch_size: 2
- seed: 27634
- gradient_accumulation_steps: 8
- total_train_batch_size: 32
- optimizer: Adam with betas=(0.9,0.95) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.05
- num_epochs: 2.0
### Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| 3.0959 | 0.1 | 50 | 2.9671 | 0.4245 |
| 2.9975 | 0.19 | 100 | 2.8691 | 0.4371 |
| 2.8938 | 0.29 | 150 | 2.8271 | 0.4419 |
| 2.9027 | 0.39 | 200 | 2.7973 | 0.4457 |
| 2.8983 | 0.49 | 250 | 2.7719 | 0.4489 |
| 2.8789 | 0.58 | 300 | 2.7519 | 0.4515 |
| 2.8672 | 0.68 | 350 | 2.7366 | 0.4535 |
| 2.8369 | 0.78 | 400 | 2.7230 | 0.4558 |
| 2.8271 | 0.88 | 450 | 2.7118 | 0.4569 |
| 2.7775 | 0.97 | 500 | 2.7034 | 0.4587 |
| 2.671 | 1.07 | 550 | 2.6996 | 0.4592 |
| 2.695 | 1.17 | 600 | 2.6965 | 0.4598 |
| 2.6962 | 1.27 | 650 | 2.6934 | 0.4601 |
| 2.6034 | 1.36 | 700 | 2.6916 | 0.4605 |
| 2.716 | 1.46 | 750 | 2.6901 | 0.4609 |
| 2.6968 | 1.56 | 800 | 2.6896 | 0.4608 |
| 2.6626 | 1.66 | 850 | 2.6893 | 0.4609 |
| 2.6881 | 1.75 | 900 | 2.6891 | 0.4610 |
| 2.7339 | 1.85 | 950 | 2.6891 | 0.4610 |
| 2.6729 | 1.95 | 1000 | 2.6892 | 0.4610 |
### Framework versions
- Transformers 4.36.2
- Pytorch 2.1.0
- Datasets 2.16.1
- Tokenizers 0.15.0 |