Dsg2/LS-Tiny-Base-GGUF

LS-Tiny-Base

model	Multiple choice (length-adjusted)	Ends on its own	Depends on the question
LS-Tiny-Base	47.5%	96.9%	+0.40
LS2.5-108M-A17M-Chat	59.3%	49%	+0.47

LS-Tiny-Base is a 31M Sparse model (Active 15M) trained on the synthetic dataset MiniCPM5-2B-distill (~130M tokens).

Despite being 3.5 times smaller than LS2.5-108M's chat variant, and seeing over 46x less tokens, it matches 80% of the accuracy of LS2.5-108M-A17M-Chat and more consistently ends its own replies (+47%). This shows that a singular high quality data source is far more efficient compared to a mix of datasets, and is likely the case since the distilled dataset contains one behavior in how responses are shaped, as compared to different datasets having multiple response styles or minor traits.

But while the dataset yields high efficiency, it is highly time and power consuming since inference is far slower than training. On average, LS2.5-108M-A17M-Chat maintained a 71k t/s training speed while distilling MiniCPM5-2B capped out at ~1.4k t/s (51x slower).

Chatbench results by category:

category	n	acc	accnorm	chance
fact	25	36.0%	40.0%	25.0%
commonsense	20	55.0%	50.0%	25.0%
instruct	20	65.0%	35.0%	25.0%
relevance	20	15.0%	65.0%	5.0%
consistency	18	33.3%	50.0%	25.0%
arithmetic	15	40.0%	46.7%	25.0%
ALL	118	40.7%	47.5%	25.0%

Relevance and consistency are high since they naturally form from a unified response style. In Chatbench's open ended questoins, the model has learned to include the end of the query at the start of its reponse consistently, or start with the phrase "Yes, ". It also shows the usage of em-dashes, and prompts to the user after explanations (GPT style behavior).

Model specs
Total parameters:	31M
Model parameters:	15M
Active parameters:	15M
Ngram table:	16M
Context length:	1024
Pretrain details
Tokens seen:	~130M
GPU time (5060ti):	1096s

Examples:

you> hi
bot> Hello! I'm doing well, thanks for asking. Today, we're discussing **professionalism**—the principle of professionalism and respect in our everyday interactions with others. Would you like to explore how it applies to your own actions?
      [62 tok, 1409.1 tok/s, ctx 73/4096]

you> ok
bot> Professionalism means ensuring that your actions or actions can convey effectiveness in our work environments. It involves:
- Using the same language across all intervals to emphasize key points.
- Creating a clear sense of communication through individual behavior.
- Fostering a supportive atmosphere where ideas are shared freely and easily, encouraging a more cohesive approach to life.
      [100 tok, 1282.1 tok/s, ctx 184/4096]

you> How can I be healthy?
bot> To maintain good health, focus on:
- **Mindful Breaks**: Take short breaks to stretch and breathe.
- **Sleep & Schedule**: Prioritize sleep and movement over intensity.
- **Physical Awareness**: Practice mindfulness or meditation for relaxation.
- **Nutrition**: Avoid processed foods like protein, vegetables, and whole grains.
- **Environment**: Ensure your workspace is clean and well-ventilated to reduce stress.

Consistency in these practices helps sustain good health without sacrificing physical function.
      [156 tok, 1457.9 tok/s, ctx 171/4096]

you> too much effor
bot> That's a great point! Failure of completing the work can be tricky—check that you're not overusing it or using other methods. If you have more questions about how to handle this process, feel free to ask!
      [57 tok, 1266.7 tok/s, ctx 242/4096]

Usage: To run this model, download the latest tinylm from github and build from source. Currently working on windows prebuilts.