Text Classification
Transformers
Safetensors
English
bert
fill-mask
BERT
NeuroBERT
transformer
pre-training
nlp
tiny-bert
edge-ai
low-resource
micro-nlp
quantized
iot
wearable-ai
offline-assistant
intent-detection
real-time
smart-home
embedded-systems
command-classification
toy-robotics
voice-ai
eco-ai
english
lightweight
mobile-nlp
ner
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,163 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
datasets:
|
| 4 |
+
- wikimedia/wikipedia
|
| 5 |
+
- bookcorpus/bookcorpus
|
| 6 |
+
- SetFit/mnli
|
| 7 |
+
- sentence-transformers/all-nli
|
| 8 |
+
language:
|
| 9 |
+
- en
|
| 10 |
+
new_version: v1.1
|
| 11 |
+
base_model:
|
| 12 |
+
- google-bert/bert-base-uncased
|
| 13 |
+
pipeline_tag: text-classification
|
| 14 |
+
tags:
|
| 15 |
+
- BERT
|
| 16 |
+
- MNLI
|
| 17 |
+
- NLI
|
| 18 |
+
- transformer
|
| 19 |
+
- pre-training
|
| 20 |
+
- nlp
|
| 21 |
+
- tiny-bert
|
| 22 |
+
- edge-ai
|
| 23 |
+
- transformers
|
| 24 |
+
- low-resource
|
| 25 |
+
- micro-nlp
|
| 26 |
+
- quantized
|
| 27 |
+
- iot
|
| 28 |
+
- wearable-ai
|
| 29 |
+
- offline-assistant
|
| 30 |
+
- intent-detection
|
| 31 |
+
- real-time
|
| 32 |
+
- smart-home
|
| 33 |
+
- embedded-systems
|
| 34 |
+
- command-classification
|
| 35 |
+
- toy-robotics
|
| 36 |
+
- voice-ai
|
| 37 |
+
- eco-ai
|
| 38 |
+
- english
|
| 39 |
+
- lightweight
|
| 40 |
+
- mobile-nlp
|
| 41 |
+
metrics:
|
| 42 |
+
- accuracy
|
| 43 |
+
- f1
|
| 44 |
+
- inference
|
| 45 |
+
- recall
|
| 46 |
+
library_name: transformers
|
| 47 |
+
---
|
| 48 |
+
|
| 49 |
+

|
| 50 |
+
|
| 51 |
+
# π§ boltuix/bert-mini β Ultra Lightweight BERT for Real-Time NLP π
|
| 52 |
+
|
| 53 |
+
[](https://opensource.org/licenses/MIT)
|
| 54 |
+
[](#)
|
| 55 |
+
[](#)
|
| 56 |
+
[](#)
|
| 57 |
+
|
| 58 |
+
`bert-mini` is a compact, real-time NLP model derived from BERT but streamlined for blazing-fast inference on constrained hardware β think IoT, wearables, and mobile apps. π°οΈ
|
| 59 |
+
|
| 60 |
+
---
|
| 61 |
+
|
| 62 |
+
## π Key Features
|
| 63 |
+
|
| 64 |
+
| Feature | Description |
|
| 65 |
+
|------------------------|-------------------------------------------------------|
|
| 66 |
+
| π **Architecture** | Lightweight BERT (β4 layers, hidden size 256) |
|
| 67 |
+
| βοΈ **Parameters** | ~11M (vs. 110M in BERT-base) |
|
| 68 |
+
| πΎ **Model Size** | ~44MB (quantized) |
|
| 69 |
+
| β‘ **Speed** | Real-time inference on mobile and edge devices |
|
| 70 |
+
| π **Use Cases** | NLI, intent detection, voice assistants, offline chat |
|
| 71 |
+
| π **Datasets** | Wikipedia, BookCorpus, MNLI, All-NLI |
|
| 72 |
+
| π§ͺ **Training Tasks** | Masked LM, NLI classification |
|
| 73 |
+
| π **License** | MIT β free to use, modify, and distribute |
|
| 74 |
+
|
| 75 |
+
---
|
| 76 |
+
|
| 77 |
+
## π¦ Installation
|
| 78 |
+
|
| 79 |
+
Install dependencies:
|
| 80 |
+
|
| 81 |
+
```bash
|
| 82 |
+
pip install transformers torch
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
## π€ Quickstart: Masked Language Prediction
|
| 88 |
+
|
| 89 |
+
```python
|
| 90 |
+
from transformers import pipeline
|
| 91 |
+
|
| 92 |
+
# Load the pipeline
|
| 93 |
+
mlm_pipeline = pipeline("fill-mask", model="boltuix/bert-mini")
|
| 94 |
+
|
| 95 |
+
# Try a sentence
|
| 96 |
+
result = mlm_pipeline("The robot can [MASK] the room in minutes.")
|
| 97 |
+
print(result[0]["sequence"])
|
| 98 |
+
```
|
| 99 |
+
|
| 100 |
+
---
|
| 101 |
+
|
| 102 |
+
## π‘ Sample Outputs
|
| 103 |
+
|
| 104 |
+
```python
|
| 105 |
+
Input: The device can [MASK] quickly.
|
| 106 |
+
β¨ β the device can operate quickly.
|
| 107 |
+
β¨ β the device can function quickly.
|
| 108 |
+
|
| 109 |
+
Input: Please [MASK] the door before leaving.
|
| 110 |
+
β¨ β please open the door before leaving.
|
| 111 |
+
β¨ β please shut the door before leaving.
|
| 112 |
+
```
|
| 113 |
+
|
| 114 |
+
---
|
| 115 |
+
|
| 116 |
+
## π¬ Evaluation Metrics
|
| 117 |
+
|
| 118 |
+
| Metric | Value (Approx.) |
|
| 119 |
+
|------------|-----------------------|
|
| 120 |
+
| β
Accuracy | ~90β97% of BERT-base |
|
| 121 |
+
| π― F1 Score | Balanced performance |
|
| 122 |
+
| β‘ Latency | Fast on Raspberry Pi / Android |
|
| 123 |
+
|
| 124 |
+
---
|
| 125 |
+
|
| 126 |
+
## π Use Cases
|
| 127 |
+
|
| 128 |
+
- π **Voice Assistants**: Smart speaker command disambiguation
|
| 129 |
+
- π **Smart Homes**: On-device NLP for offline automation
|
| 130 |
+
- π€ **Toy & Robotics**: Lightweight command understanding
|
| 131 |
+
- β **Wearables**: Real-time sentiment & intent detection
|
| 132 |
+
- π§ͺ **AI on Budget**: NLP on minimal compute resources
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
## π Trained On
|
| 137 |
+
|
| 138 |
+
- **Wikipedia**
|
| 139 |
+
- **BookCorpus**
|
| 140 |
+
- **MNLI** (MultiNLI)
|
| 141 |
+
- **All-NLI** from `sentence-transformers`
|
| 142 |
+
|
| 143 |
+
---
|
| 144 |
+
|
| 145 |
+
## π·οΈ Tags
|
| 146 |
+
|
| 147 |
+
`#tiny-bert` `#edge-ai` `#intent-detection` `#nlp` `#smart-home` `#wearable-ai` `#offline-assistant` `#transformers` `#real-time`
|
| 148 |
+
|
| 149 |
+
---
|
| 150 |
+
|
| 151 |
+
## π License
|
| 152 |
+
|
| 153 |
+
MIT License β free for commercial and personal use.
|
| 154 |
+
|
| 155 |
+
---
|
| 156 |
+
|
| 157 |
+
## π Credits
|
| 158 |
+
|
| 159 |
+
Developed by [Hari Shankar S (boltuix)](https://huggingface.co/boltuix)
|
| 160 |
+
Base Model: [`google-bert/bert-base-uncased`](https://huggingface.co/google-bert/bert-base-uncased)
|
| 161 |
+
Optimized and Quantized for edge AI scenarios.
|
| 162 |
+
|
| 163 |
+
---
|