Update README.md
Browse files
README.md
CHANGED
|
@@ -106,7 +106,7 @@ emb = (out.last_hidden_state * inputs.attention_mask.unsqueeze(-1)).sum(dim=1) /
|
|
| 106 |
|
| 107 |
## Model Description
|
| 108 |
|
| 109 |
-
RexBERT-micro is an **encoder-only**,
|
| 110 |
|
| 111 |
---
|
| 112 |
|
|
|
|
| 106 |
|
| 107 |
## Model Description
|
| 108 |
|
| 109 |
+
RexBERT-micro is an **encoder-only**, 17M parameter transformer trained with a masked-language-modeling objective and optimized for **e-commerce related text**. The three-phase training curriculum improves general language understanding, extends context handling, and then **specializes** on a very large corpus of commerce data to capture domain-specific terminology and entity distributions.
|
| 110 |
|
| 111 |
---
|
| 112 |
|