Emojet

One forward pass. Every emoji, the tone of voice, and the way the message is written.

App (Android) · Training code · Results · Quick start

---

Emojet reads a whole chat message — and the message it is replying to — and scores, in a single forward pass:

Group	Outputs	Activation	What it is
emojis	1,923	softmax over the group	every distinct emoji in Unicode 18.0 (skin tones are applied by the app)
tones	65	sigmoid, independent	sarcastic, passive-aggressive, condescending, flirty, resigned, obsequious, …
styles	8	softmax over the group	assertive · passive · aggressive · passive-aggressive · manipulative · collaborative · transactional · avoidant
axes	6	sigmoid, 0–1	funny↔serious, formal↔casual, respectful↔irreverent, enthusiastic↔matter-of-fact, high↔low context, linear↔circular

2,002 scores in one pass — about 180 ms on a laptop CPU (4 threads, 4-bit) and 350–850 ms on a mid-range phone (Redmi Note 17 Pro 5G, 4 big cores). No server, no API key, no network.

It is a fine-tune of Laya (Apache-2.0, ModernBERT-large encoder) with a new classification head, trained on real chat messages rather than synthetic examples.

Why it exists

Most emoji suggestion systems match the last word you typed. Emojet reads the sentence, which is where the meaning actually lives:

You type	Word matching gives	Emojet gives
my program finally works	💻 (program)	🥳 🥲 😭 🥹
oh great, another monday meeting	📅 (monday)	🙄 🙃 😒 😮‍💨
ghum pacche khub (Banglish: very sleepy)	—	😴 💤 🛌 🥱
just adopted a puppy!!	🐶	🐶 🐕 🐾 🦮

Languages

English, Hindi and Bengali — including the romanized, code-mixed way people actually type: Hinglish (kal paper hai aur kuch nahi padha) and Banglish (bhai khub khide peyeche), in both Latin and native script, with the spelling variation that comes with it (pacche / pachhe / pache).

Results

Measured on held-out data the model never trained on. hit@6 = at least one of the six suggested emojis is one the writer actually used — this is the number that matters for a 6-emoji suggestion strip.

Real messages (2,000 held-out per language, labels = the emoji the person really typed)

	English	Hindi	Bengali
top-1	25.6%	26.8%	24.4%
hit@6	52.4%	55.4%	61.4%

For scale: picking one of 1,923 emojis at random is 0.05%. People also disagree with themselves — two people writing the same sentence rarely pick the same emoji — so ~25% exact-match is a strong result, not a weak one.

Chat, slang and everyday code-mixed phrases

Test set	Score
Chat cases (40 hand-written WhatsApp-style messages), right emoji in top 3	31/40
Slang & everyday Banglish/Hinglish probes (72), hit@6	41/72
— same, capitalised the way phone keyboards send it	36/72
Lookalike traps (8) that must not get toilet/swear emojis (Bhagavad Gita, bhagao, chhod do)	6/8
Emoji concept cases (24), hit@6	23/24
Emojinize multilingual test set, hit@6	99.3%
GoEmotions (emotion → emoji), hit@6	75.0%

Tone and style

Test	Score
Tone, right tone in top 3 (25 cases; untrained baseline 18/25)	23/25
Communication style (13 cases; untrained baseline 6/13)	11/13
Hinglish sarcasm detection (held-out test set)	81.7%

Known weaknesses

Stated plainly, because they are reproducible:

• Crude Bengali slang is still weak. Haga pacche (need the toilet) returns 😂 rather than 💩🚽. There are only ~112 such messages in 2.9 M real messages, and in most of them the writer typed 😂 — the model learned the joke, not the topic. Words with more data (ghum, 700 messages) work.
• Tone can be mis-read on very short romanized Bengali. Brishti porche baire (it's raining outside) scores 0.93 on "angry". Apps should gate offensive emojis on the tone outputs, and even that does not fully help here.
• Hindi slang trails Bengali (8/19 vs 25/44 on the probes).
• Trained on public social media, which contains political, offensive and adult language. The emoji head can suggest 🖕🤬 for messages it misreads; see the gating recipe in Responsible use.

Quick start

Python (transformers)

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("subhadeep3902/emojet")
model = AutoModelForSequenceClassification.from_pretrained("subhadeep3902/emojet").eval()

def emojet(draft, reply=None, k=6):
    text = (f"replying to: {reply.strip()}\n" if reply else "") + f"my message: {draft.strip()}"
    with torch.no_grad():
        logits = model(**tok(text, return_tensors="pt", truncation=True, max_length=256)).logits[0]
    emojis = [model.config.id2label[i] for i in logits[:1923].topk(k).indices.tolist()]
    tones = [model.config.id2label[1923 + i][5:] for i in torch.sigmoid(logits[1923:1988]).topk(3).indices.tolist()]
    style = model.config.id2label[1988 + int(logits[1988:1996].argmax())][6:]
    return emojis, tones, style

print(emojet("my program finally works"))
# -> emojis like 🥳 🥲 😭 🥹: the feeling of the sentence, not the word "program"

print(emojet("congrats bro", reply="I finally got the job!!"))

The input format matters: the model was trained on exactly replying to: …\nmy message: … (the first line only when there is a reply).

llama.cpp / GGUF (phones, laptops, no Python)

# 274 MB, 4-bit — this is the file the Android app ships
huggingface-cli download subhadeep3902/emojet Emojet-v1-Q4_K_M.gguf --local-dir .

llama-server -m Emojet-v1-Q4_K_M.gguf --embedding --pooling rank -c 512 --port 8080

curl -s localhost:8080/embedding -H 'Content-Type: application/json' \
  -d '{"content": "my message: ghum pacche khub", "embd_normalize": -1}'
# -> 2,002 scores in the order of labels.json

--pooling rank returns the classifier scores (not an embedding); embd_normalize: -1 keeps them raw. Returning all classifier scores needs a one-line llama.cpp patch, included in the training repo under engine/llama.cpp-patches/.

Reading the output

labels.json gives the 2,002 outputs in order, with the group boundaries:

0 … 1922   emojis    softmax over the group
1923 … 1987  tones   sigmoid, independent (several can be true)
1988 … 1995  styles  softmax over the group
1996 … 2001  axes    sigmoid, 0 = first word of the pair, 1 = second

Files

File	Size	Use
model.safetensors + config.json + tokenizer	796 MB	transformers, fine-tuning, research
Emojet-v1-F16.gguf	798 MB	llama.cpp, full precision
Emojet-v1-Q4_K_M.gguf	274 MB	phones and laptops — what the app downloads
labels.json	48 KB	output order and groups
emojet.json	32 KB	metadata and the metrics above

4-bit costs ~0.3 points of hit@6 against F16 and is the recommended file for anything shipping.

Architecture

ModernBertForSequenceClassification, 28 layers, hidden size 1,024, 398 M parameters, mean pooling over the sequence, a single 1,024 → 2,002 linear head. Max input 256 tokens (longer messages are clipped head + tail by the app). The encoder is Laya's, revision 7b928d8.

Training

Three stages on a single Kaggle T4, ~40 GPU-hours in total. Everything is reproducible from the training repo (docs/TRAINING.md): every data job and training run is a notebook that can be re-pushed and re-run.

1. Concept grounding — emoji names and keywords (CLDR), emojinize spans, things mentioned inside casual chat.
2. Real messages — 2.9 M public messages in English, Hindi and Bengali where the writer used an emoji. The emoji is stripped from the text and becomes the label (distant supervision). Balanced softmax (τ=0.5) stops 😂❤️👍 from swallowing the long tail.
3. Tone, style and slang — human-labelled tone/sarcasm/politeness/toxicity datasets mapped onto the 65-tone taxonomy, ~88 K messages labelled for the full taxonomy by an open LLM (Qwen3.6-35B-A3B, Apache-2.0), plus a curated 146-entry lexicon of Banglish/Hinglish chat words with their meanings and emojis.

The released weights are a weight average (WiSE-FT style) of the real-message model and the slang model, at the mixing ratio that kept every slang gain without a statistically significant loss on real messages (paired McNemar test per language).

Data sources

Emojinize-multilingual (CC-BY-4.0) · GoEmotions (Apache-2.0) · Stanford Politeness · Pavlick formality · Jigsaw toxicity · SemEval-2018 irony · Hinglish sarcasm (BM2002) · HASOC · TRAC-2 aggression · DailyDialog · SWDA · EmpatheticDialogues · ESConv · CaSiNo · MentalManip · TalkDown · Dreaddit · BanglishRev · EmoNoBa · Bengali/Banglish hate and toxicity sets · public Bluesky and Twitter corpora. Full list with licences in training/real_data.py and training/tone_data.py.

Responsible use

• The emoji head can suggest 🖕🤬🤮 for messages it misreads. Gate them on the tone outputs — the reference app suppresses them unless rude/aggressive/angry/derisive ≥ 0.5 — and remember this is a mitigation, not a fix.
• Tone and style outputs describe how a message reads, not the person. Do not use them to score, rank, moderate or profile people.
• Training data is public social media and carries its biases (political, religious, gendered). Expect worse behaviour on dialects and topics outside the three languages above.
• Not for commercial use (see licence).

Licence

Weights: CC BY-NC-SA 4.0 — free to use, share and build on, non-commercially, with attribution and share-alike. Part of the training data is non-commercial-only, which is what sets this.

The base model Laya is Apache-2.0. The training code is Apache-2.0 and the app code is MIT, both on GitHub.

Citation

@software{emojet2026,
  title  = {Emojet: whole-sentence emoji, tone and style prediction on device},
  author = {Subhadeep Roy},
  year   = {2026},
  url    = {https://huggingface.co/subhadeep3902/emojet}
}

Acknowledgements

Laya by Convai Innovations for the base encoder, llama.cpp for on-device inference, and every dataset author listed above.