metadata
license: apache-2.0
base_model_relation: finetune
base_model:
- sbintuitions/modernbert-ja-310m
tags:
- modernbert
BERT
2girls, two girls are sitting at the table, the left girl has pink hair, the right girl has black hair. The left girl is wearing a yellow hair ribbon
This modern architecture combines the LLM adapter with existing architectures.
Although the tokenization process is still done at the BERT level, the T5 ids are not entirely discarded. It can be called a text encoder with a dual tokenizer.
This implementation is also leaner than the fat, transformer-based adapters on the top of LLMs.
While ModernBERT's entire multilingual vocabulary can still be used, this is backward compatible with Anima's T5.
This repo also converges faster than her lil sister.
Source data
- anime themed, long natural sentences
- general tags
- character count tags
- colors
- negatives
- safety levels
- spatial relationships
No credits are provided; this is original work.