You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

AraNeoBERT v6

Arabic NeoBERT encoder (~248M parameters in the encoder), trained with masked language modeling on about 77B tokens of Arabic text (web text, books, Wikipedia, news, and dialectal data including tweets). Context length up to 8,192 tokens.

Research checkpoint — downstream evaluation is still in progress.

Usage

from transformers import AutoTokenizer, AutoModel

tok = AutoTokenizer.from_pretrained("miweru/AraNeoBERT-v6", trust_remote_code=True)
model = AutoModel.from_pretrained("miweru/AraNeoBERT-v6", trust_remote_code=True)

trust_remote_code=True is required for the NeoBERT architecture (local/global attention, RoPE, QK-norm). Requires xformers (SwiGLU), as the original NeoBERT.

Downloads last month: 3

Safetensors

Model size

0.3B params

Tensor type

F32