File size: 771 Bytes
80cc0d6 faa204b 80cc0d6 bd6529f eca4fbf 80cc0d6 67c3fa3 eca4fbf c342eec c892311 c342eec 80cc0d6 faa204b 80cc0d6 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 |
---
language: si
tags:
- SinBERTo
- Sinhala
- roberta
---
### Overview
SinBerto is a small language model trained on a small news corpus. SinBerto is trained on Sinhala Language which is a low resource language compared to other languages.
### Model Specifications.
model : [Roberta](https://arxiv.org/abs/1907.11692)
vocab_size=52_000,
max_position_embeddings=514,
num_attention_heads=12,
num_hidden_layers=6,
type_vocab_size=1
### How to use from the Transformers Library
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("Kalindu/SinBerto")
model = AutoModelForMaskedLM.from_pretrained("Kalindu/SinBerto")
### OR Clone the model repo
git lfs install
git clone https://huggingface.co/Kalindu/SinBerto |