Back to all models

Inference API has been turned off for this model.

Monthly model downloads

jcblaise/electra-tagalog-base-cased-generator jcblaise/electra-tagalog-base-cased-generator
last 30 days



Contributed by

jcblaise Jan Christian Blaise Cruz
13 models

How to use this model directly from the πŸ€—/transformers library:

Copy to clipboard
from transformers import AutoTokenizer, AutoModel tokenizer = AutoTokenizer.from_pretrained("jcblaise/electra-tagalog-base-cased-generator") model = AutoModel.from_pretrained("jcblaise/electra-tagalog-base-cased-generator")

ELECTRA Tagalog Base Cased Generator

Tagalog ELECTRA model pretrained with a large corpus scraped from the internet. This model is part of a larger research project. We open-source the model to allow greater usage within the Filipino NLP community.

This is the generator model used to sample synthetic text and pretrain the discriminator. Only use this model for retraining and mask-filling. For the actual model for downstream tasks, please refer to the discriminator models.


The model can be loaded and used in both PyTorch and TensorFlow through the HuggingFace Transformers package.

from transformers import TFAutoModel, AutoModel, AutoTokenizer

# TensorFlow
model = TFAutoModel.from_pretrained('jcblaise/electra-tagalog-base-cased-generator', from_pt=True)
tokenizer = AutoTokenizer.from_pretrained('jcblaise/electra-tagalog-base-cased-generator', do_lower_case=False)

# PyTorch
model = AutoModel.from_pretrained('jcblaise/electra-tagalog-base-cased-generator')
tokenizer = AutoTokenizer.from_pretrained('jcblaise/electra-tagalog-base-cased-generator', do_lower_case=False)

Finetuning scripts and other utilities we use for our projects can be found in our centralized repository at


All model details and training setups can be found in our papers. If you use our model or find it useful in your projects, please cite our work:

  title={Investigating the True Performance of Transformers in Low-Resource Languages: A Case Study in Automatic Corpus Creation},
  author={Jan Christian Blaise Cruz and Jose Kristian Resabal and James Lin and Dan John Velasco and Charibeth Cheng},
  journal={arXiv preprint arXiv:2010.11574},

Data and Other Resources

Data used to train this model as well as other benchmark datasets in Filipino can be found in my website at


If you have questions, concerns, or if you just want to chat about NLP and low-resource languages in general, you may reach me through my work email at