khalidsaifullaah commited on
Commit
dacff11
1 Parent(s): 29a4019

Few changes

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -1,10 +1,15 @@
 
 
 
 
 
1
  # Bengali GPT-2
2
 
3
  Bengali GPT-2 demo. Part of the [Huggingface JAX/Flax event](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/). Also features a [finetuned](https://huggingface.co/khalidsaifullaah/bengali-lyricist-gpt2?) model on bengali song lyrics.
4
 
5
  # Model Description
6
 
7
- OpenAI GPT-2 model was proposed in [Language Models are Unsupervised Multitask Learners](https://paperswithcode.com/paper/language-models-are-unsupervised-multitask) paper .Original GPT2 model was a causal (unidirectional) transformer pretrained using language modeling on a very large corpus of ~40 GB of text data. This model has same configuration but has been pretrained on bengali corpus of mC4(multilingual C4) dataset. This flax model has been trained using code provided by Huggingface team from [here](https://github.com/huggingface/transformers/tree/master/examples/flax/language-modeling).
8
 
9
  # Training Details
10
 
 
1
+ ---
2
+ language: bn
3
+ license: mit
4
+ ---
5
+
6
  # Bengali GPT-2
7
 
8
  Bengali GPT-2 demo. Part of the [Huggingface JAX/Flax event](https://discuss.huggingface.co/t/open-to-the-community-community-week-using-jax-flax-for-nlp-cv/). Also features a [finetuned](https://huggingface.co/khalidsaifullaah/bengali-lyricist-gpt2?) model on bengali song lyrics.
9
 
10
  # Model Description
11
 
12
+ OpenAI GPT-2 model was proposed in [Language Models are Unsupervised Multitask Learners](https://paperswithcode.com/paper/language-models-are-unsupervised-multitask) paper .Original GPT2 model was a causal (unidirectional) transformer pretrained using language modeling on a very large corpus of ~40 GB of text data. This model has same configuration but has been pretrained on bengali corpus of mC4(multilingual C4) dataset. The code for training the model has all been open-sourced [here](https://huggingface.co/flax-community/gpt2-bengali/tree/main.
13
 
14
  # Training Details
15