Updated README

README.md

@@ -3,7 +3,7 @@
 language: ta
 license: MIT
 datasets:
-- OSCAR
+- oscar
 - IndicNLP
 widget:
 - text: 'ஒரு ஊரிலே ஒரு காக்கைக்கு'
@@ -11,51 +11,60 @@ widget:
 ---
 # GPT2-Tamil
 
-This repository is created as part of the Flax/Jax community week by Huggingface. The aim of this project is to train a language model using GPT-2 specifically for Tamil language.
+This repository is created as part of the Flax/Jax community week by Huggingface. The aim of this project is to pretrain a language model using GPT-2 specifically for Tamil language. 
 
 ## Setup:
 To setup the project, run the following command,
-``` pip install -r requirements.txt
+```python
+pip install -r requirements.txt
 ```
+
+## Model
+Pretrained model on Tamil language using a causal language modeling (CLM) objective.
  
 ## Dataset Used:
-The GTP-2 model is trained using OSCAR (Tamil) and IndicNLP (Tamil) dataset
+The GTP-2 model is trained on [oscar dataset - ta](https://huggingface.co/datasets/oscar) and [IndicNLP dataset - ta](https://indicnlp.ai4bharat.org/corpora/)
+
+## Intended uses & limitations
+You can use the raw model for next sentence prediction, but it's mostly intended to be fine-tuned on a downstream task. See the [model hub](https://huggingface.co/models?filter=gpt) to look for fine-tuned versions on a task that interests you.
 
-## Train model:
+## How to pretrain the model:
 To perform training, do the following steps,
 
 - Export the model directory (where you want to store the model artifacts like config, tokenizer, etc.)
-```
- export MODEL_DIR=<model_dir>
+```python
+>>> export MODEL_DIR=<model_dir>
 ```
 - Create the config.json by running the following command,
-```
- python src/create_config.py
+```python
+>>> python src/create_config.py
 ```
 - Create the tokenizer by running the following command,
-```
- python src/train_tokenizer.py
+```python
+>>> python src/train_tokenizer.py
 ```
 - Once the config and tokenizer is created, run the following script to start training the flax model
-```
- python scripts/train_gpt2-oscar-tamil.sh
+```python
+>>> python scripts/train_gpt2-oscar-tamil.sh
 ```
 
-## Inference:
+## How to use:
 To perform language generation using the model, pipeline can be used directly.
 
 - First convert the flax model to pytorch using the following command,
-```
- python src/convert_flax_to_pytorch.py
+```python
+python src/convert_flax_to_pytorch.py
 ```
 - Use the following snippet to perform language generation,
-```
+```python
  from transformers import AutoTokenizer, AutoModelWithLMHead, pipeline
  model_name = 'abinayam/gpt-2-tamil'
  model = AutoModelWithLMHead.from_pretrained(model_name)
  tokenizer = AutoTokenizer.from_pretrained(model_name)
+ set_seed(42)
  input_text = "ஒரு ஊரிலே ஒரு காக்கைக்கு"
  max_len = 300
+ no_seq = 5
  generator = pipeline('text-generation', model=model, tokenizer=tokenizer)
- sequence = generator(input_text, max_length=max_len)
+ sequence = generator(input_text, max_length=max_len, num_return_sequences=no_seq)
 ```