Edit model card

Intro

Trained on IndicNLGSuit IndicQuestionGeneration data for Bengali the model is finetuned from IndicBART

Finetuned Command

python run_summarization.py --model_name_or_path bnQG_models/checkpoint-32000 --do_eval --train_file train_bn.json 
--validation_file valid_bn.json --output_dir bnQG_models --overwrite_output_dir --per_device_train_batch_size=2 
--per_device_eval_batch_size=4 --predict_with_generate --text_column src --summary_column tgt --save_steps 4000 
--evaluation_strategy steps --gradient_accumulation_steps 4 --eval_steps 1000 --learning_rate 0.001 --num_beams 4 
--forced_bos_token "<2bn>" --num_train_epochs 10 --warmup_steps 10000

Sample Line from train data

{"src": "प्राणबादी [SEP] अर्थाॎ, तिनि छिलेन एकजन सर्बप्राणबादी। </s> <2bn>", "tgt": "<2bn> कोन दार्शनिक दृष्टिभङ्गि ओय़ाइटजेर छिल? </s>"}

Inference

script = "সুভাষ ১৮৯৭ খ্রিষ্টাব্দের ২৩ জানুয়ারি ব্রিটিশ ভারতের অন্তর্গত বাংলা প্রদেশের উড়িষ্যা বিভাগের (অধুনা, ভারতের ওড়িশা রাজ্য) কটকে জন্মগ্রহণ করেন।"
answer = "১৮৯৭ খ্রিষ্টাব্দের ২৩ জানুয়ারি"
inp = answer +" [SEP] "+script + " </s> <2bn>"
inp_tok = tokenizer(inp, add_special_tokens=False, return_tensors="pt", padding=True).input_ids
model.eval() # Set dropouts to zero

model_output=model.generate(inp_tok, use_cache=True, 
                            num_beams=4, 
                            max_length=20, 
                            min_length=1, 
                            early_stopping=True, 
                            pad_token_id=pad_id, 
                            bos_token_id=bos_id, 
                            eos_token_id=eos_id, 
                            decoder_start_token_id=tokenizer._convert_token_to_id_with_added_voc("<2bn>")
                        )
decoded_output=tokenizer.decode(model_output[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)

Citations

@inproceedings{dabre2021indicbart,
    title={IndicBART: A Pre-trained Model for Natural Language Generation of Indic Languages}, 
    author={Raj Dabre and Himani Shrotriya and Anoop Kunchukuttan and Ratish Puduppully and Mitesh M. Khapra and Pratyush Kumar},
    year={2022},
    booktitle={Findings of the Association for Computational Linguistics},
    }    


@misc{kumar2022indicnlg,
  title={IndicNLG Suite: Multilingual Datasets for Diverse NLG Tasks in Indic Languages}, 
  author={Aman Kumar and Himani Shrotriya and Prachi Sahu and Raj Dabre and Ratish Puduppully and Anoop Kunchukuttan and Amogh Mishra and Mitesh M. Khapra and Pratyush Kumar},
  year={2022},
  eprint={2203.05437},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}   
Downloads last month
9
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.