Package: turkish-lm-tuner
🦖 Turkish LM TunerOverviewTurkish LM Tuner is a library for fine-tuning Turkish language models on various NLP tasks. It is built on top ofHugging Face Transformerslibrary. It supports finetuning with conditional generation and sequence classification tasks. The library is designed to be modular and extensible. It is easy to add new tasks and models. The library also provides data loaders for various Turkish NLP datasets.InstallationYou can installturkish-lm-tunervia PyPI:pipinstallturkish-lm-tunerAlternatively, you can use the following command to install the library:pipinstallgit+https://github.com/boun-tabi-LMG/turkish-lm-tuner.gitModel SupportAny Encoder or ConditionalGeneration model that is compatible with Hugging Face Transformers library can be used with Turkish LM Tuner. The following models are tested and supported.TURNAmT5mBARTBERTurkTask and Dataset SupportTaskDatasetsText ClassificationProduct Reviews,TTC4900,Tweet SentimentNatural Language InferenceNLI_TR,SNLI_TR,MultiNLI_TRSemantic Textual SimilaritySTSb_TRNamed Entity RecognitionWikiANN,Milliyet NERPart-of-Speech TaggingBOUN,IMSTText SummarizationTR News,MLSUM,Combined TR News and MLSUMTitle GenerationTR News,MLSUM,Combined TR News and MLSUMParaphrase GenerationOpenSubtitles,Tatoeba,TED TalksUsageThe tutorials in thedocumentationcan help you get started withturkish-lm-tuner.ExamplesFine-tune and evaluate a conditional generation modelfromturkish_lm_tunerimportDatasetProcessor,TrainerForConditionalGenerationdataset_name="tr_news"task="summarization"task_format="conditional_generation"model_name="boun-tabi-LMG/TURNA"max_input_length=764max_target_length=128dataset_processor=DatasetProcessor(dataset_name=dataset_name,task=task,task_format=task_format,task_mode='',tokenizer_name=model_name,max_input_length=max_input_length,max_target_length=max_target_length)train_dataset=dataset_processor.load_and_preprocess_data(split='train')eval_dataset=dataset_processor.load_and_preprocess_data(split='validation')test_dataset=dataset_processor.load_and_preprocess_data(split="test")training_params={'num_train_epochs':10'per_device_train_batch_size':4,'per_device_eval_batch_size':4,'output_dir':'./','evaluation_strategy':'epoch','save_strategy':'epoch','predict_with_generate':True}optimizer_params={'optimizer_type':'adafactor','scheduler':False,}model_trainer=TrainerForConditionalGeneration(model_name=model_name,task=task,optimizer_params=optimizer_params,training_params=training_params,model_save_path="turna_summarization_tr_news",max_input_length=max_input_length,max_target_length=max_target_length,postprocess_fn=dataset_processor.dataset.postprocess_data)trainer,model=model_trainer.train_and_evaluate(train_dataset,eval_dataset,test_dataset)model.save_pretrained(model_save_path)dataset_processor.tokenizer.save_pretrained(model_save_path)Evaluate a conditional generation model with custom generation configfromturkish_lm_tunerimportDatasetProcessor,EvaluatorForConditionalGenerationdataset_name="tr_news"task="summarization"task_format="conditional_generation"model_name="boun-tabi-LMG/TURNA"task_mode=''max_input_length=764max_target_length=128dataset_processor=DatasetProcessor(dataset_name,task,task_format,task_mode,model_name,max_input_length,max_target_length)test_dataset=dataset_processor.load_and_preprocess_data(split="test")test_params={'per_device_eval_batch_size':4}model_path="turna_tr_news_summarization"generation_params={'num_beams':4,'length_penalty':2.0,'no_repeat_ngram_size':3,'early_stopping':True,'max_length':128,'min_length':30,}evaluator=EvaluatorForConditionalGeneration(model_path,model_name,task,max_input_length,max_target_length,test_params,generation_params,dataset_processor.dataset.postprocess_data)results=evaluator.evaluate_model(test_dataset)print(results)ReferenceIf you use this repository, please cite the following relatedpaper:@misc{uludoğan2024turna,title={TURNA: A Turkish Encoder-Decoder Language Model for Enhanced Understanding and Generation},author={Gökçe Uludoğan and Zeynep Yirmibeşoğlu Balal and Furkan Akkurt and Melikşah Türker and Onur Güngör and Susan Üsküdarlı},year={2024},eprint={2401.14373},archivePrefix={arXiv},primaryClass={cs.CL}}LicenseNote that all datasets belong to their respective owners. If you use the datasets provided by this library, please cite the original source.This code base is licensed under the MIT license. SeeLICENSEfor details.