Package: tlidb
The Transfer Learning in Dialogue Benchmarking ToolkitOverviewTLiDB is a tool used to benchmark methods of transfer learning in conversational AI.
TLiDB can easily handledomain adaptation, task transfer, multitasking, continual learning, and other transfer learning settings.
TLiDB maintains a unified json format for all datasets and tasks, easing the new code necessary for new datasets and tasks. We highly encourage community contributions to the project.The main features of TLiDB are:Dataset class to easily load a dataset for use across modelsUnified metrics to standardize evaluation across datasetsExtensible Model and Algorithm classes to support fast prototypingInstallationRequirementspython>=3.6torch>=1.10nltk>=3.6.5scikit-learn>=1.0transformers>=4.11.3sentencepiece>=0.1.96bert-score==0.3.11To use TLiDB, you can simply install viapip:pipinstalltlidbOR, you can install TLiDB from source. This is recommended if you want to edit or contribute:gitclonegit@github.com:alon-albalak/TLiDB.gitcdTLiDB
pipinstall-e.How to use TLiDBTLiDB can be used from the command line or as a python command. If you have installed the package from source, we highly recommend running commands from inside the tlidb/examples/ directory.Quick StartFor a very simple set up, you can use the following commands.From command line:tlidb--source_datasetsFriends--source_tasksemory_emotion_recognition--target_datasetsFriends--target_tasksreading_comprehension--do_train--do_finetune--do_eval--model_config=bertAs python command:python3run_experiment.py--source_datasetsFriends--source_tasksemory_emotion_recognition--target_datasetsFriends--target_tasksreading_comprehension--do_train--do_finetune--do_eval--model_config=bertDetailed UsageTLiDB has 2 main folders of interest:tlidb/examplestlidb/TLiDBtlidb/examples/is recommended for use if you would like to utilize our training scripts. It contains sample code for models, learning algorithms, and sample training scripts.
For detailed examples, see theExamples README.tlidb/TLiDB/holds the code related to data (datasets, dataloaders, metrics, etc.). If you are interested in utilizing our datasets and metrics but would like to train models using your own training scripts, take a look at the example usage inTLiDB README.Folder descriptions:tlidb/TLiDB is the folder holding the code for data handlingtlidb/TLiDB/data_loaders contains code for data_loaderstlidb/TLiDB/data is the destination folder for downloaded datasets (if installed from source, otherwise data is in .cache/tlidb/data)tlidb/TLiDB/datasets contains code for dataset loading and preprocessingtlidb/TLiDB/metrics contains code for loss and evaluation metricstlidb/TLiDB/utils contains utility filestlidb/examples contains sample code for training and evaluating modelstlidb/examples/algorithms contains code which trains and evaluates a modeltlidb/examples/models contains code to define a modeltlidb/examples/configs contains code for model configurations/dataset_preprocessing is for reproducability purposes. It contains scripts used to preprocess the TLiDB datasets from their original form into the standardized TLiDB formatComments, Questions, and FeedbackIf you find issues, pleaseopen an issue here.If you have dataset or model requests, pleaseadd a new discussion here.We encourage outside contributions to the project!CitationIf you use TLiDB in your work, please cite the repository:@software{Albalak_The_Transfer_Learning_2022,
author = {Albalak, Alon},
doi = {10.5281/zenodo.6374360},
month = {3},
title = {{The Transfer Learning in Dialogue Benchmarking Toolkit}},
url = {https://github.com/alon-albalak/TLiDB},
version = {1.0.0},
year = {2022}
}AcknowledgementsThe design of TLiDB was based thewildsproject, and theOpen Graph Benchmark.