Package: parlai
ParlAI(pronounced “par-lay”) is a python framework for
sharing, training and testing dialogue models, from open-domain chitchat, to
task-oriented dialogue, to visual question answering.Its goal is to provide researchers:100+ popular datasets available all in one place, with the same API, among themPersonaChat,DailyDialog,Wizard of Wikipedia,Empathetic Dialogues,SQuAD,MS MARCO,QuAC,HotpotQA,QACNN & QADailyMail,CBT,BookTest,bAbI Dialogue tasks,Ubuntu Dialogue,OpenSubtitles,Image Chat,VQA,VisDialandCLEVR. See the complete listhere.a wide set ofreference models-- from retrieval baselines to Transformers.a largezoo ofpretrained modelsready to use off-the-shelfseamlessintegration ofAmazon Mechanical Turkfor data collection and human evaluationintegration withFacebook Messengerto connect agents with humans in a chat interfacea large range ofhelpers to create your own agentsand train on several tasks withmultitaskingmultimodality, some tasks use text and imagesParlAI is described in the following paper:“ParlAI: A Dialog Research Software Platform", arXiv:1705.06476or see thesemore up-to-date slides.Follow us onTwitterand check out ourRelease
notesto see the latest
information about new features & updates, and the websitehttp://parl.aifor further docs. For an archived list of updates,
check outNEWS.md.Interactive TutorialFor those who want to start with ParlAI now, you can try ourColab Tutorial.Installing ParlAIOperating SystemParlAI should work as inteded under Linux or macOS. We do not support Windows at this time, but many usersreport success on Windows using Python 3.8and issues with Python 3.9. We are happy to accept patches that improve Windows support.Python InterpreterParlAI currently requires Python3.8+.RequirementsParlAI supportsPytorch1.6 or higher.
All requirements of the core modules are listed inrequirements.txt. However, some models included (inparlai/agents) have additional requirements.Virtual EnvironmentWestronglyrecommend you install ParlAI in a virtual environment usingvenvorconda.End User InstallationIf you want to use ParlAI without modifications, you can install it with:cd/path/to/your/parlai-app
python3.8-mvenvvenv
venv/bin/pipinstall--upgradepipsetuptoolswheel
venv/bin/pipinstallparlaiDeveloper InstallationMany users will want to modify some parts of ParlAI. To set up a development
environment, run the following commands to clone the repository and install
ParlAI:gitclonehttps://github.com/facebookresearch/ParlAI.git~/ParlAIcd~/ParlAI
python3.8-mvenvvenv
venv/bin/pipinstall--upgradepipsetuptoolswheel
venv/bin/pythonsetup.pydevelopNoteSometimes the install from source maynot work due to dependencies (specially in PyTorch related packaged).
In that case try building a fresh conda environment and running the similar to the following:conda install pytorch==2.0.0 torchvision torchaudio torchtext pytorch-cuda=11.8 -c pytorch -c nvidia.
Check torch setup documentation for your CUDA and OS versions.All needed data will be downloaded to~/ParlAI/data. If you need to clear out
the space used by these files, you can safely delete these directories and any
files needed will be downloaded again.DocumentationQuick StartBasics: world, agents, teachers, action and observationsCreating a new dataset/taskList of available tasks/datasetsCreating a seq2seq agentList of available agentsModel zoo (list pretrained models)Running crowdsourcing tasksPlug into Facebook MessengerExamplesA large set of scripts can be found inparlai/scripts. Here are a few of them.
Note: If any of these examples fail, check theinstallation sectionto see if you have missed something.Display 10 random examples from the SQuAD taskparlaidisplay_data-tsquadEvaluate an IR baseline model on the validation set of the Personachat task:parlaieval_model-mir_baseline-tpersonachat-dtvalidTrain a single layer transformer on PersonaChat (requires pytorch and torchtext).
Detail: embedding size 300, 4 attention heads,  2 epochs using batchsize 64, word vectors are initialized with fasttext and the other elements of the batch are used as negative during training.parlaitrain_model-tpersonachat-mtransformer/ranker-mf/tmp/model_tr6--n-layers1--embedding-size300--ffn-size600--n-heads4--num-epochs2-veps0.25-bs64-lr0.001--dropout0.1--embedding-typefasttext_cc--candidatesbatchCode OrganizationThe code is set up into several main directories:core: contains the primary code for the frameworkagents: contains agents which can interact with the different tasks (e.g. machine learning models)scripts: contains a number of useful scripts, like training, evaluating, interactive chatting, ...tasks: contains code for the different tasks available from within ParlAImturk: contains code for setting up Mechanical Turk, as well as sample MTurk tasksmessenger: contains code for interfacing with Facebook Messengerutils: contains a wide number of frequently used utility methodscrowdsourcing: contains code for running crowdsourcing tasks, such as on Amazon Mechanical Turkchat_service: contains code for interfacing with services such as Facebook Messengerzoo: contains code to directly download and use pretrained models from our model zooSupportIf you have any questions, bug reports or feature requests, please don't hesitate to post on ourGithub Issues page.
You may also be interested in checking out ourFAQand
ourTips n Tricks.Please remember to follow ourCode of Conduct.ContributingWe welcome PRs from the community!You can find information about contributing to ParlAI in ourContributingdocument.The TeamParlAI is currently maintained by Moya Chen, Emily Dinan, Dexter Ju, Mojtaba
Komeili, Spencer Poff, Pratik Ringshia, Stephen Roller, Kurt Shuster,
Eric Michael Smith, Megan Ung, Jack Urbanek, Jason Weston, Mary Williamson,
and Jing Xu. Kurt Shuster is the current Tech Lead.Former major contributors and maintainers include Alexander H. Miller, Margaret
Li, Will Feng, Adam Fisch, Jiasen Lu, Antoine Bordes, Devi Parikh, Dhruv Batra,
Filipe de Avila Belbute Peres, Chao Pan, and Vedant Puri.CitationPlease cite thearXiv paperif you use ParlAI in your work:@article{miller2017parlai,
  title={ParlAI: A Dialog Research Software Platform},
  author={{Miller}, A.~H. and {Feng}, W. and {Fisch}, A. and {Lu}, J. and {Batra}, D. and {Bordes}, A. and {Parikh}, D. and {Weston}, J.},
  journal={arXiv preprint arXiv:{1705.06476}},
  year={2017}
}LicenseParlAI is MIT licensed. See theLICENSEfile for details.