Package: deeppavlov
DeepPavlov is an open-source conversational AI library built onPyTorch.DeepPavlov is designed fordevelopment of production ready chat-bots and complex conversational systems,research in the area of NLP and, particularly, of dialog systems.Quick LinksDemodemo.deeppavlov.aiDocumentationdocs.deeppavlov.aiModel Listdocs:features/Contribution Guidedocs:contribution_guide/Issuesgithub/issues/Forumforum.deeppavlov.aiBlogsmedium.com/deeppavlovExtended colab tutorialsDocker Hubhub.docker.com/u/deeppavlov/Docker Images Documentationdocs:docker-images/Please leave usyour feedbackon how we can improve the DeepPavlov framework.ModelsNamed Entity Recognition|Intent/Sentence Classification|Question Answering over Text (SQuAD)|Knowledge Base Question AnsweringSentence Similarity/Ranking|TF-IDF RankingSyntactic Parsing|Morphological TaggingAutomatic Spelling Correction|Entity ExtractionOpen Domain Questions Answering|Russian SuperGLUERelation ExtractionEmbeddingsBERT embeddings for the Russian, Polish, Bulgarian, Czech, and informal EnglishELMo embeddings for the Russian languageFastText embeddings for the Russian languageAuto MLTuning ModelsIntegrationsREST API|Socket APIAmazon AWSInstallationDeepPavlov supportsLinux,Windows 10+(through WSL/WSL2),MacOS(Big Sur+) platforms,Python 3.6,3.7,3.8,3.9and3.10.
Depending on the model used, you may need from 4 to 16 GB RAM.Create and activate a virtual environment:Linuxpython -m venv env
source ./env/bin/activateInstall the package inside the environment:pip install deeppavlovQuickStartThere is a bunch of great pre-trained NLP models in DeepPavlov. Each model is
determined by its config file.List of models is available onthe doc pagein
thedeeppavlov.configs(Python):fromdeeppavlovimportconfigsWhen you're decided on the model (+ config file), there are two ways to train,
evaluate and infer it:viaCommand line interface (CLI)andviaPython.GPU requirementsBy default, DeepPavlov installs models requirements from PyPI. PyTorch from PyPI could not support your device CUDA
capability. To run supported DeepPavlov models on GPU you should haveCUDAcompatible with used GPU andPyTorch versionrequired by DeepPavlov models.
Seedocsfor details.
GPU with Pascal or newer architecture and 4+ GB VRAM is recommended.Command line interface (CLI)To get predictions from a model interactively through CLI, runpython-mdeeppavlovinteract<config_path>[-d][-i]-ddownloads required data - pretrained model files and embeddings (optional).-iinstalls model requirements (optional).You can train it in the same simple way:python-mdeeppavlovtrain<config_path>[-d][-i]Dataset will be downloaded regardless of whether there was-dflag or not.To train on your own data you need to modify dataset reader path in thetrain config doc.
The data format is specified in the corresponding model doc page.There are even more actions you can perform with configs:python-mdeeppavlov<action><config_path>[-d][-i]<action>can beinstallto install model requirements (same as-i),downloadto download model's data (same as-d),trainto train the model on the data specified in the config file,evaluateto calculate metrics on the same dataset,interactto interact via CLI,riseapito run a REST API server (seedoc),predictto get prediction for samples fromstdinor from<file_path>if-f <file_path>is specified.<config_path>specifies path (or name) of model's config file-ddownloads required data-iinstalls model requirementsPythonTo get predictions from a model interactively through Python, runfromdeeppavlovimportbuild_modelmodel=build_model(<config_path>,install=True,download=True)# get predictions for 'input_text1', 'input_text2'model(['input_text1','input_text2'])whereinstall=Trueinstalls model requirements (optional),download=Truedownloads required data from web - pretrained model files and embeddings (optional),<config_path>is model name (e.g.'ner_ontonotes_bert_mult'), path to the chosen model's config file (e.g."deeppavlov/configs/ner/ner_ontonotes_bert_mult.json"),  ordeeppavlov.configsattribute (e.g.deeppavlov.configs.ner.ner_ontonotes_bert_multwithout quotation marks).You can train it in the same simple way:fromdeeppavlovimporttrain_modelmodel=train_model(<config_path>,install=True,download=True)To train on your own data you need to modify dataset reader path in thetrain config doc.
The data format is specified in the corresponding model doc page.You can also calculate metrics on the dataset specified in your config file:fromdeeppavlovimportevaluate_modelmodel=evaluate_model(<config_path>,install=True,download=True)DeepPavlov alsoallowsto build a model from components for
inference using Python.LicenseDeepPavlov is Apache 2.0 - licensed.