Package: shallowpavlov
DeepPavlov is an open-source conversational AI library built onPyTorch.DeepPavlov is designed fordevelopment of production ready chat-bots and complex conversational systems,research in the area of NLP and, particularly, of dialog systems.Quick LinksDemodemo.deeppavlov.aiDocumentationdocs.deeppavlov.aiModel Listdocs:features/Contribution Guidedocs:contribution_guide/Issuesgithub/issues/Forumforum.deeppavlov.aiBlogsmedium.com/deeppavlovTutorialsexamples/andextended colab tutorialsDocker Hubhub.docker.com/u/deeppavlov/Docker Images Documentationdocs:docker-images/Please leave usyour feedbackon how we can improve the DeepPavlov framework.ModelsNamed Entity Recognition|Intent/Sentence Classification|Question Answering over Text (SQuAD)|Knowledge Base Question AnsweringSentence Similarity/Ranking|TF-IDF RankingAutomatic Spelling Correction|Entity LinkingRussian SuperGLUESkillsOpen Domain Questions AnsweringFrequently Asked Questions AnsweringEmbeddingsBERT embeddings for the Russian, Polish, Bulgarian, Czech, and informal EnglishELMo embeddings for the Russian languageFastText embeddings for the Russian languageAuto MLTuning ModelsIntegrationsREST API|Socket APIAmazon AWSInstallationWe supportLinuxplatform,Python 3.6,3.7,3.8and3.9Python 3.5is not supported!Create and activate a virtual environment:Linuxpython -m venv env
source ./env/bin/activateInstall the package inside the environment:pip install deeppavlovQuickStartThere is a bunch of great pre-trained NLP models in DeepPavlov. Each model is
determined by its config file.List of models is available onthe doc pagein
thedeeppavlov.configs(Python):fromdeeppavlovimportconfigsWhen you're decided on the model (+ config file), there are two ways to train,
evaluate and infer it:viaCommand line interface (CLI)andviaPython.GPU requirementsTo run supported DeepPavlov models on GPU you should haveCUDAcompatible
with used GPU andlibrary PyTorch version.Command line interface (CLI)To get predictions from a model interactively through CLI, runpython-mdeeppavlovinteract<config_path>[-d]-ddownloads required data -- pretrained model files and embeddings
(optional).You can train it in the same simple way:python-mdeeppavlovtrain<config_path>[-d]Dataset will be downloaded regardless of whether there was-dflag or not.To train on your own data you need to modify dataset reader path in thetrain config doc.
The data format is specified in the corresponding model doc page.There are even more actions you can perform with configs:python-mdeeppavlov<action><config_path>[-d]<action>can bedownloadto download model's data (same as-d),trainto train the model on the data specified in the config file,evaluateto calculate metrics on the same dataset,interactto interact via CLI,riseapito run a REST API server (seedoc),predictto get prediction for samples fromstdinor from<file_path>if-f <file_path>is specified.<config_path>specifies path (or name) of model's config file-ddownloads required dataPythonTo get predictions from a model interactively through Python, runfromdeeppavlovimportbuild_modelmodel=build_model(<config_path>,download=True)# get predictions for 'input_text1', 'input_text2'model(['input_text1','input_text2'])wheredownload=Truedownloads required data from web -- pretrained model
files and embeddings (optional),<config_path>is path to the chosen model's config file (e.g."deeppavlov/configs/ner/ner_ontonotes_bert_mult.json") ordeeppavlov.configsattribute (e.g.deeppavlov.configs.ner.ner_ontonotes_bert_multwithout quotation marks).You can train it in the same simple way:fromdeeppavlovimporttrain_modelmodel=train_model(<config_path>,download=True)download=Truedownloads pretrained model, therefore the pretrained
model will be, first, loaded and then train (optional).Dataset will be downloaded regardless of whether there was-dflag or
not.To train on your own data you need to modify dataset reader path in thetrain config doc.
The data format is specified in the corresponding model doc page.You can also calculate metrics on the dataset specified in your config file:fromdeeppavlovimportevaluate_modelmodel=evaluate_model(<config_path>,download=True)LicenseDeepPavlov is Apache 2.0 - licensed.