Package: bidaf-keras
bidaf-kerasImplementation ofBidirectional Attention Flow for Machine Comprehensionin Keras 2What is this project about?Machine Comprehension is a task in the field of NLP & NLU where the machine is provided with a passage and a question, and the machine tries to find an answer to the asked question from that given passage, by understanding the syntax and semantics of human language (here, English) and by establishing and understanding the relations betweeen the passage and the question.We have implemented a model suggested in the paper Bidirectional Attention Flow for Machine Comprehension by a team of allennlp, popularly known as BiDAF.What you can do with this projectTrain/Retrain this model with your own dataset.Use the pretrained model for extending your own model or to try this one out.Modify the code of this model to develop a new model architecture out of it.PrerequisitesPython 3.6CUDA and cuDNN support for Tensorflow GPU (not mandatory, but it's better to have it)UsageThis project is available for use as a complete module. You can use this project via command-line arguments or by importing functionalities from it.:Usage via command-line arguments:To see the usage info, runpython3 -m bidaf --help.Using this module via command-line arguments will provide you with limited functionalities. To get the real power, we recommend using it by importing its functionalities.The CLI mode will provide you with two basic functionalities: train and predict.Train:To get usage information for training models, runpython3 -m bidaf train --helporbidaf-keras train --helpFor example:python3 -m bidaf --model_name=bidaf_50.h5 --do_lowercase train --epochs=1 --steps_per_epoch=1 --validation_steps=1orbidaf-keras --model_name=bidaf_50.h5 --do_lowercase train --epochs=1 --steps_per_epoch=1 --validation_steps=1Predict:To get usage information for running predictions on a model, runpython3 -m bidaf predict --helporbidaf-keras predict --helpFor example:python3 -m bidaf --model_name=bidaf_50.h5 --do_lowercase predict --passage "This is tree." --question "What is this?" --return_char_loc --return_confidence_scoreorbidaf-keras --model_name=bidaf_50.h5 --do_lowercase predict --passage "This is tree." --question "What is this?" --return_char_loc --return_confidence_scoreNotethat, some parameters like "--do_lowercase" are dependent on some other parameters like "--model_name". For example, if the model is trained to perform predictions on lowercase inputs, it might not work well if the inputs are not in lowercase.If you still have any queries on the need/use of any CLI argument, open an issue and we will reach you as soon as possible.Usage by importing functionalities:You can use the module by importing it in many different ways. So, we will just mention two example code snippets from which you will gain most of the knowledge required.Train:from bidaf.models import BidirectionalAttentionFlow
from bidaf.scripts import load_data_generators
bidaf_model = BidirectionalAttentionFlow(400)
bidaf_model.load_bidaf("/path/to/model.h5") # when you want to resume training
train_generator, validation_generator = load_data_generators(24, 400)
keras_model = bidaf_model.train_model(train_generator, validation_generator=validation_generator)Predict:from bidaf.models import BidirectionalAttentionFlow
bidaf_model = BidirectionalAttentionFlow(400)
bidaf_model.load_bidaf("/path/to/model.h5")
bidaf_model.predict_ans("This is a tree", "What is this?")Notethat the above are the simplest code snippets. You can explore all functionalities available by looking at the parameters avaiable at different level. If you can't understand the parameters, just open an issue here. We may write the usage of all parameters in future if required.Also notethat, in the above snippets,bidaf_modelis just an object of classBidirectionalAttentionFlowand not the real Keras model. You can access the Keras model by usingkeras_model = bidaf_model.model.FeaturesSupports both SQUAD-v1.1 and SQUAD-v2.0.Can predict answers from any length of passage and question but your memory should support it's size.Has multi-GPU support.Supports various embedding dimensions.Support for flexible answer span length.Support for fixed length passage and question.Ability to run predictions on a list of passages and questions.Sample shuffling can be enabled.Ability to return confidence score as well as character locations in the passage.Variable number of highway layers and decoders.Pre-trained ModelsModel Name:bidaf_50.h5Model Configuration:lowercase: Truebatch size: 16max passage length: Nonemax question length: Noneembedding dimension: 400squad version: 1.1Model Name:bidaf_10.h5Model Configuration:lowercase: Truebatch size: 16max passage length: Nonemax question length: Noneembedding dimension: 400squad version: 1.1Project flowFirst of all, the project will download the data required for training the model. This consists of two categories: Magnitude and SQuAD.SQuAD files are dataset files on which the model can be trained. Based on it's version specified, it will download the required files and preprocess them. The SQuAD data will be stored indata/squad. For more information on the SQuAD dataset,visit this site.Magnitude files are a form of compressed and encoded files that we use for word embeddings at word-level and character-level. The project will download the required files based on the mebedding dimension specified by the user, in thedata/magnitudedirectory. For more information on Magnitude,visit this siteNow, users can train the model. The users have various options available while training to save models after every epoch and save the history of training as well. All these items will be saved in a directory calledsaved_items.To run predictions on any trained model, make sure that the model lies in the directorysaved_itemsand then pass the name of the model to loading script. After loading the model, call the predict function.To resume training or to use pretrained models provided above, just place the model insidesaved_itemsdirectory and pass it's name to the project.Improvements in future releasesDownload pre-trained models automaticallySupport to specify model path instead of model nameProvide two modes for preprocessing: Strict and Moderate.Measure accuracy of model with Moderate mode.GUI supportDatabase supportYour suggestions...?WarningsThere's a dependency flaw in our project. The package we used to generate word embeddings, pymagnitude, hasn't yet added suport for fixed length inputs -https://github.com/plasticityai/magnitude/issues/50. Hence, trying to use this model with fixed length input will throw an error. We have looked into their source code and found the problem. We have forked that repo, fixed the issue and then generated a pull request but it seems like the author is no more maintaining that project. So, we have to wait. Till then, you may use the patched version of that package to make this functionality work. You can find it here -https://github.com/ParikhKadam/magnitude/tree/patch-1If the author is no more supporting this project, we are planning to maintain our forked version in future. But we don't know how to publish such a forked version on PyPI. You can help us in case you know it. Or wait until we learn xD.For now, you can install the patched version of this requirement by running this command:python3 -m pip install -U git+https://github.com/ParikhKadam/magnitude.git@patch-1IssuesOpen:...Solved:https://github.com/tensorflow/tensorflow/issues/24519https://github.com/keras-team/keras/issues/11978https://github.com/keras-team/keras/issues/12263ContributionsThoughts, samples codes, modifications and any other type of contributions are appreciated. This is my first project in Deep Learning and also first being open source. I will need as much help as possible as I don't know the path I need to follow. Thank you..My teamDharanee PatelZeel PatelOur GuideProf. Tushar A. ChampaneriaSpecial Thanks to the researchers of BiDAFSir,Minjoon Seofor helping me out personallyThe team of allenai (http://www.allenai.org/)