Package: putput
Aboutputputis a library that generates labeled data for conversational AI. It's simple to use, highly customizable, and can handle big data generation on a consumer grade laptop.putputtakes minutes to setup and seconds to generate millions of labeled data points.putput's labeled data could be used to:train a ML model when you do not have real data.augment training specific patterns in a ML model.test existing ML models for specific patterns.putputprovides an API to itsPipelinethat specifies how to generate labeled data. It ships with presets that configure thePipelinefor common NLU providers such asLUISandspaCy.putputexcels at generating custom datasets, even for problems that have yet to be solved commercially and for which no publicly available datasets exist. For instance, checkout thisjupyter notebookthat usesputputto generate a dataset formulti-intentrecognition and trains a LSTM withKerasto recognize multi-intent and entities.Here is an example prediction from the LSTM trained withputputdata:Note that the trained LSTM can deal with real life complexity such as handling multi-intent ("add" and "remove"groups) and disambiguating between the same word in different contexts (the quantity "ten" vs. "ten" in the item "ten chicken strips").Installationputputcurrently supports python >= 3.5. To install the production release, executepip install putput.Samplesputputships with several dockerized samples that show how to generate data.Clone the repo:git clone https://github.com/michaelperel/putput.gitMove into the project directory:cd putputEnsure docker is running:docker --versionBuild the runtime environment:docker build -t putput .The project ships with several usage samples which you can execute, for example:docker run putput smart_speakerordocker run putput restaurant.putputalso ships with annotated jupyter notebooks in thesamples/directory that useputputto solve real world NLU problems. Note: Github cannot correctly render certain graphics, so the notebooks should be viewed onnbviewer.DevelopmentThere are various checks that Travis (our CI server) executes to ensure code quality.
You can also run the checks locally:Install the development dependencies via:pip install -e .[dev]Run the linter:python setup.py pylintRun the type checker:python setup.py mypyRun the tests:python setup.py testAlternatively, you can run all the steps via Docker:docker build --target=build -t putput .Usageputputis a pipeline that works by reshaping thepattern definition, a user defined yaml file of patterns, into labeled data.ExampleHere is an example of apattern definitionthat generates labeled data for a smart speaker.base_tokens:-PERSONAL_PRONOUNS:[he,she]-SPEAKER:[cortana,siri,alexa,google]token_patterns:-static:-WAKE:-[[hi,hey],SPEAKER]-PLAY:-[PERSONAL_PRONOUNS,[wants,would like],[to],[play]]-[[play]]-dynamic:-ARTIST-SONGgroups:-PLAY_SONG:[PLAY,SONG]-PLAY_ARTIST:[PLAY,ARTIST]utterance_patterns:-[WAKE,PLAY_SONG]-[WAKE,PLAY_ARTIST]-[WAKE,1-2,PLAY_SONG]Focusing on the firstutterance_pattern,[WAKE, PLAY_SONG],putputwould generate hundreds ofutterances,tokens, andgroupsof the form:utterance- hi cortana he wants to play here comes the sunPattern definition referenceIn thepattern definition, the two most important sections aretoken_patternsandutterance_patterns. Atoken_patterndescribes a sequence of components whose product constitutes atoken. For instance, the soletoken_patternfor theWAKEtokenis[[hi, hey], [cortana, siri, alexa, google]](thebase_token,SPEAKER, is replaced with its value[cortana, siri, alexa, google]at runtime). The product of thistoken_pattern:hi cortanahi sirihi alexahi googlehey cortanahey sirihey alexahey googlerepresents theWAKEtoken.Within thetoken_patternssection, there arestaticanddynamicsections.staticmeans all of thetoken_patternscan be specified before the application runs.dynamicmeans thetoken_patternswill be specified at runtime. In our example,WAKEis defined underneathstaticbecause all ways to awake the smart speaker are known before runtime.ARTISTandSONG, however, are defined underneathdynamicbecause the artists and songs in your music catalog may change frequently. The values for thesetokenscan be passed in as arguments toPipelineat runtime.Within eachtoken_pattern,base_tokensmay be used to keep yourself from repeating the same components. For instance, in our example, we could potentially usePERSONAL_PRONOUNSin many different places, so we'd like to only have to define it once.Anutterance_patterndescribes the product oftokensthat make up anutterance. For instance, the firstutterance_pattern,[WAKE, PLAY, SONG], is a product of all of the products oftoken_patternsforWAKE,PLAY, andSONG(thegroup,PLAY_SONG, is replaced with its value[PLAY, SONG]). Exampleutterancesgenerated from thisutterance_patternwould be:hi cortana play here comes the sunhi cortana he would like to play here comes the sunWithin eachutterance_pattern,groupsmay be used to keep yourself from repeating the sametokens. For instance, in our example, we could potentially usePLAY_SONGin many different places, so we'd like to only have to define it once. Unlikebase_tokens,putputkeeps track ofgroups. For instance, recall one potential output corresponding to theutterance_pattern,[WAKE, PLAY_SONG]:SincePLAY_SONGis the onlygroupin theutterance_pattern, theWAKEtokenis assigned thegroupNONEwhereas thePLAYandSONGtokensare assigned thegroupPLAY_SONG.Thinking in terms of commercial NLU providers,groupscould be used to match tointentsandtokenscould be used to matchentities.utterance_patternsandgroupssupport range syntax. Looking at the lastutterance_pattern,[WAKE, 1-2, PLAY_SONG], we see the range, 1-2. Putput will expand thisutterance_patternto twoutterance_patterns,[WAKE, PLAY_SONG]and[WAKE, WAKE, PLAY_SONG]. Ranges are inclusive and may also be specified as a single number, which would expand into oneutterance_pattern.Finally,groupsmay be defined withingroups. For instance:-groups:-PLAY_SONG:[PLAY,SONG]-WAKE_PLAY_SONG:[WAKE,PLAY_SONG,10]is valid syntax.Single Intent Providers (LUIS, Rasa, Lex, etc.)If your NLU provider only supports single intent utterances you can still use putput to generate utterances in the more familiar intent/entities paradigm. To specify single intents, simply add another level to the utterance patterns with the intent as the key and all it's utterance patterns beneath. To specify entities add a new section called 'entities' with a list of tokens that you want to be picked up as entities. For example:base_tokens:-PERSONAL_PRONOUNS:[he,she]-SPEAKER:[cortana,siri,alexa,google]token_patterns:-static:-WAKE:-[[hi,hey],SPEAKER]-PLAY:-[PERSONAL_PRONOUNS,[wants,would like],[to],[play]]-[[play]]-dynamic:-ARTIST-SONGentities:[ARTIST,SONG]# Here we specify which tokens are our entitiesutterance_patterns:-SONG_INTENT:# Here we specify our intents and which utterance patterns belong to them-[WAKE,PLAY,SONG]-[WAKE,1-2,PLAY,SONG]-ARTIST_INTENT:-[WAKE,PLAY,ARTIST]For a full example using the single intent pattern checkout thisLUIS examplePipelineAfter defining thepattern definition, the final step to generating labeled data is instantiatingputput'sPipelineand callingflow.dynamic_token_patterns_map={'SONG':('here comes the sun','stronger'),'ARTIST':('the beatles','kanye west')}p=Pipeline(pattern_def_path,dynamic_token_patterns_map=dynamic_token_patterns_map)forutterance,tokens,groupsinp.flow():print(utterance)print(tokens)print(groups)flowyields results oneutteranceat a time. While the results could be the tuple(utterance, tokens, groups)for each iteration, they could also be customized by specifying arguments toPipeline. Some common use cases are limiting the size of the output, oversampling/undersamplingutterance_patterns, specifying howtokensandgroupsare tokenized, etc. Customization of thePipelineis extensive and is covered in thePipeline'sdocs. Commonpresetconfigurations are covered in thepresetdocs.