M ixing B oard: a Knowledgeable Stylized Integrated Text Generation Platform (2020)

Abstract: We present MixingBoard, a platform for quickly building demos with a focus on knowledge grounded stylized text generation. We unify existing text generation algorithms in a shared codebase and further adapt earlier algorithms for constrained generation. To borrow advantages from different models, we implement strategies for cross-model integration, from the token probability level to the latent space level. An interface to external knowledge is provided via a module that retrieves, on-the-fly, relevant knowledge from passages on the web or a document collection. A user interface for local development, remote webpage access, and a RESTful API are provided to make it simple for users to build their own demos.

## introduction neural text generation algorithms have seen great improvements over the past several years @xcite . however each algorithm and neural model usually focuses on a specific task and may differ significantly from each other in terms of architecture, implementation, interface, and training domains. it is challenging to unify these algorithms theoretically, but a framework to organically integrate multiple algorithms and components can benefit the community in several ways, as it provides (1) a shared codebase to reproduce and compare the state-of-the-art algorithms from different groups without time consuming trial and errors, (2) a platform to experiment the cross-model integration of these algorithms, and (3) a framework to build demo quickly upon these components. this framework can be built upon existing deep learning libraries @xcite 1 source code available at github.com/microsoft/ mixingboard figure 1: mixingboard is designed as a platform to organically and quickly integrate separate nlp algorithms into compelling demos @xcite and neural nlp toolkits (hug-gingface, 2019; @xcite @xcite @xcite foot_0 , as illustrated in fig. 1 . there are several challenges to do such integration. firstly, engineering efforts are needed to unify the interface of different implementation. secondly, a top-level manager needs to be designed to utilize different models together. finally, different models are trained using different data with different performance. cross-model integration, instead of calling each isolated model individually, can potentially improve the overall performance. in this work, we unified the models of different implementation in a single codebase, implemented demos as top-level managers to access different models, and provide strategies to allow more organic integration across the models, including token probability interpolation, cross-mode scoring, latent interpolation, and unified hypothesis ranking. this work is also aimed to promote the development of grounded text generation. the exist-ing works focusing on the knowledge grounded text generation @xcite @xcite usually assume the knowledge passage is given. however in practice this is not true. we provide the component to retrieve knowledge passage on-the-fly from web or customized document, to allow engineers or researchers test existing or new generation models. keyphrase constrained generation @xcite is another type of grounded generation, broadly speaking. similarly the keyphrase needs to be provided to apply such constraints. we provide tools to extract constraints from knowledge passage or stylized corpus. finally, friendly user interface is a component usually lacking in the implementation of neural models but it is necessary for a demo-centric framework. we provide scripts to build local terminal demo, webpage demo, and restful api demo. ## design our goal is to build a framework that will allow users to quickly build text generation demos using existing modeling techniques. this design allows the framework to be almost agnostic to the ongoing development of text generation techniques @xcite . instead, we focus on the organic integration of models and the interfaces for the final demo/app. from a top-down view, our design are bounded to two markets: text processing assistant, and conversational ai, as illustrated in fig. 2 . two demos are present as examples in these domains: document auto-completion and sherlock holmes. we further breakdown these demos into several tasks, designed to be shared across different demos. we also designed several strategies to integrate multiple models to generate text. these strategies allow each model to plug-in without heavy constraints on the architecture of the models, as detailed in section 4. as the goal is not another deep learning nlp toolkit, we rely on existing ones @xcite @xcite and online api services bing web search provided in azure cognitive service foot_1 and tagme. 4 similarly, most tasks are using existing algorithms: language modeling @xcite , knowledge grounded generation @xcite or span retrieval @xcite , style transfer @xcite and constrained generation @xcite . ## cross-model integration multiple models may be called for the same query and returns different responses. we propose the following ways to organically integrate multiple models, as illustrated in fig. 4 . user can apply these strategies with customized models. ## user interface we provided the following three ways to access the demos introduced above for local developer, remote human user, and interface for other programs. ## conclusion mixingboard is a new open-source platform to organically integrate multiple state-of-the-art nlp algorithms to build demo quickly with user friendly interface. we unified these nlp algorithms in a single codebase, implemented demos as top-level managers to access different models, and provide strategies to allow more organic integration across the models. we provide the component to retrieve knowledge passage on-the-fly from web or customized document for grounded text generation. for future work, we plan to keep adding the stateof-the-art algorithms, reduce latency and fine-tune the implemented models on larger and/or more comprehensive corpus to improve performance.