V isua LLM : Easy Web-based Visualization for Neural Language Generation (2023)

Abstract: VisuaLLM is a Python library that enables interactive visualization of common tasks in natural language generation with pretrained language models (using HuggingFace’s model API), with tight integration of benchmark datasets and fine-grained generation control. The system runs as a local generation backend server and features a web-based frontend, allowing simple interface configuration by minimal Python code. The currently implemented views include data visualization, next-token prediction with probability distributions, and decoding parameter control, with simple extension to additional tasks.

## introduction while pretrained language models (plms) reached state-of-the-art performance on many natural language generation (nlg) benchmarks @xcite @xcite , they are hard to control directly and are often used as a black-box architecture, where the developer feeds linearized training data and retrieves outputs in a batch-wise manner, without access to fine-grained model behavior. interfaces for interactive plm testing (such as the openai playground foot_0 or the huggingface website) foot_1 typically only allow very basic operation, do not show any information beyond inputs/prompts and outputs, and are not connected to benchmark datasets. this paper presents visuallm, a simple, extensible library for visualization of plm generation processes, built as a web-based frontend on top of the huggingface transformers and datasets frameworks @xcite , taking inspiration from generic analysis tools such as tensorboard foot_2 or wandb. 4 the current version allows to visualize tabular nlg datasets and the processes of ## system architecture we build on huggingface transformers @xcite as the most commonly used plm framework. we expect the programmer to load huggingface model and dataset objects and pass them to our framework as shown in figure 1 . vi-suallm is used as a local server running on the user's machine, similar to e.g. tensorboard. the code is divided into a web-based frontend (written in vue.js) and a python backend (based on flask). the frontend is built in a modular fashion, 100% configurable from python code on the backend. any frontend-backend communication is therefore abstracted away from the user. the whole setup is designed to use as little code as possible; customizing for any new huggingface-based model or task takes typically just a few dozen lines of code. ## usecases we demonstrate visuallm by visualizing dialogue generation on the personachat benchmark @xcite . the presented views can easily be extended or modified for other nlg tasks. for instance, we are currently working on an extension to allow interactive user input. dataset visualization figure 2 shows a single data instance from personachat. the dataset assumes a short persona description and preceding dialogue context as inputs for response generation. the interface shows both a tabular human-readable representation and a low-level linearized model input (configured for a specific trained model). next token prediction visualization visualizing next-token probability distributions is another common debugging task for plms. in figure 3 , we show that visuallm allows the user to interactively step through sequence generation and control which token is selected, showing next-token probability distributions at each step. this view is also configured to include low-level model inputs and human reference outputs.