Package: codewit-semeru
What-If-Code-ToolVisualization Tool for Code Generation Model AnalysisMain IdeaGoogle WITwas the main inspiration for this project. Our goal is to create a similar tool purely for focusing on ML models revolving around software engineering design and principles, such as code completion and code generation.BertVizis a good first example for where our tool will go. We hope to support a full dashboard of several views that researchers would find helpful in order to analyze their models. This would probably include newly generated word count charts, probability distributions for new tokens, and attention views.DevelopmentPip tool: user can install this tool from pip/conda and utilize with their NLP modelPython Backend: user designates dataset and model as parameters for our tool. Our tool then runs the model and produces some vector dataset in its object.Jupyter-Dash Frontend: Jupyter-Dash allows for easy creation for data dashboard. Provides ability for easy callback methods with just Python.Development PlansCode concept groupings view: categorize each of the tokens generated in output based on what type they are in code language (declaration, assignment, functions, etc.)Display some statistics about the generated output with specific model (median, max, min, etc.)Dynamics re-execution of pipeline when:User edits # of tokensUser edits # of input sequencesUser changes modelUser selects new descriptive statisticImplement bertviz attention models inside app with Dash if possibleCurrent DiagramsComponents UMLSequence DiagramSupported Features4 different views to visually classify code generation models (ind. token, token distrubtion, python token types, token type distribtuion)4 pre-trained models for code generation from Hugging Face (GPT2, CodeGen, CodeParrot, GPT-Neo)Descriptive stats for datasets with many input sequencesDynamic re-execution on user inputsInstallationFirst prototype is currently available on PyPi. User will need to generate their own Hugging Face API token.%pip install codewit-semeru
%load_ext autoreload
%autoreload 2%pip install datasets

from datasets.load import load_dataset
import pandas as pd

DATA_LEN = 1024
NUM_DATA = 20

dataset = load_dataset("code_x_glue_cc_code_completion_line", "python", split="train")

pruned_dataset = []
for i, input_seq in enumerate(dataset):
    temp = input_seq["input"]  # type: ignore
    if len(temp) <= DATA_LEN:
        pruned_dataset.append(temp)
    if len(pruned_dataset) >= NUM_DATA:
        break
pd.DataFrame(pruned_dataset).describe()import os

os.environ["HF_API_TOKEN"] = "{Insert token here}"

from codewit_semeru import WITCode
WITCode("codeparrot/codeparrot-small", pruned_dataset)These lines can be run directly from your notebook. Python 3.8 is required. First chunk installs pip module, load auto-reload function. Second chunk loads up the CodeXGlue Code Completion dataset to be utilized with our tool. The last block is the actual implementaion in notebook to run our tool. User needs to supply their own api token to query HF models.Build and Run Docker ImageStart dockerNavigate to project folder and rundocker-compose up -d --buildto build imageNavigate tolocalhost:8888to run jupyter notebook. password iswitTo stop docker container rundocker-compose downBuild and Run Docker ImageStart dockerNavigate to project folder and rundocker-compose up -d --buildto build imageNavigate tolocalhost:8888to run jupyter notebook. password iswitTo stop docker container rundocker-compose down