Package: jupyter-voicepilot
jupyter_voicepilotA JupyterLab extension for generating code and interacting with JupyterLab via voice commands. This extension can also be used for some basic nagivation around in JupyterLab Notebook. It is built around OpenAI'sWhisper-1andGPT-3APIs. You will need to have an OpenAI API key to use this extension.Click on the image below to see a demo of the extension:RequirementsJupyterLab >= 3.0InstallTo install the extension, execute:pipinstalljupyter_voicepilotUninstallTo remove the extension, execute:pipuninstalljupyter_voicepilotUsageClick theVoice Pilotbutton in the JupyterLab to start recording your instruction. When done, click the button again to stop the recording. The extension will then process your instruction and execute the appropriate action.Generating codeIf a cell is of typecode, the extension will generate code based on the input you provided.
The generated code will be inserted in the cell. An exception to this is when you provide a set phrase which is mapped to a specific Notebook navigation action. See below for more details.DictationIf a cell is of typemarkdown, the extension will insert the text you provided in the cell.
An exception to this is when you provide a set phrase which is mapped to a specific Notebook navigation action. See below for more details.ChatGPT interactionIf you start your voice message with "hey", the message will be sent to theChatGPTmodel.
If the current cell is empty, it will be converted to a markdown cell and the response will be added there. If the current cell is not empty, a newmarkdowncell will be inserted below the current cell, containing the response from ChatGPT.Notebook navigationThe extension can also be used for some basic nagivation around in JupyterLab Notebook. The following table shows the Notebook actions that a supported, and the corresponding phrases that you can use to trigger them.ActionPhraserun"run", "run cell", "run the cell", "execute"runAll"run all", "run all cells", "execute all", "execute all cells"runAndAdvance"run and advance", "run cell and advance", "execute and advance"runAndInsert"run and insert", "run cell and insert", "execute and insert"runAllAbove"run all above", "run all cells above", "execute all above", "execute all cells above"runAllBelow"run all below", "run all cells below", "execute all below", "execute all cells below"deleteCells"delete", "delete cell", "delete cells", "delete the cell", "delete the cells"clearAllOutputs"clear all outputs", "clear all the outputs", "clear outputs"selectLastRunCell"select last run cell", "select the last run cell"undo"undo"redo"redo"cut"cut"copy"copy"paste"paste"changeCellTypeToMarkdown"markdown", "to markdown", "markdown cell", "convert to markdown", "cast to markdown"changeCellTypeToCode"code", "to code", "code cell", "convert to code", "cast to code"insertMarkdownCellBelow"insert markdown cell below", "add markdown below"insertMarkdownCellAbove"insert markdown cell above", "add markdown above"insertCodeCellBelow"insert code cell below", "add code below"insertCodeCellAbove"insert code cell above", "add code above"ConfigurationIn the advanced settings editor, you can set the following configuration options:Open API Key (required): Your OpenAI API key. You can get onehere.ContributingDevelopment installNote: You will need NodeJS to build the extension package.Thejlpmcommand is JupyterLab's pinned version ofyarnthat is installed with JupyterLab. You may useyarnornpmin lieu ofjlpmbelow.# Clone the repo to your local environment# Change directory to the jupyter_voicepilot directory# Install package in development modepipinstall-e"."# Link your development version of the extension with JupyterLabjupyterlabextensiondevelop.--overwrite# Rebuild extension Typescript source after making changesjlpmbuildYou can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.# Watch the source directory in one terminal, automatically rebuilding when neededjlpmwatch# Run JupyterLab in another terminaljupyterlabWith the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).By default, thejlpm buildcommand generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:jupyterlabbuild--minimize=FalseDevelopment uninstallpipuninstalljupyter_voicepilotIn development mode, you will also need to remove the symlink created byjupyter labextension developcommand. To find its location, you can runjupyter labextension listto figure out where thelabextensionsfolder is located. Then you can remove the symlink namedvoicepilotwithin that folder.