Package: jabberwocky
JabberwockyVideo demo:https://user-images.githubusercontent.com/40480855/132139847-0d0014b9-022e-4684-80bf-d46031ca4763.mp4This was not really designed to be used as a standalone library - it was mostly used as a convenient way to structure and import code in other parts of theproject. Some components may be reusable for other projects combining GPT-3 with audio, however.Project DescriptionThis is the core library powering a GUI that provides an audio interface to GPT-3. My main goal was to provide a convenient way to interact with various experts or public figures: imagine discussing physics with Einstein or hip hop with Kanye (or hip hop with Einstein? 🤔). I often find writing and speaking to be wildly different experiences and I imagined the same would be true when interacting with GPT-3. This turned out to be partially true - the default Mac text-to-speech functionality I'm using here is certainly not as lifelike as I'd like. Perhaps more powerful audio generation methods will pop up in a future release...We also provide Task Mode containing built-in prompts for a number of sample tasks:SummarizationExplain like I'm 5TranslationHow To (step by step instructions for performing everyday tasks)Writing Style AnalysisExplain machine learning concepts in simple languageGenerate ML paper abstractsMMA Fight Analysis and PredictionGetting StartedClone the repo.git clone https://github.com/hdmamin/jabberwocky.gitInstall the necessary packages. I recommend using a virtual environment of some kind (virtualenv, conda, etc). If you're not using Mac OS, you could try installing portaudio with whatever package manager you're using, but app behavior on other systems is unknown.brew install portaudio
pip install -r requirements.txt
python -m nltk.downloader punktIf you havemakeinstalled you can simply use the command:make installAdd your openai API key somewhere the program can access it. There are two ways to do this:echo your_openai_api_key > ~/.openaiorexport OPENAI_API_KEY=your_openai_api_key(Make sure to use your actual key, not the literal textyour_openai_api_key.)Run the app.python gui/main.pyOr withmake:make runUsageConversation ModeIn conversation mode, you can chat with a number of pre-defined personas or add new ones. New personas can be autogenerated or defined manually.Seedata/conversation_personasfor examples of autogenerated prompts. You can likely achieve better results using custom prompts though.Conversation mode only supports spoken input, though you can edit flawed transcriptions manually. Querying GPT-3 with nonsensical or ungrammatical text will negatively affect response quality.Task ModeIn task mode, you can ask GPT-3 to perform a number pre-defined tasks. Written and spoken input are both supported. By default, GPT-3's response is both typed out and read aloud.Transcripts of responses from a small subset of non-conversation tasks can be found in thedata/completionsdirectory. You can also save your own completions while using the app.Usage NotesThe first time you speak, the speech transcription back end will take a few seconds to calibrate to the level of ambient noise in your environment. You will know it's ready for transcription when you see a "Listening..." message appear below the Record button. Calibration only occurs once to save time.HotkeysCTRL + SHIFT: Start recording audio (same as pressing the "Record" button).CTRL + a: Get GPT-3's response to whatever input you've recorded (same as pressing the "Get Response" button).Project MembersHarrison MaminRepo Structurejabberwocky/
├── data         # Raw and processed data. Some files are excluded from github but the ones needed to run the app are there.
├── notes        # Miscellaneous notes from the development process stored as raw text files.
├── notebooks    # Jupyter notebooks for experimentation and exploratory analysis.
├── reports      # Markdown reports (performance reports, blog posts, etc.)
├── gui          # GUI scripts. The main script should be run from the project root directory. 
└── lib          # Python package. Code can be imported in analysis notebooks, py scripts, etc.Start of auto-generated file data.Last updated: 2022-07-24 14:12:52FileSummaryLine CountLast ModifiedSize__init__.py_12022-07-23 20:30:4822.00 bcli.pyAt the moment, the only useful part of this is the `update_prompt_readme`command which updates the readme in data/prompts to make it easier to reviewwhat prompts are currently available.The chat functionality was a brief experiment with an old version ofJabberwocky and no longer reflects the current API. I'm keeping it around forfuture reference since I'm still interested in combining prompt_toolkit withjabberwocky and it may be useful to see my old thoughts on implementation(or at the very least, serve as a sort of API reference about theprompt_toolkit features I care about).1082022-06-25 11:08:063.71 kbconfig.pyDefine constants used throughout the project.722022-07-07 20:05:072.49 kbcore.pyCore functionality that ties together multiple APIs.6122022-04-22 21:46:5224.36 kbexternal_data.pyFunctionality to fetch and work with YouTube transcripts and Wikipedia data.4262022-07-14 20:35:0915.56 kbopenai_utils.pyUtility functions for interacting with the gpt3 api.A note on query functions:There are a number of different services (both paid and free) that provideaccess to GPT-like models. GPTBackend.query() provides a convenient interfaceto them by calling different query functions under the hood. These queryfunctions are all defined in this module and have names starting with'query_gpt'. There are a number of steps to defining a new query function:1. Use the `mark` decorator to set "batch_support" to either True or False.True means you can pass in a list of prompts, False means the prompt must be asingle string. GPTBackend will support batching either way, but it needs toknow whether each query function supports this natively in order to determinehow to do this.2. The first parameter should be 'prompt' with no default. The other parametersshould have defaults, but keep in mind many of these will be ignored whencalled by GPTBackend.query - the latter has its own defaults which are passedin as kwargs. Speaking of which...3. It must accept kwargs.4. If it supports multiple completions, include a parameter "n" with a defaultvalue of 1 in the signature. GPTBackend.query will support it either way, butagain it needs to know how to achieve this.5. If multiple engines are available through this backend, updatejabberwocky.config.C.backend_engines, using the other examples as guidance. Inaddition, in this scenario we recommend retrieving the engine name explicitlyas query_gpt_huggingface does:```engine = GPTBackend.engine(engine, backend='huggingface')```rather than simply calling ```GPTBackend.engine(engine)```.6. We recommend popping off kwargs that you actually do want to use and thenproviding a warning so the user can see the remaining unused kwargs if thereare any.7. The fuction should either return a (str, dict-like) tuple or(list[str], list[dict-like]). Use the latter if batch_support is True andmultiple completions per prompt are supported (i.e. n is in its parameters).Technically I suppose there could be a backend that supported 1 but not theother, but I haven't seen it yet so I'll figure that case out if/when needed.8. When done, update some or all of the following GPTBackend class attributes.- name2base- name2func- skip_truncUse the comments and examples in the class as guidance.30902022-07-19 19:43:49130.72 kbspeech.pyModule to help us interact with mac's speech command. This lets the GUI readresponses out loud.1172021-08-20 21:05:114.16 kbstreaming.pyFunctionality to support streaming gpt responses, both from backends thatnatively support it like openai and those that don't. This handles things likeupdating prompt_index in each streamed response dict so we can more easily tellwhich prompt a token belongs to, truncating before stop words for backends thatdon't (only openai supports this natively), and updating finish_reason whenwe find stop words.Note: if adding a streaming function for a new backend, make sure to registerit so it ends up in BACKEND2STREAMER (see _stream_openai_generator for anexample). Each function must accept a first param which is the result ofcallingGPT.query(prompt, stream=True)while using the desired backend. It must also accept **kwargs. Be careful whenchanging param names - they must match the stream_response() call injabberwocky.openai_utils.GPTBackend._query().5752022-05-09 20:53:5123.75 kbutils.pyGeneral purpose utilities.6992022-07-19 18:57:3523.36 kbEnd of auto-generated file data. Do not add anything below this.