Package: gptextual
GPTextualA Terminal-Based Chat Client for various LLMs built withTextualFeaturesLocally persisted conversations with LLMs from multiple API providers (OpenAI, Google, SAP, more to come)Configurable as to which providers and models you want to chat withMultiline text inputLLM streaming (if the model supports it)Search through all conversations and navigate to the exact message you selectExport conversations to markdown for sharing/archivingConversations are stored as.parquetfiles, so can easily be processed/exported into other formats if requiredCopy messages or only code blocks to clibboardLLM Function callingAutomatic trimming of conversation to the context window size of the LLM (if tokenization model is available)Light and Dark themeRelated Projects and CreditsThis project is a fork of theEliaproject by Darren Burns, with most of the runtime logic and parts of the UI rewritten, especially to decouple the UI from the conversational runtime and to abstract the app from the concrete LLM it should connect to.This app also integratesTooLongas the in-place log file viewer.The LLMs are called and streamed viaLangChainAPI clients.The connection to SAPs Generative AI Hub is provided by the correspondingSDK library.InstallationPrerequisites:Tested only with Python >= 3.10StepsTo avoid any conflicts with other packages or system-wide Python installations, it is recommended to use virtual environments (venv or conda). Create and activate a virtual environment using the following commands (for conda, see documentation):Change into directory where you want to create the virtual environment. Then:python3-mvenvgptextual-envActivate:On Windows:.\gptextual-env\Scripts\activateOn Unix or MacOS:sourcegptextual-env/bin/activateInstall the package:pipinstallgptextual[all]gptextual can be installed with these flavors:gptextual[openai]Only the native OpenAI clients will be installedgptextual[google]Only the native Google GenAI clients will be installedgptextual[sap]Only the generative ai hub SDK clients by SAP will be installedgptextual[all]All supported clients in any given release will be installedThis way you can limit your installation footprint if you only want to use native OpenAI, for example.Quickstart1. Create file~/.gptextual/config.ymland enter your API key for the API provider(s) you want to connect toapi_config:# Open AIopenai:api_key:<your key># Google Gen AI APIsgoogle:api_key:<your key># SAP Gen AI Hub for enterprise scenariosgen-ai-hub:client_id:strclient_secret:strauth_url:strapi_base:strresource_group:strIf you don't need one or more of the API providers, just remove them from the yaml file.
Other API providers (like Huggingface) will be added soon.2. Start the app by opening a terminal, then:gptxMake sure you activated the virtual environment.Terminals to useOn MacIt is recommended to rungptextualwithiTerm2, because it gives the best user experience. Markdown links in LLM responses will also be fully clickable by holding the Command key and all colors look as intended.On WindowsIt is recommended to useWindows Terminal.In VS CodeFor developers, the VSCode integrated terminal also renders the app well, markdown links are clickable via the Command key. This gives a nice IDE integration of LLMs for development support.More InfoFor more info on general settings and optimizations when using Textual based apps, consult theTextual FAQ.Using the UITextual based apps can be navigated with a mouse or by keyboard only.It is recommended to learn the key shortcuts and tab orders of the UI elements for the best and most efficient user experience.Configuration GuideIn order to usegptextualyou need to create a YAML file at~/.gptextual/config.ymland maintain the configuration for at least 1 API provider (OpenAI, Google, etc.).Note: For any configuration changes in theconfig.ymlto take effect, an application restart is required.API ProvidersThe following outlines the mandatory config to get started, the full configuration will be explained further below.Open AIapi_config:openai:api_key:<your key># mandatoryGoogleapi_config:google:api_key:<your key># mandatoryConfiguring both, OpenAI and Google would hence look like this:api_config:openai:api_key:<your key>google:api_key:<your key>SAP GenAI Hubapi_config:gen-ai-hub:client_id:strclient_secret:strauth_url:strapi_base:strresource_group:strYou will get these settings from your SAP GenAI Hub setup.gptextualwill automatically create the required config.json file at~/.aicore/config.json.ModelsFor each API provider you can specify which models should be selectable in the app.
This is optional for all API providers exceptgen-ai-hub.
For OpenAI and Google, default models will be selected, reflecting the models available at the time of thegptextualrelease.For example:api_config:openai:api_key:<your key># specify which models you want to chat with# the model name has to exist on the API platformmodels:# Examplesgpt-3.5-turbo:context_window:4096gpt-4:context_window:8192gpt-4-0125-preview:context_window:128000Double check the model names since they will be passed to the LangChain API client as-is.Thecontex_windowparameter is used toautomatically default a reasonable value for the maximum number of output tokens requestedautomatically trim the next request to only contain conversation messages that still fit into the context window sizeFunction Callinggptextualsupports LLM function calling of functions developed by you or provided as python packages you install.Currently, the following function calling protocols are supported:Open AI Tool Calls (New parallel API)Open AI Function Calls (Legacy API)Configuring Function CallingFunction Call SupportIn theconfig.ymlyou can configure the type of function call support for each model.
For example, from theOpen AI Websitewe see that only a few models support the parallel tool calling API, most of the rest will support the legacy function calling. So we could add the following to theconfig.ymlapi_config:openai:api_key:...function_call_support:# either listing the models one by onegpt-4-turbo-preview:openai_toolgpt-4-0125-preview:openai_tool# Or using a comma separated list of model names'gpt-4-1106-preview,gpt-3.5-turbo-0125,gpt-3.5-turbo-1106':'openai_tool'# or also specify a wildcard meaning "all other models not specified by name''*':openai_functionRegistering functions1. Develop your function in a new pip package and decorate it with the provided function decoratorregister_for_function_calling:For example:fromgptextual.runtime.function_callingimportregister_for_function_calling,get_function_config@register_for_function_callingdefmultiply(a:int,a:int):"""Multiply two integers together.Args:a: First integerb: Second integer"""returna*b@register_for_function_callingasyncdefgoogle_web_search(query:str)->str:"""Executes a Google search with the specified search string and returns the top 10 search results.For each result, a title, URL and preview snippet is returned.Args:query: the query string"""# Functions can also access an optional config from the config.ymlconfig=get_function_config(google_web_search)api_key,cx_id=None,Noneifconfig:api_key=config.get("api_key",None)cx_id=config.get("cx_id",None)...When developing the function, following the guidelines byLangChain.Specifically:Provide the function description for the LLM in the doc stringProvide the parameters in the docstring as given in the example2. Define an entry point with namegptextual_functionin thesetup.py/pyproject.tomlof your project:entry_points={"gptextual_function":["module_name = your.package:your_function"],},Note:gptextualwill only load your entry point, and expect each function to have the decorator above. So specifying one function per module in the entry point is enough, because all functions will be loaded when the module loads.3. Configure which functions should be used at runtimeEach function in your python environment that is registered via the decorator and entry point can be used for LLM function calling. In order to actually pick the functions you want to use, you need to list them in theconfig.yml:functions:# In this example the function 'google_web_search' will be exposed to LLMs that support function callinggoogle_web_search:# Functions can also have an optional configuration, in this case Google Cloud API keysapi_key:...# API Keycx_id:...# Custom search engine IDmultiply:{}# empty configFunctions Included withgptextualCurrentlygptextualcomes with the following example functions:google_web_searchExecutes a google search with your personal Google cloud key and custom search engine ID.More functions are planned in the future.UI ConfigurationThere is a config section directly related to the textual UI:textual:# When LLMs support streaming, this number specifies after how many response chunks# a refresh and re-layouting of the chat message will be triggered in the textual framework.# A smaller number will give a smoother streaming experience but will lead to more refresh calls.refresh_no_stream_chunks:3# gptextual comes with a light and dark theme.theme:light|darkLogginggptextualwrites into a JSON-lines log file located at~/.gptextual/logging/gptextual.jsonlThe log file can be opened in app with theCtrl+Lkeyboard shortcut. This will openTooLong, which is a very fast, real-time tailing log file viewer.Inspecting the log can be useful to check if the LLM is called like you expect or to inspect function calling results, for example.WithinTooLong, in addition to navigating line by line and searching, you can also navigate via log entry timestamps:Press:m: Navigate one minute forwardM: Navigate one minute backwardh: Navigate one hour forwardH: Navigate one hour backwardConfiguring the log levelBy default,gptextualsets the log level toINFO.You can change this in theconfig.ymlwith:# other config...log_level:DEBUG|INFO|WARNING|ERROR|CRITICALconfig.ymltemplateA completeconfig.ymlfor your reference:api_config:openai:api_key:...models:# Example: Only want to chat with these 3 models from OpenAI:gpt-3.5-turbo:context_window:4096gpt-4:context_window:8192gpt-4-0125-preview:context_window:128000function_call_support:# onlu activate function calling for GPT-4'gpt-4,gpt-4-0125-preview':'openai_tool'google:api_key:...# No models are specified, gptextual will select 'gemini-pro' by default# No SAP API specified, so you will not see that option in the UItextual:refresh_no_stream_chunks:3theme:light# theme: darkfunctions:# The gpt-4 models above will be able to call this function!google_web_search:api_key:...cx_id:...log_level:INFOConversation StorageThe conversations are stored in folder~/.gptextual/conversationsFor each conversation, there will beajsonheader file containing id, title, timestamp etcaparquetfile containing all conversation messagesTheparquetfile can be read with any library that supports it.gptextualusespolarsinternally.Markdown ExportWhen you export a conversation to markdown in the app, they are stored in folder~/.gptextual/exportsCopying Messages to clipboardMessages in the chatview are focusable. You see the selected message marked. When a message is selected, you canpresscto copy the whole message textpress ` (markdown code block char) to copy only the code blocks inside the message (if any)Bug ReportsSubmit bugs via GitHub issues in this repo.