Package: pgml-chat
PostgresML Chatbot BuilderA command line tool to build and deploy aknowledge basedchatbot using PostgresML and OpenAI API.There are two stages in building a knowledge based chatbot:Build a knowledge base by ingesting documents, chunking documents, generating embeddings and indexing these embeddings for fast queryGenerate responses to user queries by retrieving relevant documents and generating responses using OpenAI andOpenSourceAI APIThis tool automates the above two stages and provides a command line interface to build and deploy a knowledge based chatbot.PrerequisitesBefore you begin, make sure you have the following:PostgresML Database: Sign up for a freeGPU-powered databasePython version >=3.8(Optional) OpenAI API keyGetting startedCreate a virtual environment and installpgml-chatusingpip:pipinstallpgml-chatpgml-chatwill be installed in your PATH.Download.env.templatefile from PostgresML Github repository.wgethttps://raw.githubusercontent.com/postgresml/postgresml/master/pgml-apps/pgml-chat/.env.templateCopy the template file to.envUpdate environment variables with your PostgresML database credentials and OpenAI API key (optional).DATABASE_URL=<POSTGRES_DATABASE_URLstartswithpostgres://>OPENAI_API_KEY=<OPENAI_API_KEY># OptionalUsageYou can get help on the command line interface by running:(pgml-bot-builder-py3.9)pgml-chat%pgml-chat%pgml-chat--help
usage:pgml-chat[-h]--collection_nameCOLLECTION_NAME[--root_dirROOT_DIR][--stage{ingest,chat}][--chat_interface{cli,slack,discord}][--chat_historyCHAT_HISTORY][--bot_nameBOT_NAME][--bot_languageBOT_LANGUAGE][--bot_topicBOT_TOPIC][--bot_topic_primary_languageBOT_TOPIC_PRIMARY_LANGUAGE][--bot_personaBOT_PERSONA][--chat_completion_modelCHAT_COMPLETION_MODEL][--max_tokensMAX_TOKENS][--vector_recall_limitVECTOR_RECALL_LIMIT]PostgresMLChatbotBuilder

options:-h,--helpshowthishelpmessageandexit--collection_nameCOLLECTION_NAMENameofthecollection(schema)tostorethedatainPostgresMLdatabase(default:None)--root_dirROOT_DIRInputfoldertoscanformarkdownfiles.Requiredforingeststage.Notrequiredforchatstage(default:None)--stage{ingest,chat}Stagetorun(default:chat)--chat_interface{cli,slack,discord}Chatinterfacetouse(default:cli)--chat_historyCHAT_HISTORYNumberofmessagesfromhistoryusedforgeneratingresponse(default:0)--bot_nameBOT_NAMENameofthebot(default:PgBot)--bot_languageBOT_LANGUAGELanguageofthebot(default:English)--bot_topicBOT_TOPICTopicofthebot(default:PostgresML)--bot_topic_primary_languageBOT_TOPIC_PRIMARY_LANGUAGEPrimaryprogramminglanguageofthetopic(default:SQL)--bot_personaBOT_PERSONAPersonaofthebot(default:Engineer)--chat_completion_modelCHAT_COMPLETION_MODEL--max_tokensMAX_TOKENSMaximumnumberoftokenstogenerate(default:256)--vector_recall_limitVECTOR_RECALL_LIMITMaximumnumberofdocumentstoretrievefromvectorrecall(default:1)IngestIn this step, we ingest documents, chunk documents, generate embeddings and index these embeddings for fast query.LOG_LEVEL=DEBUGpgml-chat--root_dir<directory>--collection_name<collection_name>--stageingestYou will see output logging the pipelines progress.ChatYou can interact with the bot using the command line interface or Slack.Command Line InterfaceIn this step, we start chatting with the chatbot at the command line. You can increase the log level to ERROR to suppress the logs. CLI is the default chat interface.LOG_LEVEL=ERRORpgml-chat--collection_name<collection_name>--stagechat--chat_interfacecliYou should be able to interact with the bot as shown below. Control-C to exit.User(Ctrl-Ctoexit):Whoareyou?
PgBot:IamPgBot,anAIassistantheretoansweryourquestionsaboutPostgresML,anopensourcesoftware.HowcanIassistyoutoday?
User(Ctrl-Ctoexit):WhatisPostgresML?
Foundrelevantdocumentation....PgBot:PostgresMLisanopensourcesoftwarethatallowsyoutounlockthefullpotentialofyourdataanddrivemoresophisticatedinsightsanddecision-makingprocesses.Itprovidesadashboardwithanalyticalviewsofthetrainingdataandmodelperformance,aswellasintegratednotebooksforrapiditeration.PostgresMLisprimarilywritteninRustusingRocketasalightweightwebframeworkandSQLxtointeractwiththedatabase.

Ifyouhaveanyfurtherquestionsorneedmoreinformation,pleasefeelfreetosendanemailtoteam@postgresml.orgorjointhePostgresMLDiscordcommunityathttps://discord.gg/DmyJP3qJ7U.SlackSetupYou need SLACK_BOT_TOKEN and SLACK_APP_TOKEN to run the chatbot on Slack. You can get these tokens by creating a Slack app. Follow the instructionshereto create a Slack app.Include the following environment variables in your .env file:SLACK_BOT_TOKEN=<SLACK_BOT_TOKEN>SLACK_APP_TOKEN=<SLACK_APP_TOKEN>In this step, we start chatting with the chatbot on Slack. You can increase the log level to ERROR to suppress the logs.LOG_LEVEL=ERRORpgml-chat--collection_name<collection_name>--stagechat--chat_interfaceslackIf you have set up the Slack app correctly, you should see the following output:⚡️ Bolt app is running!Once the slack app is running, you can interact with the chatbot on Slack as shown below. In the example here, name of the bot isPgBot. This app responds only to direct messages to the bot.DiscordSetupYou need DISCORD_BOT_TOKEN to run the chatbot on Discord. You can get this token by creating a Discord app. Follow the instructionshereto create a Discord app. Include the following environment variables in your .env file:DISCORD_BOT_TOKEN=<DISCORD_BOT_TOKEN>In this step, we start chatting with the chatbot on Discord. You can increase the log level to ERROR to suppress the logs.pgml-chat--collection_name<collection_name>--stagechat--chat_interfacediscordIf you have set up the Discord app correctly, you should see the following output:2023-08-0216:09:57INFOdiscord.clientlogginginusingstatictokenOnce the discord app is running, you can interact with the chatbot on Discord as shown below. In the example here, name of the bot ispgchat. This app responds only to direct messages to the bot.Prompt EngineeringIn addition to relevant context retrieved from vector search, system prompt to generate accurate responses with minimum hallucinations requires prompt engineering.
Different chat completion models require different system prompts. Since the prompts including the context are long, they suffer fromlost in the middleproblem described inthis paper. Below are some of the prompts that we have used for different chat completion models.Default prompt (GPT-3.5 and open source models)Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say that you don't know, don't try to make up an answer.
Use three sentences maximum and keep the answer as concise as possible.
Always say "thanks for asking!" at the end of the answer.GPT-4 System promptYou are an assistant to answer questions about {topic}.\ 
Your name is {name}. You speak like {persona} in {language}. Use the given list of documents to answer user's question.\
Use the conversation history if it is applicable to answer the question. \n Use the following steps:\n \
1. Identify if the user input is really a question. \n \
2. If the user input is not related to the {topic} then respond that it is not related to the {topic}.\n \
3. If the user input is related to the {topic} then first identify relevant documents from the list of documents. \n \
4. If the documents that you found relevant have information to completely and accurately answers the question then respond with the answer.\n \
5. If the documents that you found relevant have code snippets then respond with the code snippets. \n \
6. Most importantly, don't make up code snippets that are not present in the documents.\n \
7. If the user input is generic like Cool, Thanks, Hello, etc. then respond with a generic answer. \n"Developer GuideClone this repository, start a poetry shell and install dependenciesgitclonehttps://github.com/postgresml/postgresmlcdpostgresml/pgml-apps/pgml-chat
poetryshell
poetryinstall
pipinstall.Create a .env file in the root directory of the project and add all the environment variables discussed inGetting Startedsection.All the logic is inpgml_chat/main.pyCheck theroadmapfor features that you would like to work on.If you are looking for features that are not included here, please open an issue and we will add it to the roadmap.RoadmapUse a collection for chat history that can be retrieved and used to generate responses.Support for file formats like rst, html, pdf, docx, etc.Support for open source models in addition to OpenAI for chat completion.Support for multi-turn converstaions using converstaion buffer.