Package: openchatkit
OpenChatKitOpenChatKit provides a powerful, open-source base to create both specialized and general purpose chatbots for various applications. The kit includes an instruction-tuned 20 billion parameter language model, a 6 billion parameter moderation model, and an extensible retrieval system for including up-to-date responses from custom repositories. It was trained on the OIG-43M training dataset, which was a collaboration between Together, LAION, and Ontocord. Much more than a model release, this is the beginning of an open source project. We are releasing a set of tools and processes for ongoing improvement with community contributions.In this repo, you'll find code for:Training an OpenChatKit modelTesting inference using the modelAugmenting the model with additional context from a retrieval indexContentsRequirementsPre-trained WeightsDatasetsData ContributionsPretrained Base ModelTraining and Finetuning(Optional) 8bit AdamTrain GPT-NeoX-Chat-Base-20BConverting Weights to Huggingface FormatInferenceMonitoringLoguruWeights & BiasesRetrieval-Augmented ModelsLicenseCiting OpenChatKitAcknowledgementsRequirementsBefore you begin, you need to install PyTorch and other dependencies.InstallMinicondafrom their website.Create an environment called OpenChatKit using theenvironment.ymlfile at the root of this repo.condaenvcreate-fenvironment.ymlThis repo also usesGit LFSto manage some files. Install it using the instructions on their site then run:gitlfsinstallPre-trained WeightsGPT-NeoXT-Chat-Base-20B is a 20B-parameter variant of GPT-NeoX, fine-tuned on conversational datasets. We are releasing pre-trained weights for this model astogethercomputer/GPT-NeoXT-Chat-Base-20Bon Huggingface.More details can be found on the model card forGPT-NeoXT-Chat-Base-20Bon Huggingface.DatasetsThe chat model was trained on theOIGdataset built by LAION, Together, and Ontocord. To download the dataset from Huggingface run the command below from the root of the repo.pythondata/OIG/prepare.pyOnce the command completes, the data will be in thedata/OIG/filesdirectory.Data ContributionsYou can help make this chat model better by contributing data! See theOpenDataHubrepo for more details.Pretrained Base ModelAs mentioned above, the chat model is a fine-tuned variant of GPT-NeoX-20B from Eleuther AI. To download GPT-NeoX-20B and prepare it for fine tuning, run this command from the root of the repo.pythonpretrained/GPT-NeoX-20B/prepare.pyThe weights for this model will be in thepretrained/GPT-NeoX-20B/EleutherAI_gpt-neox-20b.Training and Finetuning(Optional) 8bit AdamTo use 8bit-adam during training, install thebitsandbytespackage.pipinstallbitsandbytes# optional, to use 8bit-adamTrain GPT-NeoX-Chat-Base-20BThetraining/finetune_GPT-NeoXT-Chat-Base-20B.shscript configures and runs the training loop. After downloading the dataset and the base model, run:bashtraining/finetune_GPT-NeoXT-Chat-Base-20B.shThe script launches 8 processes with a pipeline-parallel degree of 8 and a data-parallel degree of 1.As the training loop runs, checkpoints are saved to themodel_ckptsdirectory at the root of the repo.Please seethe training READMEfor more details about customizing the training run.Converting Weights to Huggingface FormatBefore you can use this model to perform inference, it must be converted to the Hugginface format.mkdirhuggingface_models\&&pythontools/convert_to_hf_gptneox.py\--ckpt-pathmodel_ckpts/GPT-Neo-XT-Chat-Base-20B/checkpoint_5--save-path/huggingface_models/GPT-NeoXT-Chat-Base-20B--n-stages8--n-layer-per-stage6InferenceTo help you test the model, we provide a simple test command line test harness to interact with the bot.pythoninference/bot.pyBy default the script will load the model named GPT-NeoXT-Chat-Base-20B model under thehuggingface_modelsdirectory, but you can override that behavior by specifying--model.For example, if you want to load the base model from our Huggingface, repo, you can run the following command which downloads the weights from HuggingFace.pythoninference/bot.py--modeltogethercomputer/GPT-NeoXT-Chat-Base-20BOnce the model has loaded, enter text at the prompt and the model will reply.$pythoninference/bot.pyLoading/home/csris/src/github.com/togethercomputer/OpenChatKit/inference/../huggingface_models/GPT-NeoXT-Chat-Base-20Btocuda:1...
WelcometoOpenChatKitshell.Type/helpor/?tolistcommands.

>>>Hello.
Setting`pad_token_id`to`eos_token_id`:0foropen-endgeneration.
Hellohuman.

>>>Commands are prefixed with a/, and the/quitcommand exits.MonitoringBy default, the training script simply prints the loss as training proceeds, but it can also output metrics to a file usingloguruor report them to Weights & Biases.LoguruAdd the flag--train-log-backend loguruto your training script to log to./logs/file_{time}.logWeights & BiasesTo use Weights & Biases, first login with your Weights & Biases token.wandbloginAnd set--train-log-backend wandbin the training script to enable logging to Weights & Biases.Retrieval-Augmented ModelsThe code in/retrievalimplements a python package for querying a Faiss index of Wikipedia. The following steps explain how to use this index to augment queries in the test harness with context from the retriever.Donwload the Wikipedia index.pythondata/wikipedia-3sentence-level-retrieval-index/prepare.pyRun the bot with the--retrievalflag.pythoninference/bot.py--retrievalAfter starting, the bot will load both the chat model and the retrieval index, which takes a long time. Once the model and the index are loaded, all queries will be augmented with extra context.$pythoninference/bot.py--retrieval
Loading/OpenChatKit/inference/../huggingface_models/GPT-NeoXT-Chat-Base-20Btocuda:0...
Loadingretrievalindex...
WelcometoOpenChatKitshell.Type/helpor/?tolistcommands.

>>>WhereisZurich?
Setting`pad_token_id`to`eos_token_id`:0foropen-endgeneration.
WhereisZurich?
ZurichislocatedinSwitzerland.

>>>LicenseAll code in this repository was developed by Together Computer except where otherwise noted.  Copyright (c) 2023, Together Computer.  All rights reserved. The code is licensed under the Apache 2.0 license.Copyright 2023 Together Computer

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.This repository also contains code written by a number of other authors. Such contributions are marked and the relevant licensing is included where appropriate.For full terms, see the LICENSE file. If you have any questions, comments, or concerns about licensing pleasecontact us.Citing OpenChatKit@software{openchatkit,title={{OpenChatKit: An Open Toolkit and Base Model for Dialogue-style Applications}},author={Together Computer},url={https://github.com/togethercomputer/OpenChatKit}month={3},year = {2023},version = {0.15},}AcknowledgementsOur model is a fine-tuned version ofgpt-neox-20b, a large language model trained byEleuther AI. We evaluated our model onHELMprovided by theCenter for Research on Foundation Models. And we collaborated with bothCRFMandHazyResearchat Stanford to build this model.