Package: anyGPT
anyGPTanyGPT is a general purpose library for training any type of GPT model. Support for gpt-1, gpt-2, and gpt-3 models.
Inspired bynanoGPTbyAndrej Karpathy, the goal
of this project is to provide tools for the training and usage of GPT style large language models. The aim is to provide
a tool that isproduction readyeasily configurablescalablefree and open-sourceaccessible by general software engineers and enthusiastseasily reproducible and deployableYou don't need a Ph.D. in Machine Learning or Natural Language Processing to use anyGPT.InstallationNOTE: It is recommended that you set up of a python virtual environment
usingmamba,conda,
orpoetry.
To install anyGPT:$pipinstallanyGPTUsing DockerThe Docker image supports GPU passthrough for training and inference. In order to enable GPU
passthrough please follow the guide for installing theNVidia Container Toolkitfor your OS.NOTEOn Windows you need to follow the guide to getNVidia Container Toolkitsetup on WSL2.Docker WSL2 Backendis required.Once NVidia Container Toolkit and Docker is setup correctly, build the Docker image$dockerbuild-tanygpt.Use the following command to login to the container interactively, and use
anygpt as if it was on your local host$dockerrun--gpusall-itanygptMounting VolumesIt is recommended to mount a local directory into your container in order to
share data between your local host and the container. This will allow you to
save trained checkpoints, reuse datasets between runs and more.$dockerrun--gpusall-v/path/to/local/dir:/data-itanygptThe above example mounts/path/to/local/dirto the/datadirectory in the container, and all data and changes are shared between them dynamically.Non interactive DockerThe above documentation explains how to run a Docker container with an interactive session of anyGPT. You can
also run anyGPT commands to completion using Docker by overriding the entrypoint$dockerrun--gpus=all-v/path/to/your/data:/data--entrypointanygpt-run-itanygpt/data/test.pt"hello world"The above command runsanygpt-runwith the parameters/data/test.pt "hello world"Dependenciestorch >= 2.0.0numpytransformersdatasetstiktokenwandbtqdmPyYAMLlightningtensorboardFeaturesCurrentCLI and config file driven GPT trainingSupports CPU, GPU, TPU, IPU, and HPUDistributed training strategies for training at scaleEasy spin up using DockerFastAPI end-points for containerized microservice deploymentHuggingFace integrationLoad pre-trained gpt modelsRoadmapDocumentationHuggingFace integrationpush to hubEasy spin VM spinup/getting started withDownloading of pre-trained modelsGradio ChatGPT style interface for testing and experimentationFine-tuning of pre-trained modelsReinforcement Learning from Human Feedback and Rules Base Reward Modeling for LLM alignmentMore dataformat support beyond hosted text filesUsageData Preparation$anygpt-prepare-data-nshakespeare_complete-uhttps://www.gutenberg.org/cache/epub/100/pg100.txtTrainingCreate a config file. In this example, I'll call itgpt-2-30M.yaml. You can also check out theexample configuration files.model_config:name:'gpt-2-30M'block_size:256dropout:0.2embedding_size:384num_heads:6num_layers:6training_config:learning_rate:1.0e-3batch_size:8accumulate_gradients:8beta2:0.99min_lr:1.0e-4max_steps:5000val_check_interval:200limit_val_batches:100io_config:experiment_name:'gpt-2'dataset:'shakespeare_complete'$anygpt-traingpt-2-30M.yamlInference$anygpt-runresults/gpt-2-pretrain/version_0/checkpoints/last.pt\"JAQUES.All the world’s a stage,And all the men and women merely players;They have their exits and their entrances,And one man in his time plays many parts,"Inference MicroserviceanyGPT supports running models as a hosted microservice with a singular endpoint for inference.
To launch the microservice, use theanygpt-serveentrypoint.Commandline Options$anygpt-serve-h
usage:anyGPTinferenceservice[-h][--portPORT][--log-levelLOG_LEVEL]model

LoadsananyGPTmodelandhostsitonasimplemicroservicethatcanruninferenceoverthenetwork.

positionalarguments:modelPatht0thetrainedmodelcheckpointtoload

options:-h,--helpshowthishelpmessageandexit--portPORTPorttostartthemicroserviceon(default:5000)--hostHOSTHosttobindmicroserviceto(default:127.0.0.1)--log-levelLOG_LEVELuvicornloglevel(default:info)Example$anygpt-serveresults/gpt-2-pretrain/version_0/checkpoints/last.pt--port5000--log-levelinfoSending Requestsanygpt-serveusesFastAPIto serve the microservice.
To see the available microservice api go to the/docsendpoint in your browser once the microservice is startedSpinning up Microservice in Docker$dockerrun--gpus=all-v/path/to/your/data:/data-p5000:5000--entrypointanygpt-serve-itanygpt/data/test.pt--port5000--host0.0.0.0--log-levelinfo
INFO:Startedserverprocess[1]INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://127.0.0.1:5000(PressCTRL+Ctoquit)DocumentationLimitationsTBDLicenseThe goal of this project is to enable organizations, both large and small, to train and use GPT style
Large Language Models. I believe the future is open-source, with people and organizations being able to
train from scratch or fine-tune models and deploy to production without relying on gatekeepers. So I'm releasing this
under anMIT licensefor the benefit of all and in the hope that the community will find it useful.Released underMITby@any-LABS.