Package: leptonai
Lepton AIA Pythonic framework to simplify AI service buildingHomepage•API Playground•Examples•Documentation•CLI References•Twitter•BlogThe LeptonAI Python library allows you to build an AI service from Python code with ease. Key features include:A Pythonic abstractionPhoton, allowing you to convert research and modeling code into a service with a few lines of code.Simple abstractions to launch models like those onHuggingFacein few lines of code.Prebuilt examples for common models such as Llama, SDXL, Whisper, and others.AI tailored batteries included such as autobatching, background jobs, etc.A client to automatically call your service like native Python functions.Pythonic configuration specs to be readily shipped in a cloud environment.Getting started with one-linerInstall the library with:pipinstall-UleptonaiThis installs theleptonaiPython library, as well as the commandline interfacelep. You can then launch a HuggingFace model, saygpt2, in one line of code:lepphotonrun--namegpt2--modelhf:gpt2--localIf you have access to the Llama2 model (apply for access here) and you have a reasonably sized GPU, you can launch it with:# hint: you can also write `-n` and `-m` for shortlepphotonrun-nllama2-mhf:meta-llama/Llama-2-7b-chat-hf--local(Be sure to use the-hfversion for Llama2, which is compatible with huggingface pipelines.)You can then access the service with:fromleptonai.clientimportClient,localc=Client(local(port=8080))# Use the following to print the docprint(c.run.__doc__)print(c.run(inputs="I enjoy walking with my cute dog"))Fully managed Llama2 models and CodeLlama models can be found in theplayground.Many standard HuggingFace pipelines are supported - find out more details in thedocumentation. Not all HuggingFace models are supported though, as many of them contain custom code and are not standard pipelines. If you find a popular model you would like to support, pleaseopen an issue or a PR.Checking out more examplesYou can find out more examples from theexamples repository. For example, launch the Stable Diffusion XL model with:gitclonegit@github.com:leptonai/examples.gitcdexampleslepphotonrun-nsdxl-madvanced/sdxl/sdxl.py--localOnce the service is running, you can access it with:fromleptonai.clientimportClient,localc=Client(local(port=8080))img_content=c.run(prompt="a cat launching rocket",seed=1234)withopen("cat.png","wb")asfid:fid.write(img_content)or access the mounted Gradio UI athttp://localhost:8080/ui. Check theREADME filefor more details.A fully managed SDXL is hosted athttps://dashboard.lepton.ai/playground/sdxlwith API access.Writing your own photonsWriting your own photon is simple: write a Python Photon class and decorate functions with@Photon.handler. As long as your input and output are JSON serializable, you are good to go. For example, the following code launches a simple echo service:# my_photon.pyfromleptonai.photonimportPhotonclassEcho(Photon):@Photon.handlerdefecho(self,inputs:str)->str:"""A simple example to return the original input."""returninputsYou can then launch the service with:lepphotonrun-necho-mmy_photon.py--localThen, you can use your service as follows:fromleptonai.clientimportClient,localc=Client(local(port=8080))# will print available pathsprint(c.paths())# will print the doc for c.echo. You can also use `c.echo?` in Jupyter.print(c.echo.__doc__)# will actually call echo.c.echo(inputs="hello world")For more details, checkout thedocumentationand theexamples.ContributingContributions and collaborations are welcome and highly appreciated. Please check out thecontributor guidefor how to get involved.LicenseThe Lepton AI Python library is released under the Apache 2.0 license.Developer Note: early development of LeptonAI was in a separate mono-repo, which is why you may see commits from theleptonai/leptonrepo. We intend to use this open source repo as the source of truth going forward.