Package: replicate
Replicate Python clientThis is a Python client forReplicate. It lets you run models from your Python code or Jupyter notebook, and do various other things on Replicate.👋Check out an interactive version of this tutorial onGoogle Colab.RequirementsPython 3.8+InstallpipinstallreplicateAuthenticateBefore running any Python scripts that use the API, you need to set your Replicate API token in your environment.Grab your token fromreplicate.com/accountand set it as an environment variable:export REPLICATE_API_TOKEN=<your token>We recommend not adding the token directly to your source code, because you don't want to put your credentials in source control. If anyone used your API key, their usage would be charged to your account.Run a modelCreate a new Python file and add the following code:>>>importreplicate>>>replicate.run("stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",input={"prompt":"a 19th century portrait of a wombat gentleman"})['https://replicate.com/api/models/stability-ai/stable-diffusion/files/50fcac81-865d-499e-81ac-49de0cb79264/out-0.png']Some models, likemethexis-inc/img2prompt, receive images as inputs. To pass a file as an input, use a file handle or URL:>>>output=replicate.run("salesforce/blip:2e1dddc8621f72155f24cf2e0adbde548458d3cab9f00c0139eea840d0ac4746",input={"image":open("path/to/mystery.jpg","rb")},)"an astronaut riding a horse"[!NOTE]
You can also use the Replicate client asynchronously by prependingasync_to the method name.Here's an example of how to run several predictions concurrently and wait for them all to complete:importasyncioimportreplicate# https://replicate.com/stability-ai/sdxlmodel_version="stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b"prompts=[f"A chariot pulled by a team of{count}rainbow unicorns"forcountin["two","four","six","eight"]]asyncwithasyncio.TaskGroup()astg:tasks=[tg.create_task(replicate.async_run(model_version,input={"prompt":prompt}))forpromptinprompts]results=awaitasyncio.gather(*tasks)print(results)Run a model and stream its outputReplicate’s API supports server-sent event streams (SSEs) for language models.
Use thestreammethod to consume tokens as they're produced by the model.importreplicate# https://replicate.com/meta/llama-2-70b-chatmodel_version="meta/llama-2-70b-chat:02e509c789964a7ea8736978a43525956ef40397be9033abf9fd2badfe68c9e3"foreventinreplicate.stream(model_version,input={"prompt":"Please write a haiku about llamas.",},):print(str(event),end="")For more information, see"Streaming output"in Replicate's docs.Run a model in the backgroundYou can start a model and run it in the background:>>>model=replicate.models.get("kvfrans/clipdraw")>>>version=model.versions.get("5797a99edc939ea0e9242d5e8c9cb3bc7d125b1eac21bda852e5cb79ede2cd9b")>>>prediction=replicate.predictions.create(version=version,input={"prompt":"Watercolor painting of an underwater submarine"})>>>predictionPrediction(...)>>>prediction.status'starting'>>>dict(prediction){"id":"...","status":"starting",...}>>>prediction.reload()>>>prediction.status'processing'>>>print(prediction.logs)iteration:0,render:loss:-0.6171875iteration:10,render:loss:-0.92236328125iteration:20,render:loss:-1.197265625iteration:30,render:loss:-1.3994140625>>>prediction.wait()>>>prediction.status'succeeded'>>>prediction.output'https://.../output.png'Run a model in the background and get a webhookYou can run a model and get a webhook when it completes, instead of waiting for it to finish:model=replicate.models.get("ai-forever/kandinsky-2.2")version=model.versions.get("ea1addaab376f4dc227f5368bbd8eff901820fd1cc14ed8cad63b29249e9d463")prediction=replicate.predictions.create(version=version,input={"prompt":"Watercolor painting of an underwater submarine"},webhook="https://example.com/your-webhook",webhook_events_filter=["completed"])For details on receiving webhooks, seereplicate.com/docs/webhooks.Compose models into a pipelineYou can run a model and feed the output into another model:laionide=replicate.models.get("afiaka87/laionide-v4").versions.get("b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05")swinir=replicate.models.get("jingyunliang/swinir").versions.get("660d922d33153019e8c263a3bba265de882e7f4f70396546b6c9c8f9d47a021a")image=laionide.predict(prompt="avocado armchair")upscaled_image=swinir.predict(image=image)Get output from a running modelRun a model and get its output while it's running:iterator=replicate.run("pixray/text2image:5c347a4bfa1d4523a58ae614c2194e15f2ae682b57e3797a5bb468920aa70ebf",input={"prompts":"san francisco sunset"})forimageiniterator:display(image)Cancel a predictionYou can cancel a running prediction:>>>model=replicate.models.get("kvfrans/clipdraw")>>>version=model.versions.get("5797a99edc939ea0e9242d5e8c9cb3bc7d125b1eac21bda852e5cb79ede2cd9b")>>>prediction=replicate.predictions.create(version=version,input={"prompt":"Watercolor painting of an underwater submarine"})>>>prediction.status'starting'>>>prediction.cancel()>>>prediction.reload()>>>prediction.status'canceled'List predictionsYou can list all the predictions you've run:replicate.predictions.list()# [<Prediction: 8b0ba5ab4d85>, <Prediction: 494900564e8c>]Lists of predictions are paginated. You can get the next page of predictions by passing thenextproperty as an argument to thelistmethod:page1=replicate.predictions.list()ifpage1.next:page2=replicate.predictions.list(page1.next)Load output filesOutput files are returned as HTTPS URLs. You can load an output file as a buffer:importreplicatefromPILimportImagefromurllib.requestimporturlretrieveout=replicate.run("stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",input={"prompt":"wavy colorful abstract patterns, oceans"})urlretrieve(out[0],"/tmp/out.png")background=Image.open("/tmp/out.png")List modelsYou can the models you've created:replicate.models.list()Lists of models are paginated. You can get the next page of models by passing thenextproperty as an argument to thelistmethod, or you can use thepaginatemethod to fetch pages automatically.# Automatic pagination using `replicate.paginate` (recommended)models=[]forpageinreplicate.paginate(replicate.models.list):models.extend(page.results)iflen(models)>100:break# Manual pagination using `next` cursorspage=replicate.models.list()whilepage:models.extend(page.results)iflen(models)>100:breakpage=replicate.models.list(page.next)ifpage.nextelseNoneYou can also find collections of featured models on Replicate:>>>collections=[collectionforpageinreplicate.paginate(replicate.collections.list)forcollectioninpage]>>>collections[0].slug"vision-models">>>collections[0].description"Multimodal large language models with vision capabilities like object detection and optical character recognition (OCR)">>>replicate.collections.get("text-to-image").models[<Model:stability-ai/sdxl>,...]Create a modelYou can create a model for a user or organization
with a given name, visibility, and hardware SKU:importreplicatemodel=replicate.models.create(owner="your-username",name="my-model",visibility="public",hardware="gpu-a40-large")Here's how to list of all the available hardware for running models on Replicate:>>>[hw.skuforhwinreplicate.hardware.list()]['cpu','gpu-t4','gpu-a40-small','gpu-a40-large']Fine-tune a modelUse thetraining APIto fine-tune models to make them better at a particular task.
To see whatlanguage modelscurrently support fine-tuning,
check out Replicate'scollection of trainable language models.If you're looking to fine-tuneimage models,
check out Replicate'sguide to fine-tuning image models.Here's how to fine-tune a model on Replicate:training=replicate.trainings.create(model="stability-ai/sdxl",version="39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",input={"input_images":"https://my-domain/training-images.zip","token_string":"TOK","caption_prefix":"a photo of TOK","max_train_steps":1000,"use_face_detection_instead":False},# You need to create a model on Replicate that will be the destination for the trained version.destination="your-username/model-name")DevelopmentSeeCONTRIBUTING.md