Package: clarifai
Clarifai Python SDKThis is the official Python client for interacting with our powerfulAPI. The Clarifai Python SDK offers a comprehensive set of tools to integrate Clarifai's AI platform to leverage computer vision capabilities like classification , detection ,segementation and natural language capabilities like classification , summarisation , generation , Q&A ,etc into your applications. With just a few lines of code, you can leverage cutting-edge artificial intelligence to unlock valuable insights from visual and textual content.Website|Schedule Demo|Signup for a Free Account|API Docs|Clarifai Community|Python SDK Docs|Examples|Colab Notebooks|DiscordGive the repo a star ⭐Table Of ContentsInstallationGetting StartedInteracting with DatasetsInteracting with InputsInput UploadInput ListingInteracting with ModelsModel PredictModel TrainingModel ListingInteracting with WorkflowsWorkflow PredictWorkflow ListingWorkflow CreateWorkflow ExportSearchSmart Image SearchSmart Text SearchFiltersRetrieval Augmented Generation (RAG)More Examples:rocket: InstallationInstall from PyPi:pipinstall-UclarifaiInstall from Source:gitclonehttps://github.com/Clarifai/clarifai-python.gitcdclarifai-python
python3-mvenv.venvsource.venv/bin/activate
pipinstall-rrequirements.txt
pythonsetup.pyinstall:memo: Getting startedClarifai usesPersonal Access Tokens(PATs)to validate requests. You can create and manage PATs under your Clarifai account security settings.🔗Create PAT:Log into Portal → Profile Icon → Security Settings → Create Personal Access Token → Set the scopes → Confirm🔗Get User ID:Log into Portal → Profile Icon → Account → Profile → User-IDExport your PAT as an environment variable. Then, import and initialize the API Client.Set PAT as environment variable through terminal:export CLARIFAI_PAT={your personal access token}# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.userimportUserclient=User(user_id="user_id")# Get all appsapps_generator=client.list_apps()apps=list(apps_generator)ORPAT can be passed as constructor argumentfromclarifai.client.userimportUserclient=User(user_id="user_id",pat="your personal access token"):floppy_disk: Interacting with DatasetsClarifai datasets help in managing datasets used for model training and evaluation. It provides functionalities like creating datasets,uploading datasets and exporting datasets as .zip files.# Note: CLARIFAI_PAT must be set as env variable.# Create app and datasetapp=client.create_app(app_id="demo_app",base_workflow="Universal")dataset=app.create_dataset(dataset_id="demo_dataset")# execute data upload to Clarifai app datasetfromclarifai.datasets.upload.laoders.coco_detectionimportCOCODetectionDataLoadercoco_dataloader=COCODetectionDataLoader("images_dir","coco_annotation_filepath")dataset.upload_dataset(dataloader=coco_dataloader,get_upload_status=True)#upload text from csvdataset.upload_from_csv(csv_path='csv_path',input_type='text',csv_type='raw',labels=True)#upload data from folderdataset.upload_from_folder(folder_path='folder_path',input_type='text',labels=True)# Export Datasetfromclarifai.client.datasetimportDataset# Note: clarifai-data-protobuf.zip is acquired through exporting datasets within the Clarifai Platform.Dataset().export(save_path='output.zip',local_archive_path='clarifai-data-protobuf.zip'):floppy_disk: Interacting with InputsYou can useinputs()for adding and interacting with input data. Inputs can be uploaded directly from a URL or a file. You can also view input annotations and concepts.Input Upload# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.userimportUserapp=User(user_id="user_id").app(app_id="app_id")input_obj=app.inputs()#input upload from urlinput_obj.upload_from_url(input_id='demo',image_url='https://samples.clarifai.com/metro-north.jpg')#input upload from filenameinput_obj.upload_from_file(input_id='demo',video_file='demo.mp4')# text uploadinput_obj.upload_text(input_id='demo',raw_text='This is a test')Input Listing#listing inputsinput_generator=input_obj.list_inputs(page_no=1,per_page=10,input_type='image')inputs_list=list(input_generator)#listing annotationsannotation_generator=input_obj.list_annotations(batch_input=inputs_list)annotations_list=list(annotation_generator)#listing conceptsall_concepts=list(app.list_concepts())Input Download#listing inputsinput_generator=input_obj.list_inputs(page_no=1,per_page=1,input_type='image')inputs_list=list(input_generator)#downloading_inputsinput_bytes=input_obj.download_inputs(inputs_list)withopen('demo.jpg','wb')asf:f.write(input_bytes[0]):brain: Interacting with ModelsTheModelClass allows you to perform predictions using Clarifai models. You can specify which model to use by providing the model URL or ID. This gives you flexibility in choosing models. TheAppClass also allows listing of all available Clarifai models for discovery.
For greater control over model predictions, you can pass in anoutput_configto modify the model output as demonstrated below.Model Predict# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.modelimportModel"""Get Model information on details of model(description, usecases..etc) and info on training or# other inference parameters(eg: temperature, top_k, max_tokens..etc for LLMs)"""gpt_4_model=Model("https://clarifai.com/openai/chat-completion/models/GPT-4")print(gpt_4_model)# Model Predictmodel_prediction=Model("https://clarifai.com/anthropic/completion/models/claude-v2").predict_by_bytes(b"Write a tweet on future of AI",input_type="text")# Customizing Model Inference Outputmodel_prediction=gpt_4_model.predict_by_bytes(b"Write a tweet on future of AI","text",inference_params=dict(temperature=str(0.7),max_tokens=30))# Return predictions having prediction confidence > 0.98model_prediction=model.predict_by_filepath(filepath="local_filepath",input_type,output_config={"min_value":0.98})# Supports image, text, audio, video# Supports prediction by urlmodel_prediction=model.predict_by_url(url="url",input_type)# Supports image, text, audio, video# Return predictions for specified interval of videovideo_input_proto=[input_obj.get_input_from_url("Input_id",video_url=BEER_VIDEO_URL)]model_prediction=model.predict(video_input_proto,input_type="video",output_config={"sample_ms":2000})Model Training# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.appimportAppfromclarifai.client.modelimportModel"""Create model with trainable model_type"""app=App(user_id="user_id",app_id="app_id")model=app.create_model(model_id="model_id",model_type_id="visual-classifier")(or)model=Model('url')"""List training templates for the model_type"""templates=model.list_training_templates()print(templates)"""Get parameters for the model."""params=model.get_params(template='classification_basemodel_v1',save_to='model_params.yaml')"""Update the model params yaml and pass it to model.train()"""model_version_id=model.train('model_params.yaml')"""Training status and saving logs"""status=model.training_status(version_id=model_version_id,training_logs=True)print(status)Evaluate your trained modelWhen your model is trained and ready, you can evaluate by the following codefromclarifai.client.modelimportModelmodel=Model('url')model.evaluate(dataset_id='your-dataset-id')Compare the evaluation results of your models.fromclarifai.client.modelimportModelfromclarifai.client.datasetimportDatasetfromclarifai.utils.evaluationimportEvalResultComparemodels=['model url1','model url2']# or [Model(url1), Model(url2)]dataset='dataset url'# or Dataset(dataset_url)compare=EvalResultCompare(models=models,datasets=dataset,attempt_evaluate=True# attempt evaluate when the model is not evaluated with the dataset)compare.all('output/folder/')Models Listing# Note: CLARIFAI_PAT must be set as env variable.# List all model versionsall_model_versions=list(model.list_versions())# Go to specific model versionmodel_v1=client.app("app_id").model(model_id="model_id",model_version_id="model_version_id")# List all models in an appall_models=list(app.list_models())# List all models in community filtered by model_type, descriptionall_llm_community_models=App().list_models(filter_by={"query":"LLM","model_type_id":"text-to-text"},only_in_app=False)all_llm_community_models=list(all_llm_community_models):fire: Interacting with WorkflowsWorkflows offer a versatile framework for constructing the inference pipeline, simplifying the integration of diverse models. You can use theWorkflowclass to create and manage workflows usingYAMLconfiguration.
For starting or making quick adjustments to existing Clarifai community workflows using an initial YAML configuration, the SDK provides an export feature.Workflow Predict# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.workflowimportWorkflow# Workflow Predictworkflow=Workflow("workflow_url")# Example: https://clarifai.com/clarifai/main/workflows/Face-Sentimentworkflow_prediction=workflow.predict_by_url(url="url",input_type="image")# Supports image, text, audio, video# Customizing Workflow Inference Outputworkflow=Workflow(user_id="user_id",app_id="app_id",workflow_id="workflow_id",output_config={"min_value":0.98})# Return predictions having prediction confidence > 0.98workflow_prediction=workflow.predict_by_filepath(filepath="local_filepath",input_type="text")# Supports image, text, audio, videoWorkflows Listing# Note: CLARIFAI_PAT must be set as env variable.# List all workflow versionsall_workflow_versions=list(workflow.list_versions())# Go to specific workflow versionworkflow_v1=Workflow(workflow_id="workflow_id",workflow_version=dict(id="workflow_version_id"),app_id="app_id",user_id="user_id")# List all workflow in an appall_workflow=list(app.list_workflow())# List all workflow in community filtered by descriptionall_face_community_workflows=App().list_workflows(filter_by={"query":"face"},only_in_app=False)# Get all face related workflowsall_face_community_workflows=list(all_face_community_workflows)Workflow CreateCreate a new workflow specified by a yaml config file.# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.appimportAppapp=App(app_id="app_id",user_id="user_id")workflow=app.create_workflow(config_filepath="config.yml")Workflow ExportExport an existing workflow from Clarifai as a local yaml file.# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.workflowimportWorkflowworkflow=Workflow("https://clarifai.com/clarifai/main/workflows/Demographics")workflow.export('demographics_workflow.yml'):mag: SearchSmart Image SearchClarifai's Smart Search feature leverages vector search capabilities to power the search experience. Vector search is a type of search engine that uses vectors to search and retrieve text, images, and videos.Instead of traditional keyword-based search, where exact matches are sought, vector search allows for searching based on visual and/or semantic similarity by calculating distances between vector embedding representations of the data.Here is an example of how to use vector search to find similar images:# Note: CLARIFAI_PAT must be set as env variable.fromclarifai.client.searchimportSearchsearch=Search(user_id="user_id",app_id="app_id",top_k=1,metric="cosine")# Search by image urlresults=search.query(ranks=[{"image_url":"https://samples.clarifai.com/metro-north.jpg"}])fordatainresults:print(data.hits[0].input.data.image.url)Smart Text SearchSmart Text Search is our proprietary feature that uses deep learning techniques to sort, rank, and retrieve text data based on their content and semantic similarity.Here is an example of how to use Smart Text Search to find similar text:# Note: CLARIFAI_PAT must be set as env variable.# Search by textresults=search.query(ranks=[{"text_raw":"I love my dog"}])FiltersYou can use filters to narrow down your search results. Filters can be used to filter by concepts, metadata, and Geo Point.It is possible to add together multiple search parameters to expand your search. You can even combine negated search terms for more advanced tasks.For example, you can combine two concepts as below.# query for images that contain concept "deer" or "dog"results=search.query(ranks=[{"image_url":"https://samples.clarifai.com/metro-north.jpg"}],filters=[{"concepts":[{"name":"deer","value":1},{"name":"dog","value":1}]}])# query for images that contain concepts "deer" and "dog"results=search.query(ranks=[{"image_url":"https://samples.clarifai.com/metro-north.jpg"}],filters=[{"concepts":[{"name":"deer","value":1}],"concepts":[{"name":"dog","value":1}]}])Input filters allows to filter by input_type, status of inputs and by inputs_dataset_idresults=search.query(filters=[{'input_types':['image','text']}])Retrieval Augmented Generation (RAG)You can setup and start your RAG pipeline in 4 lines of code. The setup method automatically creates a new app and the necessary components under the hood. By default it uses themistral-7B-Instructmodel.fromclarifai.ragimportRAGrag_agent=RAG.setup(user_id="USER_ID")rag_agent.upload(folder_path="~/docs")rag_agent.chat(messages=[{"role":"human","content":"What is Clarifai"}])If you have previously run the setup method, you can instantiate the RAG class with the prompter workflow URL:fromclarifai.ragimportRAGrag_agent=RAG(workflow_url="WORKFLOW_URL"):pushpin: More ExamplesSee many more code examples in thisrepo.
Also see the officialPython SDK docs