| {"guide": {"name": "Gradio-and-Comet", "category": "other-tutorials", "pretty_category": "Other Tutorials", "guide_index": null, "absolute_index": 56, "pretty_name": "Gradio And Comet", "content": "# Using Gradio and Comet\n\n\n\n\n## Introduction\n\nIn this guide we will demonstrate some of the ways you can use Gradio with Comet. We will cover the basics of using Comet with Gradio and show you some of the ways that you can leverage Gradio's advanced features such as [Embedding with iFrames](https://www.gradio.app/guides/sharing-your-app/#embedding-with-iframes) and [State](https://www.gradio.app/docs/#state) to build some amazing model evaluation workflows.\n\nHere is a list of the topics covered in this guide.\n\n1. Logging Gradio UI's to your Comet Experiments\n2. Embedding Gradio Applications directly into your Comet Projects\n3. Embedding Hugging Face Spaces directly into your Comet Projects\n4. Logging Model Inferences from your Gradio Application to Comet\n\n## What is Comet?\n\n[Comet](https://www.comet.com?utm_source=gradio&utm_medium=referral&utm_campaign=gradio-integration&utm_content=gradio-docs) is an MLOps Platform that is designed to help Data Scientists and Teams build better models faster! Comet provides tooling to Track, Explain, Manage, and Monitor your models in a single place! It works with Jupyter Notebooks and Scripts and most importantly it's 100% free!\n\n## Setup\n\nFirst, install the dependencies needed to run these examples\n\n```shell\npip install comet_ml torch torchvision transformers gradio shap requests Pillow\n```\n\nNext, you will need to [sign up for a Comet Account](https://www.comet.com/signup?utm_source=gradio&utm_medium=referral&utm_campaign=gradio-integration&utm_content=gradio-docs). Once you have your account set up, [grab your API Key](https://www.comet.com/docs/v2/guides/getting-started/quickstart/#get-an-api-key?utm_source=gradio&utm_medium=referral&utm_campaign=gradio-integration&utm_content=gradio-docs) and configure your Comet credentials\n\nIf you're running these examples as a script, you can either export your credentials as environment variables\n\n```shell\nexport COMET_API_KEY=\"<Your API Key>\"\nexport COMET_WORKSPACE=\"<Your Workspace Name>\"\nexport COMET_PROJECT_NAME=\"<Your Project Name>\"\n```\n\nor set them in a `.comet.config` file in your working directory. You file should be formatted in the following way.\n\n```shell\n[comet]\napi_key=<Your API Key>\nworkspace=<Your Workspace Name>\nproject_name=<Your Project Name>\n```\n\nIf you are using the provided Colab Notebooks to run these examples, please run the cell with the following snippet before starting the Gradio UI. Running this cell allows you to interactively add your API key to the notebook.\n\n```python\nimport comet_ml\ncomet_ml.init()\n```\n\n## 1. Logging Gradio UI's to your Comet Experiments\n\n[](https://colab.research.google.com/github/comet-ml/comet-examples/blob/master/integrations/model-evaluation/gradio/notebooks/Gradio_and_Comet.ipynb)\n\nIn this example, we will go over how to log your Gradio Applications to Comet and interact with them using the Gradio Custom Panel.\n\nLet's start by building a simple Image Classification example using `resnet18`.\n\n```python\nimport comet_ml\n\nimport requests\nimport torch\nfrom PIL import Image\nfrom torchvision import transforms\n\ntorch.hub.download_url_to_file(\"https://github.com/pytorch/hub/raw/master/images/dog.jpg\", \"dog.jpg\")\n\nif torch.cuda.is_available():\n device = \"cuda\"\nelse:\n device = \"cpu\"\n\nmodel = torch.hub.load(\"pytorch/vision:v0.6.0\", \"resnet18\", pretrained=True).eval()\nmodel = model.to(device)\n\n# Download human-readable labels for ImageNet.\nresponse = requests.get(\"https://git.io/JJkYN\")\nlabels = response.text.split(\"\\n\")\n\n\ndef predict(inp):\n inp = Image.fromarray(inp.astype(\"uint8\"), \"RGB\")\n inp = transforms.ToTensor()(inp).unsqueeze(0)\n with torch.no_grad():\n prediction = torch.nn.functional.softmax(model(inp.to(device))[0], dim=0)\n return {labels[i]: float(prediction[i]) for i in range(1000)}\n\n\ninputs = gr.Image()\noutputs = gr.Label(num_top_classes=3)\n\nio = gr.Interface(\n fn=predict, inputs=inputs, outputs=outputs, examples=[\"dog.jpg\"]\n)\nio.launch(inline=False, share=True)\n\nexperiment = comet_ml.Experiment()\nexperiment.add_tag(\"image-classifier\")\n\nio.integrate(comet_ml=experiment)\n```\n\nThe last line in this snippet will log the URL of the Gradio Application to your Comet Experiment. You can find the URL in the Text Tab of your Experiment.\n\n<video width=\"560\" height=\"315\" controls>\n <source src=\"https://user-images.githubusercontent.com/7529846/214328034-09369d4d-8b94-4c4a-aa3c-25e3ed8394c4.mp4\"></source>\n</video>\n\nAdd the Gradio Panel to your Experiment to interact with your application.\n\n<video width=\"560\" height=\"315\" controls>\n <source src=\"https://user-images.githubusercontent.com/7529846/214328194-95987f83-c180-4929-9bed-c8a0d3563ed7.mp4\"></source>\n</video>\n\n## 2. Embedding Gradio Applications directly into your Comet Projects\n\n<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/KZnpH7msPq0?start=9\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen></iframe>\n\nIf you are permanently hosting your Gradio application, you can embed the UI using the Gradio Panel Extended custom Panel.\n\nGo to your Comet Project page, and head over to the Panels tab. Click the `+ Add` button to bring up the Panels search page.\n\n<img width=\"560\" alt=\"adding-panels\" src=\"https://user-images.githubusercontent.com/7529846/214329314-70a3ff3d-27fb-408c-a4d1-4b58892a3854.jpeg\">\n\nNext, search for Gradio Panel Extended in the Public Panels section and click `Add`.\n\n<img width=\"560\" alt=\"gradio-panel-extended\" src=\"https://user-images.githubusercontent.com/7529846/214325577-43226119-0292-46be-a62a-0c7a80646ebb.png\">\n\nOnce you have added your Panel, click `Edit` to access to the Panel Options page and paste in the URL of your Gradio application.\n\n\n\n<img width=\"560\" alt=\"Edit-Gradio-Panel-URL\" src=\"https://user-images.githubusercontent.com/7529846/214334843-870fe726-0aa1-4b21-bbc6-0c48f56c48d8.png\">\n\n## 3. Embedding Hugging Face Spaces directly into your Comet Projects\n\n<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/KZnpH7msPq0?start=107\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen></iframe>\n\nYou can also embed Gradio Applications that are hosted on Hugging Faces Spaces into your Comet Projects using the Hugging Face Spaces Panel.\n\nGo to your Comet Project page, and head over to the Panels tab. Click the `+ Add` button to bring up the Panels search page. Next, search for the Hugging Face Spaces Panel in the Public Panels section and click `Add`.\n\n<img width=\"560\" height=\"315\" alt=\"huggingface-spaces-panel\" src=\"https://user-images.githubusercontent.com/7529846/214325606-99aa3af3-b284-4026-b423-d3d238797e12.png\">\n\nOnce you have added your Panel, click Edit to access to the Panel Options page and paste in the path of your Hugging Face Space e.g. `pytorch/ResNet`\n\n<img width=\"560\" height=\"315\" alt=\"Edit-HF-Space\" src=\"https://user-images.githubusercontent.com/7529846/214335868-c6f25dee-13db-4388-bcf5-65194f850b02.png\">\n\n## 4. Logging Model Inferences to Comet\n\n<iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/KZnpH7msPq0?start=176\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen></iframe>\n\n[](https://colab.research.google.com/github/comet-ml/comet-examples/blob/master/integrations/model-evaluation/gradio/notebooks/Logging_Model_Inferences_with_Comet_and_Gradio.ipynb)\n\nIn the previous examples, we demonstrated the various ways in which you can interact with a Gradio application through the Comet UI. Additionally, you can also log model inferences, such as SHAP plots, from your Gradio application to Comet.\n\nIn the following snippet, we're going to log inferences from a Text Generation model. We can persist an Experiment across multiple inference calls using Gradio's [State](https://www.gradio.app/docs/#state) object. This will allow you to log multiple inferences from a model to a single Experiment.\n\n```python\nimport comet_ml\nimport gradio as gr\nimport shap\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nif torch.cuda.is_available():\n device = \"cuda\"\nelse:\n device = \"cpu\"\n\nMODEL_NAME = \"gpt2\"\n\nmodel = AutoModelForCausalLM.from_pretrained(MODEL_NAME)\n\n# set model decoder to true\nmodel.config.is_decoder = True\n# set text-generation params under task_specific_params\nmodel.config.task_specific_params[\"text-generation\"] = {\n \"do_sample\": True,\n \"max_length\": 50,\n \"temperature\": 0.7,\n \"top_k\": 50,\n \"no_repeat_ngram_size\": 2,\n}\nmodel = model.to(device)\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\nexplainer = shap.Explainer(model, tokenizer)\n\n\ndef start_experiment():\n \"\"\"Returns an APIExperiment object that is thread safe\n and can be used to log inferences to a single Experiment\n \"\"\"\n try:\n api = comet_ml.API()\n workspace = api.get_default_workspace()\n project_name = comet_ml.config.get_config()[\"comet.project_name\"]\n\n experiment = comet_ml.APIExperiment(\n workspace=workspace, project_name=project_name\n )\n experiment.log_other(\"Created from\", \"gradio-inference\")\n\n message = f\"Started Experiment: [{experiment.name}]({experiment.url})\"\n\n return (experiment, message)\n\n except Exception as e:\n return None, None\n\n\ndef predict(text, state, message):\n experiment = state\n\n shap_values = explainer([text])\n plot = shap.plots.text(shap_values, display=False)\n\n if experiment is not None:\n experiment.log_other(\"message\", message)\n experiment.log_html(plot)\n\n return plot\n\n\nwith gr.Blocks() as demo:\n start_experiment_btn = gr.Button(\"Start New Experiment\")\n experiment_status = gr.Markdown()\n\n # Log a message to the Experiment to provide more context\n experiment_message = gr.Textbox(label=\"Experiment Message\")\n experiment = gr.State()\n\n input_text = gr.Textbox(label=\"Input Text\", lines=5, interactive=True)\n submit_btn = gr.Button(\"Submit\")\n\n output = gr.HTML(interactive=True)\n\n start_experiment_btn.click(\n start_experiment, outputs=[experiment, experiment_status]\n )\n submit_btn.click(\n predict, inputs=[input_text, experiment, experiment_message], outputs=[output]\n )\n```\n\nInferences from this snippet will be saved in the HTML tab of your experiment.\n\n<video width=\"560\" height=\"315\" controls>\n <source src=\"https://user-images.githubusercontent.com/7529846/214328610-466e5c81-4814-49b9-887c-065aca14dd30.mp4\"></source>\n</video>\n\n## Conclusion\n\nWe hope you found this guide useful and that it provides some inspiration to help you build awesome model evaluation workflows with Comet and Gradio.\n\n## How to contribute Gradio demos on HF spaces on the Comet organization\n\n- Create an account on Hugging Face [here](https://huggingface.co/join).\n- Add Gradio Demo under your username, see this [course](https://huggingface.co/course/chapter9/4?fw=pt) for setting up Gradio Demo on Hugging Face.\n- Request to join the Comet organization [here](https://huggingface.co/Comet).\n\n## Additional Resources\n\n- [Comet Documentation](https://www.comet.com/docs/v2/?utm_source=gradio&utm_medium=referral&utm_campaign=gradio-integration&utm_content=gradio-docs)\n", "tags": ["COMET", "SPACES"], "spaces": [], "url": "/guides/Gradio-and-Comet/", "contributor": "the Comet team"}} |