{"guide": {"name": "Gradio-and-ONNX-on-Hugging-Face", "category": "other-tutorials", "pretty_category": "Other Tutorials", "guide_index": null, "absolute_index": 57, "pretty_name": "Gradio And ONNX On Hugging Face", "content": "# Gradio and ONNX on Hugging Face\n\n\n\n\n\n## Introduction\n\nIn this Guide, we'll walk you through:\n\n- Introduction of ONNX, ONNX model zoo, Gradio, and Hugging Face Spaces\n- How to setup a Gradio demo for EfficientNet-Lite4\n- How to contribute your own Gradio demos for the ONNX organization on Hugging Face\n\nHere's an [example](https://onnx-efficientnet-lite4.hf.space/) of an ONNX model.\n\n## What is the ONNX Model Zoo?\n\nOpen Neural Network Exchange ([ONNX](https://onnx.ai/)) is an open standard format for representing machine learning models. ONNX is supported by a community of partners who have implemented it in many frameworks and tools. For example, if you have trained a model in TensorFlow or PyTorch, you can convert it to ONNX easily, and from there run it on a variety of devices using an engine/compiler like ONNX Runtime.\n\nThe [ONNX Model Zoo](https://github.com/onnx/models) is a collection of pre-trained, state-of-the-art models in the ONNX format contributed by community members. Accompanying each model are Jupyter notebooks for model training and running inference with the trained model. The notebooks are written in Python and include links to the training dataset as well as references to the original paper that describes the model architecture.\n\n## What are Hugging Face Spaces & Gradio?\n\n### Gradio\n\nGradio lets users demo their machine learning models as a web app all in python code. Gradio wraps a python function into a user interface and the demos can be launched inside jupyter notebooks, colab notebooks, as well as embedded in your own website and hosted on Hugging Face Spaces for free.\n\nGet started [here](https://gradio.app/getting_started)\n\n### Hugging Face Spaces\n\nHugging Face Spaces is a free hosting option for Gradio demos. Spaces comes with 3 SDK options: Gradio, Streamlit and Static HTML demos. Spaces can be public or private and the workflow is similar to github repos. There are over 2000+ spaces currently on Hugging Face. Learn more about spaces [here](https://huggingface.co/spaces/launch).\n\n### Hugging Face Models\n\nHugging Face Model Hub also supports ONNX models and ONNX models can be filtered through the [ONNX tag](https://huggingface.co/models?library=onnx&sort=downloads)\n\n## How did Hugging Face help the ONNX Model Zoo?\n\nThere are a lot of Jupyter notebooks in the ONNX Model Zoo for users to test models. Previously, users needed to download the models themselves and run those notebooks locally for testing. With Hugging Face, the testing process can be much simpler and more user-friendly. Users can easily try certain ONNX Model Zoo model on Hugging Face Spaces and run a quick demo powered by Gradio with ONNX Runtime, all on cloud without downloading anything locally. Note, there are various runtimes for ONNX, e.g., [ONNX Runtime](https://github.com/microsoft/onnxruntime), [MXNet](https://github.com/apache/incubator-mxnet).\n\n## What is the role of ONNX Runtime?\n\nONNX Runtime is a cross-platform inference and training machine-learning accelerator. It makes live Gradio demos with ONNX Model Zoo model on Hugging Face possible.\n\nONNX Runtime inference can enable faster customer experiences and lower costs, supporting models from deep learning frameworks such as PyTorch and TensorFlow/Keras as well as classical machine learning libraries such as scikit-learn, LightGBM, XGBoost, etc. ONNX Runtime is compatible with different hardware, drivers, and operating systems, and provides optimal performance by leveraging hardware accelerators where applicable alongside graph optimizations and transforms. For more information please see the [official website](https://onnxruntime.ai/).\n\n## Setting up a Gradio Demo for EfficientNet-Lite4\n\nEfficientNet-Lite 4 is the largest variant and most accurate of the set of EfficientNet-Lite models. It is an integer-only quantized model that produces the highest accuracy of all of the EfficientNet models. It achieves 80.4% ImageNet top-1 accuracy, while still running in real-time (e.g. 30ms/image) on a Pixel 4 CPU. To learn more read the [model card](https://github.com/onnx/models/tree/main/vision/classification/efficientnet-lite4)\n\nHere we walk through setting up a example demo for EfficientNet-Lite4 using Gradio\n\nFirst we import our dependencies and download and load the efficientnet-lite4 model from the onnx model zoo. Then load the labels from the labels_map.txt file. We then setup our preprocessing functions, load the model for inference, and setup the inference function. Finally, the inference function is wrapped into a gradio interface for a user to interact with. See the full code below.\n\n```python\nimport numpy as np\nimport math\nimport matplotlib.pyplot as plt\nimport cv2\nimport json\nimport gradio as gr\nfrom huggingface_hub import hf_hub_download\nfrom onnx import hub\nimport onnxruntime as ort\n\n# loads ONNX model from ONNX Model Zoo\nmodel = hub.load(\"efficientnet-lite4\")\n# loads the labels text file\nlabels = json.load(open(\"labels_map.txt\", \"r\"))\n\n# sets image file dimensions to 224x224 by resizing and cropping image from center\ndef pre_process_edgetpu(img, dims):\n    output_height, output_width, _ = dims\n    img = resize_with_aspectratio(img, output_height, output_width, inter_pol=cv2.INTER_LINEAR)\n    img = center_crop(img, output_height, output_width)\n    img = np.asarray(img, dtype='float32')\n    # converts jpg pixel value from [0 - 255] to float array [-1.0 - 1.0]\n    img -= [127.0, 127.0, 127.0]\n    img /= [128.0, 128.0, 128.0]\n    return img\n\n# resizes the image with a proportional scale\ndef resize_with_aspectratio(img, out_height, out_width, scale=87.5, inter_pol=cv2.INTER_LINEAR):\n    height, width, _ = img.shape\n    new_height = int(100. * out_height / scale)\n    new_width = int(100. * out_width / scale)\n    if height > width:\n        w = new_width\n        h = int(new_height * height / width)\n    else:\n        h = new_height\n        w = int(new_width * width / height)\n    img = cv2.resize(img, (w, h), interpolation=inter_pol)\n    return img\n\n# crops the image around the center based on given height and width\ndef center_crop(img, out_height, out_width):\n    height, width, _ = img.shape\n    left = int((width - out_width) / 2)\n    right = int((width + out_width) / 2)\n    top = int((height - out_height) / 2)\n    bottom = int((height + out_height) / 2)\n    img = img[top:bottom, left:right]\n    return img\n\n\nsess = ort.InferenceSession(model)\n\ndef inference(img):\n  img = cv2.imread(img)\n  img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)\n\n  img = pre_process_edgetpu(img, (224, 224, 3))\n\n  img_batch = np.expand_dims(img, axis=0)\n\n  results = sess.run([\"Softmax:0\"], {\"images:0\": img_batch})[0]\n  result = reversed(results[0].argsort()[-5:])\n  resultdic = {}\n  for r in result:\n      resultdic[labels[str(r)]] = float(results[0][r])\n  return resultdic\n\ntitle = \"EfficientNet-Lite4\"\ndescription = \"EfficientNet-Lite 4 is the largest variant and most accurate of the set of EfficientNet-Lite model. It is an integer-only quantized model that produces the highest accuracy of all of the EfficientNet models. It achieves 80.4% ImageNet top-1 accuracy, while still running in real-time (e.g. 30ms/image) on a Pixel 4 CPU.\"\nexamples = [['catonnx.jpg']]\ngr.Interface(inference, gr.Image(type=\"filepath\"), \"label\", title=title, description=description, examples=examples).launch()\n```\n\n## How to contribute Gradio demos on HF spaces using ONNX models\n\n- Add model to the [onnx model zoo](https://github.com/onnx/models/blob/main/.github/PULL_REQUEST_TEMPLATE.md)\n- Create an account on Hugging Face [here](https://huggingface.co/join).\n- See list of models left to add to ONNX organization, please refer to the table with the [Models list](https://github.com/onnx/models#models)\n- Add Gradio Demo under your username, see this [blog post](https://huggingface.co/blog/gradio-spaces) for setting up Gradio Demo on Hugging Face.\n- Request to join ONNX Organization [here](https://huggingface.co/onnx).\n- Once approved transfer model from your username to ONNX organization\n- Add a badge for model in model table, see examples in [Models list](https://github.com/onnx/models#models)\n", "tags": ["ONNX", "SPACES"], "spaces": ["https://huggingface.co/spaces/onnx/EfficientNet-Lite4"], "url": "/guides/Gradio-and-ONNX-on-Hugging-Face/", "contributor": "Gradio and the <a href=\"https://onnx.ai/\">ONNX</a> team"}}