+ + +
+ +[中文](https://docs.ultralytics.com/zh/) | [한국어](https://docs.ultralytics.com/ko/) | [日本語](https://docs.ultralytics.com/ja/) | [Русский](https://docs.ultralytics.com/ru/) | [Deutsch](https://docs.ultralytics.com/de/) | [Français](https://docs.ultralytics.com/fr/) | [Español](https://docs.ultralytics.com/es/) | [Português](https://docs.ultralytics.com/pt/) | [Türkçe](https://docs.ultralytics.com/tr/) | [Tiếng Việt](https://docs.ultralytics.com/vi/) | [العربية](https://docs.ultralytics.com/ar/)+ + +
+ +[中文](https://docs.ultralytics.com/zh/) | [한국어](https://docs.ultralytics.com/ko/) | [日本語](https://docs.ultralytics.com/ja/) | [Русский](https://docs.ultralytics.com/ru/) | [Deutsch](https://docs.ultralytics.com/de/) | [Français](https://docs.ultralytics.com/fr/) | [Español](https://docs.ultralytics.com/es/) | [Português](https://docs.ultralytics.com/pt/) | [Türkçe](https://docs.ultralytics.com/tr/) | [Tiếng Việt](https://docs.ultralytics.com/vi/) | [العربية](https://docs.ultralytics.com/ar/)
+
+
+
+ Watch: How to Train Image Classification Model using Caltech-256 Dataset with Ultralytics HUB
+
+
+
+
+ Watch: How to Train an Image Classification Model with CIFAR-10 Dataset using Ultralytics YOLOv8
+
+
+
+
+ Watch: How to do Image Classification on Fashion MNIST Dataset using Ultralytics YOLOv8
+
+
+
+
+ Watch: African Wildlife Animals Detection using Ultralytics YOLOv8
+
+
+
+
+ Watch: Brain Tumor Detection using Ultralytics HUB
+
+
+
+
+ Watch: Ultralytics COCO Dataset Overview
+
+
+
+
+ Watch: Ultralytics COCO Dataset Overview
+
+
+
+
+ Watch: YOLO World training workflow with LVIS dataset
+
+ +
+ +## Key Features + +- LVIS contains 160k images and 2M instance annotations for object detection, segmentation, and captioning tasks. +- The dataset comprises 1203 object categories, including common objects like cars, bicycles, and animals, as well as more specific categories such as umbrellas, handbags, and sports equipment. +- Annotations include object bounding boxes, segmentation masks, and captions for each image. +- LVIS provides standardized evaluation metrics like mean Average Precision (mAP) for object detection, and mean Average Recall (mAR) for segmentation tasks, making it suitable for comparing model performance. +- LVIS uses exactly the same images as [COCO](./coco.md) dataset, but with different splits and different annotations. + +## Dataset Structure + +The LVIS dataset is split into three subsets: + +1. **Train**: This subset contains 100k images for training object detection, segmentation, and captioning models. +2. **Val**: This subset has 20k images used for validation purposes during model training. +3. **Minival**: This subset is exactly the same as COCO val2017 set which has 5k images used for validation purposes during model training. +4. **Test**: This subset consists of 20k images used for testing and benchmarking the trained models. Ground truth annotations for this subset are not publicly available, and the results are submitted to the [LVIS evaluation server](https://eval.ai/web/challenges/challenge-page/675/overview) for performance evaluation. + +## Applications + +The LVIS dataset is widely used for training and evaluating deep learning models in object detection (such as YOLO, Faster R-CNN, and SSD), instance segmentation (such as Mask R-CNN). The dataset's diverse set of object categories, large number of annotated images, and standardized evaluation metrics make it an essential resource for computer vision researchers and practitioners. + +## Dataset YAML + +A YAML (Yet Another Markup Language) file is used to define the dataset configuration. It contains information about the dataset's paths, classes, and other relevant information. In the case of the LVIS dataset, the `lvis.yaml` file is maintained at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/lvis.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/lvis.yaml). + +!!! Example "ultralytics/cfg/datasets/lvis.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/lvis.yaml" + ``` + +## Usage + +To train a YOLOv8n model on the LVIS dataset for 100 epochs with an image size of 640, you can use the following code snippets. For a comprehensive list of available arguments, refer to the model [Training](../../modes/train.md) page. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="lvis.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo detect train data=lvis.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +## Sample Images and Annotations + +The LVIS dataset contains a diverse set of images with various object categories and complex scenes. Here are some examples of images from the dataset, along with their corresponding annotations: + +![LVIS Dataset sample image](https://github.com/ultralytics/ultralytics/assets/26833433/38cc033a-68b0-47f3-a5b8-4ef554362e40) + +- **Mosaiced Image**: This image demonstrates a training batch composed of mosaiced dataset images. Mosaicing is a technique used during training that combines multiple images into a single image to increase the variety of objects and scenes within each training batch. This helps improve the model's ability to generalize to different object sizes, aspect ratios, and contexts. + +The example showcases the variety and complexity of the images in the LVIS dataset and the benefits of using mosaicing during the training process. + +## Citations and Acknowledgments + +If you use the LVIS dataset in your research or development work, please cite the following paper: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @inproceedings{gupta2019lvis, + title={LVIS: A Dataset for Large Vocabulary Instance Segmentation}, + author={Gupta, Agrim and Dollar, Piotr and Girshick, Ross}, + booktitle={Proceedings of the {IEEE} Conference on Computer Vision and Pattern Recognition}, + year={2019} + } + ``` + +We would like to acknowledge the LVIS Consortium for creating and maintaining this valuable resource for the computer vision community. For more information about the LVIS dataset and its creators, visit the [LVIS dataset website](https://www.lvisdataset.org/). + +## FAQ + +### What is the LVIS dataset, and how is it used in computer vision? + +The [LVIS dataset](https://www.lvisdataset.org/) is a large-scale dataset with fine-grained vocabulary-level annotations developed by Facebook AI Research (FAIR). It is primarily used for object detection and instance segmentation, featuring over 1203 object categories and 2 million instance annotations. Researchers and practitioners use it to train and benchmark models like Ultralytics YOLO for advanced computer vision tasks. The dataset's extensive size and diversity make it an essential resource for pushing the boundaries of model performance in detection and segmentation. + +### How can I train a YOLOv8n model using the LVIS dataset? + +To train a YOLOv8n model on the LVIS dataset for 100 epochs with an image size of 640, follow the example below. This process utilizes Ultralytics' framework, which offers comprehensive training features. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="lvis.yaml", epochs=100, imgsz=640) + ``` + + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo detect train data=lvis.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +For detailed training configurations, refer to the [Training](../../modes/train.md) documentation. + +### How does the LVIS dataset differ from the COCO dataset? + +The images in the LVIS dataset are the same as those in the [COCO dataset](./coco.md), but the two differ in terms of splitting and annotations. LVIS provides a larger and more detailed vocabulary with 1203 object categories compared to COCO's 80 categories. Additionally, LVIS focuses on annotation completeness and diversity, aiming to push the limits of object detection and instance segmentation models by offering more nuanced and comprehensive data. + +### Why should I use Ultralytics YOLO for training on the LVIS dataset? + +Ultralytics YOLO models, including the latest YOLOv8, are optimized for real-time object detection with state-of-the-art accuracy and speed. They support a wide range of annotations, such as the fine-grained ones provided by the LVIS dataset, making them ideal for advanced computer vision applications. Moreover, Ultralytics offers seamless integration with various [training](../../modes/train.md), [validation](../../modes/val.md), and [prediction](../../modes/predict.md) modes, ensuring efficient model development and deployment. + +### Can I see some sample annotations from the LVIS dataset? + +Yes, the LVIS dataset includes a variety of images with diverse object categories and complex scenes. Here is an example of a sample image along with its annotations: + +![LVIS Dataset sample image](https://github.com/ultralytics/ultralytics/assets/26833433/38cc033a-68b0-47f3-a5b8-4ef554362e40) + +This mosaiced image demonstrates a training batch composed of multiple dataset images combined into one. Mosaicing increases the variety of objects and scenes within each training batch, enhancing the model's ability to generalize across different contexts. For more details on the LVIS dataset, explore the [LVIS dataset documentation](#key-features). diff --git a/ultralytics/docs/en/datasets/detect/objects365.md b/ultralytics/docs/en/datasets/detect/objects365.md new file mode 100644 index 0000000000000000000000000000000000000000..7bef28bf8d0eceb4901209c7d8d68c22810239e2 --- /dev/null +++ b/ultralytics/docs/en/datasets/detect/objects365.md @@ -0,0 +1,140 @@ +--- +comments: true +description: Explore the Objects365 Dataset with 2M images and 30M bounding boxes across 365 categories. Enhance your object detection models with diverse, high-quality data. +keywords: Objects365 dataset, object detection, machine learning, deep learning, computer vision, annotated images, bounding boxes, YOLOv8, high-resolution images, dataset configuration +--- + +# Objects365 Dataset + +The [Objects365](https://www.objects365.org/) dataset is a large-scale, high-quality dataset designed to foster object detection research with a focus on diverse objects in the wild. Created by a team of [Megvii](https://en.megvii.com/) researchers, the dataset offers a wide range of high-resolution images with a comprehensive set of annotated bounding boxes covering 365 object categories. + +## Key Features + +- Objects365 contains 365 object categories, with 2 million images and over 30 million bounding boxes. +- The dataset includes diverse objects in various scenarios, providing a rich and challenging benchmark for object detection tasks. +- Annotations include bounding boxes for objects, making it suitable for training and evaluating object detection models. +- Objects365 pre-trained models significantly outperform ImageNet pre-trained models, leading to better generalization on various tasks. + +## Dataset Structure + +The Objects365 dataset is organized into a single set of images with corresponding annotations: + +- **Images**: The dataset includes 2 million high-resolution images, each containing a variety of objects across 365 categories. +- **Annotations**: The images are annotated with over 30 million bounding boxes, providing comprehensive ground truth information for object detection tasks. + +## Applications + +The Objects365 dataset is widely used for training and evaluating deep learning models in object detection tasks. The dataset's diverse set of object categories and high-quality annotations make it a valuable resource for researchers and practitioners in the field of computer vision. + +## Dataset YAML + +A YAML (Yet Another Markup Language) file is used to define the dataset configuration. It contains information about the dataset's paths, classes, and other relevant information. For the case of the Objects365 Dataset, the `Objects365.yaml` file is maintained at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/Objects365.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/Objects365.yaml). + +!!! Example "ultralytics/cfg/datasets/Objects365.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/Objects365.yaml" + ``` + +## Usage + +To train a YOLOv8n model on the Objects365 dataset for 100 epochs with an image size of 640, you can use the following code snippets. For a comprehensive list of available arguments, refer to the model [Training](../../modes/train.md) page. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="Objects365.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo detect train data=Objects365.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +## Sample Data and Annotations + +The Objects365 dataset contains a diverse set of high-resolution images with objects from 365 categories, providing rich context for object detection tasks. Here are some examples of the images in the dataset: + +![Dataset sample image](https://user-images.githubusercontent.com/26833433/238215467-caf757dd-0b87-4b0d-bb19-d94a547f7fbf.jpg) + +- **Objects365**: This image demonstrates an example of object detection, where objects are annotated with bounding boxes. The dataset provides a wide range of images to facilitate the development of models for this task. + +The example showcases the variety and complexity of the data in the Objects365 dataset and highlights the importance of accurate object detection for computer vision applications. + +## Citations and Acknowledgments + +If you use the Objects365 dataset in your research or development work, please cite the following paper: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @inproceedings{shao2019objects365, + title={Objects365: A Large-scale, High-quality Dataset for Object Detection}, + author={Shao, Shuai and Li, Zeming and Zhang, Tianyuan and Peng, Chao and Yu, Gang and Li, Jing and Zhang, Xiangyu and Sun, Jian}, + booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision}, + pages={8425--8434}, + year={2019} + } + ``` + +We would like to acknowledge the team of researchers who created and maintain the Objects365 dataset as a valuable resource for the computer vision research community. For more information about the Objects365 dataset and its creators, visit the [Objects365 dataset website](https://www.objects365.org/). + +## FAQ + +### What is the Objects365 dataset used for? + +The [Objects365 dataset](https://www.objects365.org/) is designed for object detection tasks in machine learning and computer vision. It provides a large-scale, high-quality dataset with 2 million annotated images and 30 million bounding boxes across 365 categories. Leveraging such a diverse dataset helps improve the performance and generalization of object detection models, making it invaluable for research and development in the field. + +### How can I train a YOLOv8 model on the Objects365 dataset? + +To train a YOLOv8n model using the Objects365 dataset for 100 epochs with an image size of 640, follow these instructions: + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="Objects365.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo detect train data=Objects365.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +Refer to the [Training](../../modes/train.md) page for a comprehensive list of available arguments. + +### Why should I use the Objects365 dataset for my object detection projects? + +The Objects365 dataset offers several advantages for object detection tasks: +1. **Diversity**: It includes 2 million images with objects in diverse scenarios, covering 365 categories. +2. **High-quality Annotations**: Over 30 million bounding boxes provide comprehensive ground truth data. +3. **Performance**: Models pre-trained on Objects365 significantly outperform those trained on datasets like ImageNet, leading to better generalization. + +### Where can I find the YAML configuration file for the Objects365 dataset? + +The YAML configuration file for the Objects365 dataset is available at [Objects365.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/Objects365.yaml). This file contains essential information such as dataset paths and class labels, crucial for setting up your training environment. + +### How does the dataset structure of Objects365 enhance object detection modeling? + +The [Objects365 dataset](https://www.objects365.org/) is organized with 2 million high-resolution images and comprehensive annotations of over 30 million bounding boxes. This structure ensures a robust dataset for training deep learning models in object detection, offering a wide variety of objects and scenarios. Such diversity and volume help in developing models that are more accurate and capable of generalizing well to real-world applications. For more details on the dataset structure, refer to the [Dataset YAML](#dataset-yaml) section. diff --git a/ultralytics/docs/en/datasets/detect/open-images-v7.md b/ultralytics/docs/en/datasets/detect/open-images-v7.md new file mode 100644 index 0000000000000000000000000000000000000000..08b7a467b52184ac1de9b9ce7fd72e52a7415fb3 --- /dev/null +++ b/ultralytics/docs/en/datasets/detect/open-images-v7.md @@ -0,0 +1,199 @@ +--- +comments: true +description: Explore the comprehensive Open Images V7 dataset by Google. Learn about its annotations, applications, and use YOLOv8 pretrained models for computer vision tasks. +keywords: Open Images V7, Google dataset, computer vision, YOLOv8 models, object detection, image segmentation, visual relationships, AI research, Ultralytics +--- + +# Open Images V7 Dataset + +[Open Images V7](https://storage.googleapis.com/openimages/web/index.html) is a versatile and expansive dataset championed by Google. Aimed at propelling research in the realm of computer vision, it boasts a vast collection of images annotated with a plethora of data, including image-level labels, object bounding boxes, object segmentation masks, visual relationships, and localized narratives. + +
+
+
+
+ Watch: Object Detection using OpenImagesV7 Pretrained Model
+
+ +
+ +## Key Features + +- Includes 100 datasets across seven domains: Aerial, Video games, Microscopic, Underwater, Documents, Electromagnetic, and Real World. +- The benchmark comprises 224,714 images across 805 classes, thanks to over 11,170 hours of labeling efforts. +- All images are resized to 640x640 pixels, with a focus on eliminating class ambiguity and filtering out underrepresented classes. +- Annotations include bounding boxes for objects, making it suitable for [training](../../modes/train.md) and evaluating object detection models. + +## Dataset Structure + +The Roboflow 100 dataset is organized into seven categories, each with a distinct set of datasets, images, and classes: + +- **Aerial**: Consists of 7 datasets with a total of 9,683 images, covering 24 distinct classes. +- **Video Games**: Includes 7 datasets, featuring 11,579 images across 88 classes. +- **Microscopic**: Comprises 11 datasets with 13,378 images, spanning 28 classes. +- **Underwater**: Contains 5 datasets, encompassing 18,003 images in 39 classes. +- **Documents**: Consists of 8 datasets with 24,813 images, divided into 90 classes. +- **Electromagnetic**: Made up of 12 datasets, totaling 36,381 images in 41 classes. +- **Real World**: The largest category with 50 datasets, offering 110,615 images across 495 classes. + +This structure enables a diverse and extensive testing ground for object detection models, reflecting real-world application scenarios. + +## Benchmarking + +Dataset benchmarking evaluates machine learning model performance on specific datasets using standardized metrics like accuracy, mean average precision and F1-score. + +!!! Tip "Benchmarking" + + Benchmarking results will be stored in "ultralytics-benchmarks/evaluation.txt" + +!!! Example "Benchmarking example" + + === "Python" + + ```python + import os + import shutil + from pathlib import Path + + from ultralytics.utils.benchmarks import RF100Benchmark + + # Initialize RF100Benchmark and set API key + benchmark = RF100Benchmark() + benchmark.set_key(api_key="YOUR_ROBOFLOW_API_KEY") + + # Parse dataset and define file paths + names, cfg_yamls = benchmark.parse_dataset() + val_log_file = Path("ultralytics-benchmarks") / "validation.txt" + eval_log_file = Path("ultralytics-benchmarks") / "evaluation.txt" + + # Run benchmarks on each dataset in RF100 + for ind, path in enumerate(cfg_yamls): + path = Path(path) + if path.exists(): + # Fix YAML file and run training + benchmark.fix_yaml(str(path)) + os.system(f"yolo detect train data={path} model=yolov8s.pt epochs=1 batch=16") + + # Run validation and evaluate + os.system(f"yolo detect val data={path} model=runs/detect/train/weights/best.pt > {val_log_file} 2>&1") + benchmark.evaluate(str(path), str(val_log_file), str(eval_log_file), ind) + + # Remove the 'runs' directory + runs_dir = Path.cwd() / "runs" + shutil.rmtree(runs_dir) + else: + print("YAML file path does not exist") + continue + + print("RF100 Benchmarking completed!") + ``` + +## Applications + +Roboflow 100 is invaluable for various applications related to computer vision and deep learning. Researchers and engineers can use this benchmark to: + +- Evaluate the performance of object detection models in a multi-domain context. +- Test the adaptability of models to real-world scenarios beyond common object recognition. +- Benchmark the capabilities of object detection models across diverse datasets, including those in healthcare, aerial imagery, and video games. + +For more ideas and inspiration on real-world applications, be sure to check out [our guides on real-world projects](../../guides/index.md). + +## Usage + +The Roboflow 100 dataset is available on both [GitHub](https://github.com/roboflow/roboflow-100-benchmark) and [Roboflow Universe](https://universe.roboflow.com/roboflow-100). + +You can access it directly from the Roboflow 100 GitHub repository. In addition, on Roboflow Universe, you have the flexibility to download individual datasets by simply clicking the export button within each dataset. + +## Sample Data and Annotations + +Roboflow 100 consists of datasets with diverse images and videos captured from various angles and domains. Here's a look at examples of annotated images in the RF100 benchmark. + ++ +
+ +The diversity in the Roboflow 100 benchmark that can be seen above is a significant advancement from traditional benchmarks which often focus on optimizing a single metric within a limited domain. + +## Citations and Acknowledgments + +If you use the Roboflow 100 dataset in your research or development work, please cite the following paper: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @misc{2211.13523, + Author = {Floriana Ciaglia and Francesco Saverio Zuppichini and Paul Guerrie and Mark McQuade and Jacob Solawetz}, + Title = {Roboflow 100: A Rich, Multi-Domain Object Detection Benchmark}, + Eprint = {arXiv:2211.13523}, + } + ``` + +Our thanks go to the Roboflow team and all the contributors for their hard work in creating and sustaining the Roboflow 100 dataset. + +If you are interested in exploring more datasets to enhance your object detection and machine learning projects, feel free to visit [our comprehensive dataset collection](../index.md). + +## FAQ + +### What is the Roboflow 100 dataset, and why is it significant for object detection? + +The **Roboflow 100** dataset, developed by [Roboflow](https://roboflow.com/?ref=ultralytics) and sponsored by Intel, is a crucial [object detection](../../tasks/detect.md) benchmark. It features 100 diverse datasets from over 90,000 public datasets, covering domains such as healthcare, aerial imagery, and video games. This diversity ensures that models can adapt to various real-world scenarios, enhancing their robustness and performance. + +### How can I use the Roboflow 100 dataset for benchmarking my object detection models? + +To use the Roboflow 100 dataset for benchmarking, you can implement the RF100Benchmark class from the Ultralytics library. Here's a brief example: + +!!! Example "Benchmarking example" + + === "Python" + + ```python + import os + import shutil + from pathlib import Path + + from ultralytics.utils.benchmarks import RF100Benchmark + + # Initialize RF100Benchmark and set API key + benchmark = RF100Benchmark() + benchmark.set_key(api_key="YOUR_ROBOFLOW_API_KEY") + + # Parse dataset and define file paths + names, cfg_yamls = benchmark.parse_dataset() + val_log_file = Path("ultralytics-benchmarks") / "validation.txt" + eval_log_file = Path("ultralytics-benchmarks") / "evaluation.txt" + + # Run benchmarks on each dataset in RF100 + for ind, path in enumerate(cfg_yamls): + path = Path(path) + if path.exists(): + # Fix YAML file and run training + benchmark.fix_yaml(str(path)) + os.system(f"yolo detect train data={path} model=yolov8s.pt epochs=1 batch=16") + + # Run validation and evaluate + os.system(f"yolo detect val data={path} model=runs/detect/train/weights/best.pt > {val_log_file} 2>&1") + benchmark.evaluate(str(path), str(val_log_file), str(eval_log_file), ind) + + # Remove 'runs' directory + runs_dir = Path.cwd() / "runs" + shutil.rmtree(runs_dir) + else: + print("YAML file path does not exist") + continue + + print("RF100 Benchmarking completed!") + ``` + +### Which domains are covered by the Roboflow 100 dataset? + +The **Roboflow 100** dataset spans seven domains, each providing unique challenges and applications for object detection models: + +1. **Aerial**: 7 datasets, 9,683 images, 24 classes +2. **Video Games**: 7 datasets, 11,579 images, 88 classes +3. **Microscopic**: 11 datasets, 13,378 images, 28 classes +4. **Underwater**: 5 datasets, 18,003 images, 39 classes +5. **Documents**: 8 datasets, 24,813 images, 90 classes +6. **Electromagnetic**: 12 datasets, 36,381 images, 41 classes +7. **Real World**: 50 datasets, 110,615 images, 495 classes + +This setup allows for extensive and varied testing of models across different real-world applications. + +### How do I access and download the Roboflow 100 dataset? + +The **Roboflow 100** dataset is accessible on [GitHub](https://github.com/roboflow/roboflow-100-benchmark) and [Roboflow Universe](https://universe.roboflow.com/roboflow-100). You can download the entire dataset from GitHub or select individual datasets on Roboflow Universe using the export button. + +### What should I include when citing the Roboflow 100 dataset in my research? + +When using the Roboflow 100 dataset in your research, ensure to properly cite it. Here is the recommended citation: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @misc{2211.13523, + Author = {Floriana Ciaglia and Francesco Saverio Zuppichini and Paul Guerrie and Mark McQuade and Jacob Solawetz}, + Title = {Roboflow 100: A Rich, Multi-Domain Object Detection Benchmark}, + Eprint = {arXiv:2211.13523}, + } + ``` + +For more details, you can refer to our [comprehensive dataset collection](../index.md). diff --git a/ultralytics/docs/en/datasets/detect/signature.md b/ultralytics/docs/en/datasets/detect/signature.md new file mode 100644 index 0000000000000000000000000000000000000000..4d25650afa10d735386b283423832bc365d70ff6 --- /dev/null +++ b/ultralytics/docs/en/datasets/detect/signature.md @@ -0,0 +1,170 @@ +--- +comments: true +description: Discover the Signature Detection Dataset for training models to identify and verify human signatures in various documents. Perfect for document verification and fraud prevention. +keywords: Signature Detection Dataset, document verification, fraud detection, computer vision, YOLOv8, Ultralytics, annotated signatures, training dataset +--- + +# Signature Detection Dataset + +This dataset focuses on detecting human written signatures within documents. It includes a variety of document types with annotated signatures, providing valuable insights for applications in document verification and fraud detection. Essential for training computer vision algorithms, this dataset aids in identifying signatures in various document formats, supporting research and practical applications in document analysis. + +## Dataset Structure + +The signature detection dataset is split into three subsets: + +- **Training set**: Contains 143 images, each with corresponding annotations. +- **Validation set**: Includes 35 images, each with paired annotations. + +## Applications + +This dataset can be applied in various computer vision tasks such as object detection, object tracking, and document analysis. Specifically, it can be used to train and evaluate models for identifying signatures in documents, which can have applications in document verification, fraud detection, and archival research. Additionally, it can serve as a valuable resource for educational purposes, enabling students and researchers to study and understand the characteristics and behaviors of signatures in different document types. + +## Dataset YAML + +A YAML (Yet Another Markup Language) file defines the dataset configuration, including paths and classes information. For the signature detection dataset, the `signature.yaml` file is located at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/signature.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/signature.yaml). + +!!! Example "ultralytics/cfg/datasets/signature.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/signature.yaml" + ``` + +## Usage + +To train a YOLOv8n model on the signature detection dataset for 100 epochs with an image size of 640, use the provided code samples. For a comprehensive list of available parameters, refer to the model's [Training](../../modes/train.md) page. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="signature.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo detect train data=signature.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +!!! Example "Inference Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("path/to/best.pt") # load a signature-detection fine-tuned model + + # Inference using the model + results = model.predict("https://ultralytics.com/assets/signature-s.mp4", conf=0.75) + ``` + + === "CLI" + + ```bash + # Start prediction with a finetuned *.pt model + yolo detect predict model='path/to/best.pt' imgsz=640 source="https://ultralytics.com/assets/signature-s.mp4" conf=0.75 + ``` + +## Sample Images and Annotations + +The signature detection dataset comprises a wide variety of images showcasing different document types and annotated signatures. Below are examples of images from the dataset, each accompanied by its corresponding annotations. + +![Signature detection dataset sample image](https://github.com/RizwanMunawar/RizwanMunawar/assets/62513924/88a453da-3110-4835-9ae4-97bfb8b19046) + +- **Mosaiced Image**: Here, we present a training batch consisting of mosaiced dataset images. Mosaicing, a training technique, combines multiple images into one, enriching batch diversity. This method helps enhance the model's ability to generalize across different signature sizes, aspect ratios, and contexts. + +This example illustrates the variety and complexity of images in the signature Detection Dataset, emphasizing the benefits of including mosaicing during the training process. + +## Citations and Acknowledgments + +The dataset has been released available under the [AGPL-3.0 License](https://github.com/ultralytics/ultralytics/blob/main/LICENSE). + +## FAQ + +### What is the Signature Detection Dataset, and how can it be used? + +The Signature Detection Dataset is a collection of annotated images aimed at detecting human signatures within various document types. It can be applied in computer vision tasks such as object detection and tracking, primarily for document verification, fraud detection, and archival research. This dataset helps train models to recognize signatures in different contexts, making it valuable for both research and practical applications. + +### How do I train a YOLOv8n model on the Signature Detection Dataset? + +To train a YOLOv8n model on the Signature Detection Dataset, follow these steps: + +1. Download the `signature.yaml` dataset configuration file from [signature.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/signature.yaml). +2. Use the following Python script or CLI command to start training: + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a pretrained model + model = YOLO("yolov8n.pt") + + # Train the model + results = model.train(data="signature.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + yolo detect train data=signature.yaml model=yolov8n.pt epochs=100 imgsz=640 + ``` + +For more details, refer to the [Training](../../modes/train.md) page. + +### What are the main applications of the Signature Detection Dataset? + +The Signature Detection Dataset can be used for: + +1. **Document Verification**: Automatically verifying the presence and authenticity of human signatures in documents. +2. **Fraud Detection**: Identifying forged or fraudulent signatures in legal and financial documents. +3. **Archival Research**: Assisting historians and archivists in the digital analysis and cataloging of historical documents. +4. **Education**: Supporting academic research and teaching in the fields of computer vision and machine learning. + +### How can I perform inference using a model trained on the Signature Detection Dataset? + +To perform inference using a model trained on the Signature Detection Dataset, follow these steps: + +1. Load your fine-tuned model. +2. Use the below Python script or CLI command to perform inference: + +!!! Example "Inference Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the fine-tuned model + model = YOLO("path/to/best.pt") + + # Perform inference + results = model.predict("https://ultralytics.com/assets/signature-s.mp4", conf=0.75) + ``` + + === "CLI" + + ```bash + yolo detect predict model='path/to/best.pt' imgsz=640 source="https://ultralytics.com/assets/signature-s.mp4" conf=0.75 + ``` + +### What is the structure of the Signature Detection Dataset, and where can I find more information? + +The Signature Detection Dataset is divided into two subsets: + +- **Training Set**: Contains 143 images with annotations. +- **Validation Set**: Includes 35 images with annotations. + +For detailed information, you can refer to the [Dataset Structure](#dataset-structure) section. Additionally, view the complete dataset configuration in the `signature.yaml` file located at [signature.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/signature.yaml). diff --git a/ultralytics/docs/en/datasets/detect/sku-110k.md b/ultralytics/docs/en/datasets/detect/sku-110k.md new file mode 100644 index 0000000000000000000000000000000000000000..3e2dab317238d0e28735f9cd4155769b4413c37f --- /dev/null +++ b/ultralytics/docs/en/datasets/detect/sku-110k.md @@ -0,0 +1,181 @@ +--- +comments: true +description: Explore the SKU-110k dataset of densely packed retail shelf images, perfect for training and evaluating deep learning models in object detection tasks. +keywords: SKU-110k, dataset, object detection, retail shelf images, deep learning, computer vision, model training +--- + +# SKU-110k Dataset + +The [SKU-110k](https://github.com/eg4000/SKU110K_CVPR19) dataset is a collection of densely packed retail shelf images, designed to support research in object detection tasks. Developed by Eran Goldman et al., the dataset contains over 110,000 unique store keeping unit (SKU) categories with densely packed objects, often looking similar or even identical, positioned in close proximity. + +
+
+
+
+ Watch: How to Train YOLOv10 on SKU-110k Dataset using Ultralytics | Retail Dataset
+
+
+
+
+ Watch: How to Train Ultralytics YOLO Models on the VisDrone Dataset for Drone Image Analysis
+
+
+
+
+ Watch: Ultralytics Explorer API Overview
+
+ +
+ +
+
+
+
+ Watch: Ultralytics Explorer Dashboard Overview
+
+ +
+ +On performing similarity search, you should see a similar result: ++ +
+ +## Ask AI + +This allows you to write how you want to filter your dataset using natural language. You don't have to be proficient in writing SQL queries. Our AI powered query generator will automatically do that under the hood. For example - you can say - "show me 100 images with exactly one person and 2 dogs. There can be other objects too" and it'll internally generate the query and show you those results. Here's an example output when asked to "Show 10 images with exactly 5 persons" and you'll see a result like this: ++ +
+ +Note: This works using LLMs under the hood so the results are probabilistic and might get things wrong sometimes + +## Run SQL queries on your CV datasets + +You can run SQL queries on your dataset to filter it. It also works if you only provide the WHERE clause. Example SQL query would show only the images that have at least one 1 person and 1 dog in them: + +```sql +WHERE labels LIKE '%person%' AND labels LIKE '%dog%' +``` + ++ +
+ +This is a Demo build using the Explorer API. You can use the API to build your own exploratory notebooks or scripts to get insights into your datasets. Learn more about the Explorer API [here](api.md). + +## FAQ + +### What is Ultralytics Explorer GUI and how do I install it? + +Ultralytics Explorer GUI is a powerful interface that unlocks advanced data exploration capabilities using the [Ultralytics Explorer API](api.md). It allows you to run semantic/vector similarity search, SQL queries, and natural language queries using the Ask AI feature powered by Large Language Models (LLMs). + +To install the Explorer GUI, you can use pip: + +```bash +pip install ultralytics[explorer] +``` + +Note: To use the Ask AI feature, you'll need to set the OpenAI API key: `yolo settings openai_api_key="..."`. + +### How does the semantic search feature in Ultralytics Explorer GUI work? + +The semantic search feature in Ultralytics Explorer GUI allows you to find images similar to a given image based on their embeddings. This technique is useful for identifying and exploring images that share visual similarities. To use this feature, select one or more images in the UI and execute a search for similar images. The result will display images that closely resemble the selected ones, facilitating efficient dataset exploration and anomaly detection. + +Learn more about semantic search and other features by visiting the [Feature Overview](#vector-semantic-similarity-search) section. + +### Can I use natural language to filter datasets in Ultralytics Explorer GUI? + +Yes, with the Ask AI feature powered by large language models (LLMs), you can filter your datasets using natural language queries. You don't need to be proficient in SQL. For instance, you can ask "Show me 100 images with exactly one person and 2 dogs. There can be other objects too," and the AI will generate the appropriate query under the hood to deliver the desired results. + +See an example of a natural language query [here](#ask-ai). + +### How do I run SQL queries on datasets using Ultralytics Explorer GUI? + +Ultralytics Explorer GUI allows you to run SQL queries directly on your dataset to filter and manage data efficiently. To run a query, navigate to the SQL query section in the GUI and write your query. For example, to show images with at least one person and one dog, you could use: + +```sql +WHERE labels LIKE '%person%' AND labels LIKE '%dog%' +``` + +You can also provide only the WHERE clause, making the querying process more flexible. + +For more details, refer to the [SQL Queries Section](#run-sql-queries-on-your-cv-datasets). + +### What are the benefits of using Ultralytics Explorer GUI for data exploration? + +Ultralytics Explorer GUI enhances data exploration with features like semantic search, SQL querying, and natural language interactions through the Ask AI feature. These capabilities allow users to: +- Efficiently find visually similar images. +- Filter datasets using complex SQL queries. +- Utilize AI to perform natural language searches, eliminating the need for advanced SQL expertise. + +These features make it a versatile tool for developers, researchers, and data scientists looking to gain deeper insights into their datasets. + +Explore more about these features in the [Explorer GUI Documentation](#explorer-gui). diff --git a/ultralytics/docs/en/datasets/explorer/explorer.ipynb b/ultralytics/docs/en/datasets/explorer/explorer.ipynb new file mode 100644 index 0000000000000000000000000000000000000000..0980d878f53ba8fb1bf87110adf5a1ab9aa2b546 --- /dev/null +++ b/ultralytics/docs/en/datasets/explorer/explorer.ipynb @@ -0,0 +1,601 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "aa923c26-81c8-4565-9277-1cb686e3702e", + "metadata": { + "id": "aa923c26-81c8-4565-9277-1cb686e3702e" + }, + "source": [ + "# VOC Exploration Example\n", + "\n", + "\n", + " \n", + "
\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9dbfe7d0-8613-4529-adb6-6e0632d7cce7", + "metadata": { + "id": "9dbfe7d0-8613-4529-adb6-6e0632d7cce7" + }, + "outputs": [], + "source": [ + "exp.plot_similar(idx=6500, limit=20)\n", + "#exp.plot_similar(idx=[100,101], limit=10) # Can also pass list of idxs or imgs\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "260e09bf-4960-4089-a676-cb0e76ff3c0d", + "metadata": { + "id": "260e09bf-4960-4089-a676-cb0e76ff3c0d" + }, + "outputs": [], + "source": [ + "exp.plot_similar(img=\"https://ultralytics.com/images/bus.jpg\", limit=10, labels=False) # Can also pass any external images\n" + ] + }, + { + "cell_type": "markdown", + "id": "faa0b7a7-6318-40e4-b0f4-45a8113bdc3a", + "metadata": { + "id": "faa0b7a7-6318-40e4-b0f4-45a8113bdc3a" + }, + "source": [ + "\n", + "\n", + "\n", + "
" + ] + }, + { + "cell_type": "markdown", + "id": "0cea63f1-71f1-46da-af2b-b1b7d8f73553", + "metadata": { + "id": "0cea63f1-71f1-46da-af2b-b1b7d8f73553" + }, + "source": [ + "## 2. Ask AI: Search or filter with Natural Language\n", + "You can prompt the Explorer object with the kind of data points you want to see and it'll try to return a dataframe with those. Because it is powered by LLMs, it doesn't always get it right. In that case, it'll return None.\n", + "\n", + "\n", + "\n", + "
\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "92fb92ac-7f76-465a-a9ba-ea7492498d9c", + "metadata": { + "id": "92fb92ac-7f76-465a-a9ba-ea7492498d9c" + }, + "outputs": [], + "source": [ + "df = exp.ask_ai(\"show me images containing more than 10 objects with at least 2 persons\")\n", + "df.head(5)" + ] + }, + { + "cell_type": "markdown", + "id": "f2a7d26e-0ce5-4578-ad1a-b1253805280f", + "metadata": { + "id": "f2a7d26e-0ce5-4578-ad1a-b1253805280f" + }, + "source": [ + "for plotting these results you can use `plot_query_result` util\n", + "Example:\n", + "```\n", + "plt = plot_query_result(exp.ask_ai(\"show me 10 images containing exactly 2 persons\"))\n", + "Image.fromarray(plt)\n", + "```\n", + "\n", + " \n", + "\n", + "
" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b1cfab84-9835-4da0-8e9a-42b30cf84511", + "metadata": { + "id": "b1cfab84-9835-4da0-8e9a-42b30cf84511" + }, + "outputs": [], + "source": [ + "# plot\n", + "from ultralytics.data.explorer import plot_query_result\n", + "from PIL import Image\n", + "\n", + "plt = plot_query_result(exp.ask_ai(\"show me 10 images containing exactly 2 persons\"))\n", + "Image.fromarray(plt)" + ] + }, + { + "cell_type": "markdown", + "id": "35315ae6-d827-40e4-8813-279f97a83b34", + "metadata": { + "id": "35315ae6-d827-40e4-8813-279f97a83b34" + }, + "source": [ + "## 3. Run SQL queries on your Dataset!\n", + "Sometimes you might want to investigate a certain type of entries in your dataset. For this Explorer allows you to execute SQL queries.\n", + "It accepts either of the formats:\n", + "- Queries beginning with \"WHERE\" will automatically select all columns. This can be thought of as a short-hand query\n", + "- You can also write full queries where you can specify which columns to select\n", + "\n", + "This can be used to investigate model performance and specific data points. For example:\n", + "- let's say your model struggles on images that have humans and dogs. You can write a query like this to select the points that have at least 2 humans AND at least one dog.\n", + "\n", + "You can combine SQL query and semantic search to filter down to specific type of results\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8cd1072f-3100-4331-a0e3-4e2f6b1005bf", + "metadata": { + "id": "8cd1072f-3100-4331-a0e3-4e2f6b1005bf" + }, + "outputs": [], + "source": [ + "table = exp.sql_query(\"WHERE labels LIKE '%person, person%' AND labels LIKE '%dog%' LIMIT 10\")\n", + "table" + ] + }, + { + "cell_type": "markdown", + "id": "debf8a00-c9f6-448b-bd3b-454cf62f39ab", + "metadata": { + "id": "debf8a00-c9f6-448b-bd3b-454cf62f39ab" + }, + "source": [ + "Just like similarity search, you also get a util to directly plot the sql queries using `exp.plot_sql_query`\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "18b977e7-d048-4b22-b8c4-084a03b04f23", + "metadata": { + "id": "18b977e7-d048-4b22-b8c4-084a03b04f23" + }, + "outputs": [], + "source": [ + "exp.plot_sql_query(\"WHERE labels LIKE '%person, person%' AND labels LIKE '%dog%' LIMIT 10\", labels=True)" + ] + }, + { + "cell_type": "markdown", + "id": "f26804c5-840b-4fd1-987f-e362f29e3e06", + "metadata": { + "id": "f26804c5-840b-4fd1-987f-e362f29e3e06" + }, + "source": [ + "## 3. Working with embeddings Table (Advanced)\n", + "Explorer works on [LanceDB](https://lancedb.github.io/lancedb/) tables internally. You can access this table directly, using `Explorer.table` object and run raw queries, push down pre and post filters, etc." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ea69260a-3407-40c9-9f42-8b34a6e6af7a", + "metadata": { + "id": "ea69260a-3407-40c9-9f42-8b34a6e6af7a" + }, + "outputs": [], + "source": [ + "table = exp.table\n", + "table.schema" + ] + }, + { + "cell_type": "markdown", + "id": "238db292-8610-40b3-9af7-dfd6be174892", + "metadata": { + "id": "238db292-8610-40b3-9af7-dfd6be174892" + }, + "source": [ + "### Run raw queries\n", + "Vector Search finds the nearest vectors from the database. In a recommendation system or search engine, you can find similar products from the one you searched. In LLM and other AI applications, each data point can be presented by the embeddings generated from some models, it returns the most relevant features.\n", + "\n", + "A search in high-dimensional vector space, is to find K-Nearest-Neighbors (KNN) of the query vector.\n", + "\n", + "Metric\n", + "In LanceDB, a Metric is the way to describe the distance between a pair of vectors. Currently, it supports the following metrics:\n", + "- L2\n", + "- Cosine\n", + "- Dot\n", + "Explorer's similarity search uses L2 by default. You can run queries on tables directly, or use the lance format to build custom utilities to manage datasets. More details on available LanceDB table ops in the [docs](https://lancedb.github.io/lancedb/)\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d74430fe-5aee-45a1-8863-3f2c31338792", + "metadata": { + "id": "d74430fe-5aee-45a1-8863-3f2c31338792" + }, + "outputs": [], + "source": [ + "dummy_img_embedding = [i for i in range(256)]\n", + "table.search(dummy_img_embedding).limit(5).to_pandas()" + ] + }, + { + "cell_type": "markdown", + "id": "587486b4-0d19-4214-b994-f032fb2e8eb5", + "metadata": { + "id": "587486b4-0d19-4214-b994-f032fb2e8eb5" + }, + "source": [ + "### Inter-conversion to popular data formats" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bb2876ea-999b-4eba-96bc-c196ba02c41c", + "metadata": { + "id": "bb2876ea-999b-4eba-96bc-c196ba02c41c" + }, + "outputs": [], + "source": [ + "df = table.to_pandas()\n", + "pa_table = table.to_arrow()\n" + ] + }, + { + "cell_type": "markdown", + "id": "42659d63-ad76-49d6-8dfc-78d77278db72", + "metadata": { + "id": "42659d63-ad76-49d6-8dfc-78d77278db72" + }, + "source": [ + "### Work with Embeddings\n", + "You can access the raw embedding from lancedb Table and analyse it. The image embeddings are stored in column `vector`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "66d69e9b-046e-41c8-80d7-c0ee40be3bca", + "metadata": { + "id": "66d69e9b-046e-41c8-80d7-c0ee40be3bca" + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "embeddings = table.to_pandas()[\"vector\"].tolist()\n", + "embeddings = np.array(embeddings)" + ] + }, + { + "cell_type": "markdown", + "id": "e8df0a49-9596-4399-954b-b8ae1fd7a602", + "metadata": { + "id": "e8df0a49-9596-4399-954b-b8ae1fd7a602" + }, + "source": [ + "### Scatterplot\n", + "One of the preliminary steps in analysing embeddings is by plotting them in 2D space via dimensionality reduction. Let's try an example\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d9a150e8-8092-41b3-82f8-2247f8187fc8", + "metadata": { + "id": "d9a150e8-8092-41b3-82f8-2247f8187fc8" + }, + "outputs": [], + "source": [ + "!pip install scikit-learn --q" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "196079c3-45a9-4325-81ab-af79a881e37a", + "metadata": { + "id": "196079c3-45a9-4325-81ab-af79a881e37a" + }, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import numpy as np\n", + "from sklearn.decomposition import PCA\n", + "import matplotlib.pyplot as plt\n", + "from mpl_toolkits.mplot3d import Axes3D\n", + "\n", + "# Reduce dimensions using PCA to 3 components for visualization in 3D\n", + "pca = PCA(n_components=3)\n", + "reduced_data = pca.fit_transform(embeddings)\n", + "\n", + "# Create a 3D scatter plot using Matplotlib's Axes3D\n", + "fig = plt.figure(figsize=(8, 6))\n", + "ax = fig.add_subplot(111, projection='3d')\n", + "\n", + "# Scatter plot\n", + "ax.scatter(reduced_data[:, 0], reduced_data[:, 1], reduced_data[:, 2], alpha=0.5)\n", + "ax.set_title('3D Scatter Plot of Reduced 256-Dimensional Data (PCA)')\n", + "ax.set_xlabel('Component 1')\n", + "ax.set_ylabel('Component 2')\n", + "ax.set_zlabel('Component 3')\n", + "\n", + "plt.show()" + ] + }, + { + "cell_type": "markdown", + "id": "1c843c23-e3f2-490e-8d6c-212fa038a149", + "metadata": { + "id": "1c843c23-e3f2-490e-8d6c-212fa038a149" + }, + "source": [ + "## 4. Similarity Index\n", + "Here's a simple example of an operation powered by the embeddings table. Explorer comes with a `similarity_index` operation-\n", + "* It tries to estimate how similar each data point is with the rest of the dataset.\n", + "* It does that by counting how many image embeddings lie closer than `max_dist` to the current image in the generated embedding space, considering `top_k` similar images at a time.\n", + "\n", + "For a given dataset, model, `max_dist` & `top_k` the similarity index once generated will be reused. In case, your dataset has changed, or you simply need to regenerate the similarity index, you can pass `force=True`.\n", + "Similar to vector and SQL search, this also comes with a util to directly plot it. Let's look at the plot first\n", + "\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "953c2a5f-1b61-4acf-a8e4-ed08547dbafc", + "metadata": { + "id": "953c2a5f-1b61-4acf-a8e4-ed08547dbafc" + }, + "outputs": [], + "source": [ + "exp.plot_similarity_index(max_dist=0.2, top_k=0.01)" + ] + }, + { + "cell_type": "markdown", + "id": "28228a9a-b727-45b5-8ca7-8db662c0b937", + "metadata": { + "id": "28228a9a-b727-45b5-8ca7-8db662c0b937" + }, + "source": [ + "Now let's look at the output of the operation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4161aaa-20e6-4df0-8e87-d2293ee0530a", + "metadata": { + "id": "f4161aaa-20e6-4df0-8e87-d2293ee0530a" + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "sim_idx = exp.similarity_index(max_dist=0.2, top_k=0.01, force=False)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b01d5b1a-9adb-4c3c-a873-217c71527c8d", + "metadata": { + "id": "b01d5b1a-9adb-4c3c-a873-217c71527c8d" + }, + "outputs": [], + "source": [ + "sim_idx" + ] + }, + { + "cell_type": "markdown", + "id": "22b28e54-4fbb-400e-ad8c-7068cbba11c4", + "metadata": { + "id": "22b28e54-4fbb-400e-ad8c-7068cbba11c4" + }, + "source": [ + "Let's create a query to see what data points have similarity count of more than 30 and plot images similar to them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "58d2557b-d401-43cf-937d-4f554c7bc808", + "metadata": { + "id": "58d2557b-d401-43cf-937d-4f554c7bc808" + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "\n", + "sim_count = np.array(sim_idx[\"count\"])\n", + "sim_idx['im_file'][sim_count > 30]" + ] + }, + { + "cell_type": "markdown", + "id": "a5ec8d76-271a-41ab-ac74-cf8c0084ba5e", + "metadata": { + "id": "a5ec8d76-271a-41ab-ac74-cf8c0084ba5e" + }, + "source": [ + "You should see something like this\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3a7b2ee3-9f35-48a2-9c38-38379516f4d2", + "metadata": { + "id": "3a7b2ee3-9f35-48a2-9c38-38379516f4d2" + }, + "outputs": [], + "source": [ + "exp.plot_similar(idx=[7146, 14035]) # Using avg embeddings of 2 images" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.6" + }, + "colab": { + "provenance": [] + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/ultralytics/docs/en/datasets/explorer/index.md b/ultralytics/docs/en/datasets/explorer/index.md new file mode 100644 index 0000000000000000000000000000000000000000..111e398ef83df82c1ee5b2c9553e314caeb04ece --- /dev/null +++ b/ultralytics/docs/en/datasets/explorer/index.md @@ -0,0 +1,108 @@ +--- +comments: true +description: Discover Ultralytics Explorer for semantic search, SQL queries, vector similarity, and natural language dataset exploration. Enhance your CV datasets effortlessly. +keywords: Ultralytics Explorer, CV datasets, semantic search, SQL queries, vector similarity, dataset visualization, python API, machine learning, computer vision +--- + +# Ultralytics Explorer + ++ +
+ + +Ultralytics Explorer is a tool for exploring CV datasets using semantic search, SQL queries, vector similarity search and even using natural language. It is also a Python API for accessing the same functionality. + +
+
+
+
+ Watch: Ultralytics Explorer API | Semantic Search, SQL Queries & Ask AI Features
+
+ +
+ +## FAQ + +### What is Ultralytics Explorer and how can it help with CV datasets? + +Ultralytics Explorer is a powerful tool designed for exploring computer vision (CV) datasets through semantic search, SQL queries, vector similarity search, and even natural language. This versatile tool provides both a GUI and a Python API, allowing users to seamlessly interact with their datasets. By leveraging technologies like LanceDB, Ultralytics Explorer ensures efficient, scalable access to large datasets without excessive memory usage. Whether you're performing detailed dataset analysis or exploring data patterns, Ultralytics Explorer streamlines the entire process. + +Learn more about the [Explorer API](api.md). + +### How do I install the dependencies for Ultralytics Explorer? + +To manually install the optional dependencies needed for Ultralytics Explorer, you can use the following `pip` command: + +```bash +pip install ultralytics[explorer] +``` + +These dependencies are essential for the full functionality of semantic search and SQL querying. By including libraries powered by [LanceDB](https://lancedb.com/), the installation ensures that the database operations remain efficient and scalable, even for large datasets like COCO. + +### How can I use the GUI version of Ultralytics Explorer? + +Using the GUI version of Ultralytics Explorer is straightforward. After installing the necessary dependencies, you can launch the GUI with the following command: + +```bash +yolo explorer +``` + +The GUI provides a user-friendly interface for creating dataset embeddings, searching for similar images, running SQL queries, and conducting semantic searches. Additionally, the integration with OpenAI's Ask AI feature allows you to query datasets using natural language, enhancing the flexibility and ease of use. + +For storage and scalability information, check out our [installation instructions](#installation-of-optional-dependencies). + +### What is the Ask AI feature in Ultralytics Explorer? + +The Ask AI feature in Ultralytics Explorer allows users to interact with their datasets using natural language queries. Powered by OpenAI, this feature enables you to ask complex questions and receive insightful answers without needing to write SQL queries or similar commands. To use this feature, you'll need to set your OpenAI API key the first time you run the GUI: + +```bash +yolo settings openai_api_key="YOUR_API_KEY" +``` + +For more on this feature and how to integrate it, see our [GUI Explorer Usage](#gui-explorer-usage) section. + +### Can I run Ultralytics Explorer in Google Colab? + +Yes, Ultralytics Explorer can be run in Google Colab, providing a convenient and powerful environment for dataset exploration. You can start by opening the provided Colab notebook, which is pre-configured with all the necessary settings: + + + +This setup allows you to explore your datasets fully, taking advantage of Google's cloud resources. Learn more in our [Google Colab Guide](../../integrations/google-colab.md). diff --git a/ultralytics/docs/en/datasets/index.md b/ultralytics/docs/en/datasets/index.md new file mode 100644 index 0000000000000000000000000000000000000000..3f607ec36d18e1b83982650cc45545c60a23309d --- /dev/null +++ b/ultralytics/docs/en/datasets/index.md @@ -0,0 +1,222 @@ +--- +comments: true +description: Explore Ultralytics' diverse datasets for vision tasks like detection, segmentation, classification, and more. Enhance your projects with high-quality annotated data. +keywords: Ultralytics, datasets, computer vision, object detection, instance segmentation, pose estimation, image classification, multi-object tracking +--- + +# Datasets Overview + +Ultralytics provides support for various datasets to facilitate computer vision tasks such as detection, instance segmentation, pose estimation, classification, and multi-object tracking. Below is a list of the main Ultralytics datasets, followed by a summary of each computer vision task and the respective datasets. + +
+
+
+
+ Watch: Ultralytics Datasets Overview
+
+ +
+ +- Try the [GUI Demo](explorer/index.md) +- Learn more about the [Explorer API](explorer/index.md) + +## [Object Detection](detect/index.md) + +Bounding box object detection is a computer vision technique that involves detecting and localizing objects in an image by drawing a bounding box around each object. + +- [Argoverse](detect/argoverse.md): A dataset containing 3D tracking and motion forecasting data from urban environments with rich annotations. +- [COCO](detect/coco.md): Common Objects in Context (COCO) is a large-scale object detection, segmentation, and captioning dataset with 80 object categories. +- [LVIS](detect/lvis.md): A large-scale object detection, segmentation, and captioning dataset with 1203 object categories. +- [COCO8](detect/coco8.md): A smaller subset of the first 4 images from COCO train and COCO val, suitable for quick tests. +- [Global Wheat 2020](detect/globalwheat2020.md): A dataset containing images of wheat heads for the Global Wheat Challenge 2020. +- [Objects365](detect/objects365.md): A high-quality, large-scale dataset for object detection with 365 object categories and over 600K annotated images. +- [OpenImagesV7](detect/open-images-v7.md): A comprehensive dataset by Google with 1.7M train images and 42k validation images. +- [SKU-110K](detect/sku-110k.md): A dataset featuring dense object detection in retail environments with over 11K images and 1.7 million bounding boxes. +- [VisDrone](detect/visdrone.md): A dataset containing object detection and multi-object tracking data from drone-captured imagery with over 10K images and video sequences. +- [VOC](detect/voc.md): The Pascal Visual Object Classes (VOC) dataset for object detection and segmentation with 20 object classes and over 11K images. +- [xView](detect/xview.md): A dataset for object detection in overhead imagery with 60 object categories and over 1 million annotated objects. +- [Roboflow 100](detect/roboflow-100.md): A diverse object detection benchmark with 100 datasets spanning seven imagery domains for comprehensive model evaluation. +- [Brain-tumor](detect/brain-tumor.md): A dataset for detecting brain tumors includes MRI or CT scan images with details on tumor presence, location, and characteristics. +- [African-wildlife](detect/african-wildlife.md): A dataset featuring images of African wildlife, including buffalo, elephant, rhino, and zebras. +- [Signature](detect/signature.md): A dataset featuring images of various documents with annotated signatures, supporting document verification and fraud detection research. + +## [Instance Segmentation](segment/index.md) + +Instance segmentation is a computer vision technique that involves identifying and localizing objects in an image at the pixel level. + +- [COCO](segment/coco.md): A large-scale dataset designed for object detection, segmentation, and captioning tasks with over 200K labeled images. +- [COCO8-seg](segment/coco8-seg.md): A smaller dataset for instance segmentation tasks, containing a subset of 8 COCO images with segmentation annotations. +- [Crack-seg](segment/crack-seg.md): Specifically crafted dataset for detecting cracks on roads and walls, applicable for both object detection and segmentation tasks. +- [Package-seg](segment/package-seg.md): Tailored dataset for identifying packages in warehouses or industrial settings, suitable for both object detection and segmentation applications. +- [Carparts-seg](segment/carparts-seg.md): Purpose-built dataset for identifying vehicle parts, catering to design, manufacturing, and research needs. It serves for both object detection and segmentation tasks. + +## [Pose Estimation](pose/index.md) + +Pose estimation is a technique used to determine the pose of the object relative to the camera or the world coordinate system. + +- [COCO](pose/coco.md): A large-scale dataset with human pose annotations designed for pose estimation tasks. +- [COCO8-pose](pose/coco8-pose.md): A smaller dataset for pose estimation tasks, containing a subset of 8 COCO images with human pose annotations. +- [Tiger-pose](pose/tiger-pose.md): A compact dataset consisting of 263 images focused on tigers, annotated with 12 keypoints per tiger for pose estimation tasks. + +## [Classification](classify/index.md) + +Image classification is a computer vision task that involves categorizing an image into one or more predefined classes or categories based on its visual content. + +- [Caltech 101](classify/caltech101.md): A dataset containing images of 101 object categories for image classification tasks. +- [Caltech 256](classify/caltech256.md): An extended version of Caltech 101 with 256 object categories and more challenging images. +- [CIFAR-10](classify/cifar10.md): A dataset of 60K 32x32 color images in 10 classes, with 6K images per class. +- [CIFAR-100](classify/cifar100.md): An extended version of CIFAR-10 with 100 object categories and 600 images per class. +- [Fashion-MNIST](classify/fashion-mnist.md): A dataset consisting of 70,000 grayscale images of 10 fashion categories for image classification tasks. +- [ImageNet](classify/imagenet.md): A large-scale dataset for object detection and image classification with over 14 million images and 20,000 categories. +- [ImageNet-10](classify/imagenet10.md): A smaller subset of ImageNet with 10 categories for faster experimentation and testing. +- [Imagenette](classify/imagenette.md): A smaller subset of ImageNet that contains 10 easily distinguishable classes for quicker training and testing. +- [Imagewoof](classify/imagewoof.md): A more challenging subset of ImageNet containing 10 dog breed categories for image classification tasks. +- [MNIST](classify/mnist.md): A dataset of 70,000 grayscale images of handwritten digits for image classification tasks. + +## [Oriented Bounding Boxes (OBB)](obb/index.md) + +Oriented Bounding Boxes (OBB) is a method in computer vision for detecting angled objects in images using rotated bounding boxes, often applied to aerial and satellite imagery. + +- [DOTA-v2](obb/dota-v2.md): A popular OBB aerial imagery dataset with 1.7 million instances and 11,268 images. + +## [Multi-Object Tracking](track/index.md) + +Multi-object tracking is a computer vision technique that involves detecting and tracking multiple objects over time in a video sequence. + +- [Argoverse](detect/argoverse.md): A dataset containing 3D tracking and motion forecasting data from urban environments with rich annotations for multi-object tracking tasks. +- [VisDrone](detect/visdrone.md): A dataset containing object detection and multi-object tracking data from drone-captured imagery with over 10K images and video sequences. + +## Contribute New Datasets + +Contributing a new dataset involves several steps to ensure that it aligns well with the existing infrastructure. Below are the necessary steps: + +### Steps to Contribute a New Dataset + +1. **Collect Images**: Gather the images that belong to the dataset. These could be collected from various sources, such as public databases or your own collection. +2. **Annotate Images**: Annotate these images with bounding boxes, segments, or keypoints, depending on the task. +3. **Export Annotations**: Convert these annotations into the YOLO `*.txt` file format which Ultralytics supports. +4. **Organize Dataset**: Arrange your dataset into the correct folder structure. You should have `train/` and `val/` top-level directories, and within each, an `images/` and `labels/` subdirectory. + + ``` + dataset/ + ├── train/ + │ ├── images/ + │ └── labels/ + └── val/ + ├── images/ + └── labels/ + ``` + +5. **Create a `data.yaml` File**: In your dataset's root directory, create a `data.yaml` file that describes the dataset, classes, and other necessary information. +6. **Optimize Images (Optional)**: If you want to reduce the size of the dataset for more efficient processing, you can optimize the images using the code below. This is not required, but recommended for smaller dataset sizes and faster download speeds. +7. **Zip Dataset**: Compress the entire dataset folder into a zip file. +8. **Document and PR**: Create a documentation page describing your dataset and how it fits into the existing framework. After that, submit a Pull Request (PR). Refer to [Ultralytics Contribution Guidelines](https://docs.ultralytics.com/help/contributing) for more details on how to submit a PR. + +### Example Code to Optimize and Zip a Dataset + +!!! Example "Optimize and Zip a Dataset" + + === "Python" + + ```python + from pathlib import Path + + from ultralytics.data.utils import compress_one_image + from ultralytics.utils.downloads import zip_directory + + # Define dataset directory + path = Path("path/to/dataset") + + # Optimize images in dataset (optional) + for f in path.rglob("*.jpg"): + compress_one_image(f) + + # Zip dataset into 'path/to/dataset.zip' + zip_directory(path) + ``` + +By following these steps, you can contribute a new dataset that integrates well with Ultralytics' existing structure. + +## FAQ + +### What datasets does Ultralytics support for object detection? + +Ultralytics supports a wide variety of datasets for object detection, including: +- [COCO](detect/coco.md): A large-scale object detection, segmentation, and captioning dataset with 80 object categories. +- [LVIS](detect/lvis.md): An extensive dataset with 1203 object categories, designed for more fine-grained object detection and segmentation. +- [Argoverse](detect/argoverse.md): A dataset containing 3D tracking and motion forecasting data from urban environments with rich annotations. +- [VisDrone](detect/visdrone.md): A dataset with object detection and multi-object tracking data from drone-captured imagery. +- [SKU-110K](detect/sku-110k.md): Featuring dense object detection in retail environments with over 11K images. + +These datasets facilitate training robust models for various object detection applications. + +### How do I contribute a new dataset to Ultralytics? + +Contributing a new dataset involves several steps: +1. **Collect Images**: Gather images from public databases or personal collections. +2. **Annotate Images**: Apply bounding boxes, segments, or keypoints, depending on the task. +3. **Export Annotations**: Convert annotations into the YOLO `*.txt` format. +4. **Organize Dataset**: Use the folder structure with `train/` and `val/` directories, each containing `images/` and `labels/` subdirectories. +5. **Create a `data.yaml` File**: Include dataset descriptions, classes, and other relevant information. +6. **Optimize Images (Optional)**: Reduce dataset size for efficiency. +7. **Zip Dataset**: Compress the dataset into a zip file. +8. **Document and PR**: Describe your dataset and submit a Pull Request following [Ultralytics Contribution Guidelines](https://docs.ultralytics.com/help/contributing). + +Visit [Contribute New Datasets](#contribute-new-datasets) for a comprehensive guide. + +### Why should I use Ultralytics Explorer for my dataset? + +Ultralytics Explorer offers powerful features for dataset analysis, including: +- **Embeddings Generation**: Create vector embeddings for images. +- **Semantic Search**: Search for similar images using embeddings or AI. +- **SQL Queries**: Run advanced SQL queries for detailed data analysis. +- **Natural Language Search**: Search using plain language queries for ease of use. + +Explore the [Ultralytics Explorer](explorer/index.md) for more information and to try the [GUI Demo](explorer/index.md). + +### What are the unique features of Ultralytics YOLO models for computer vision? + +Ultralytics YOLO models provide several unique features: +- **Real-time Performance**: High-speed inference and training. +- **Versatility**: Suitable for detection, segmentation, classification, and pose estimation tasks. +- **Pretrained Models**: Access to high-performing, pretrained models for various applications. +- **Extensive Community Support**: Active community and comprehensive documentation for troubleshooting and development. + +Discover more about YOLO on the [Ultralytics YOLO](https://www.ultralytics.com/yolo) page. + +### How can I optimize and zip a dataset using Ultralytics tools? + +To optimize and zip a dataset using Ultralytics tools, follow this example code: + +!!! Example "Optimize and Zip a Dataset" + + === "Python" + + ```python + from pathlib import Path + + from ultralytics.data.utils import compress_one_image + from ultralytics.utils.downloads import zip_directory + + # Define dataset directory + path = Path("path/to/dataset") + + # Optimize images in dataset (optional) + for f in path.rglob("*.jpg"): + compress_one_image(f) + + # Zip dataset into 'path/to/dataset.zip' + zip_directory(path) + ``` + +Learn more on how to [Optimize and Zip a Dataset](#example-code-to-optimize-and-zip-a-dataset). diff --git a/ultralytics/docs/en/datasets/obb/dota-v2.md b/ultralytics/docs/en/datasets/obb/dota-v2.md new file mode 100644 index 0000000000000000000000000000000000000000..e2cd229a4cb17e1e1dd3a7367448a4ab95232121 --- /dev/null +++ b/ultralytics/docs/en/datasets/obb/dota-v2.md @@ -0,0 +1,232 @@ +--- +comments: true +description: Explore the DOTA dataset for object detection in aerial images, featuring 1.7M Oriented Bounding Boxes across 18 categories. Ideal for aerial image analysis. +keywords: DOTA dataset, object detection, aerial images, oriented bounding boxes, OBB, DOTA v1.0, DOTA v1.5, DOTA v2.0, multiscale detection, Ultralytics +--- + +# DOTA Dataset with OBB + +[DOTA](https://captain-whu.github.io/DOTA/index.html) stands as a specialized dataset, emphasizing object detection in aerial images. Originating from the DOTA series of datasets, it offers annotated images capturing a diverse array of aerial scenes with Oriented Bounding Boxes (OBB). + +![DOTA classes visual](https://user-images.githubusercontent.com/26833433/259461765-72fdd0d8-266b-44a9-8199-199329bf5ca9.jpg) + +## Key Features + +- Collection from various sensors and platforms, with image sizes ranging from 800 × 800 to 20,000 × 20,000 pixels. +- Features more than 1.7M Oriented Bounding Boxes across 18 categories. +- Encompasses multiscale object detection. +- Instances are annotated by experts using arbitrary (8 d.o.f.) quadrilateral, capturing objects of different scales, orientations, and shapes. + +## Dataset Versions + +### DOTA-v1.0 + +- Contains 15 common categories. +- Comprises 2,806 images with 188,282 instances. +- Split ratios: 1/2 for training, 1/6 for validation, and 1/3 for testing. + +### DOTA-v1.5 + +- Incorporates the same images as DOTA-v1.0. +- Very small instances (less than 10 pixels) are also annotated. +- Addition of a new category: "container crane". +- A total of 403,318 instances. +- Released for the DOAI Challenge 2019 on Object Detection in Aerial Images. + +### DOTA-v2.0 + +- Collections from Google Earth, GF-2 Satellite, and other aerial images. +- Contains 18 common categories. +- Comprises 11,268 images with a whopping 1,793,658 instances. +- New categories introduced: "airport" and "helipad". +- Image splits: + - Training: 1,830 images with 268,627 instances. + - Validation: 593 images with 81,048 instances. + - Test-dev: 2,792 images with 353,346 instances. + - Test-challenge: 6,053 images with 1,090,637 instances. + +## Dataset Structure + +DOTA exhibits a structured layout tailored for OBB object detection challenges: + +- **Images**: A vast collection of high-resolution aerial images capturing diverse terrains and structures. +- **Oriented Bounding Boxes**: Annotations in the form of rotated rectangles encapsulating objects irrespective of their orientation, ideal for capturing objects like airplanes, ships, and buildings. + +## Applications + +DOTA serves as a benchmark for training and evaluating models specifically tailored for aerial image analysis. With the inclusion of OBB annotations, it provides a unique challenge, enabling the development of specialized object detection models that cater to aerial imagery's nuances. + +## Dataset YAML + +Typically, datasets incorporate a YAML (Yet Another Markup Language) file detailing the dataset's configuration. For DOTA v1 and DOTA v1.5, Ultralytics provides `DOTAv1.yaml` and `DOTAv1.5.yaml` files. For additional details on these as well as DOTA v2 please consult DOTA's official repository and documentation. + +!!! Example "DOTAv1.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/DOTAv1.yaml" + ``` + +## Split DOTA images + +To train DOTA dataset, we split original DOTA images with high-resolution into images with 1024x1024 resolution in multiscale way. + +!!! Example "Split images" + + === "Python" + + ```python + from ultralytics.data.split_dota import split_test, split_trainval + + # split train and val set, with labels. + split_trainval( + data_root="path/to/DOTAv1.0/", + save_dir="path/to/DOTAv1.0-split/", + rates=[0.5, 1.0, 1.5], # multiscale + gap=500, + ) + # split test set, without labels. + split_test( + data_root="path/to/DOTAv1.0/", + save_dir="path/to/DOTAv1.0-split/", + rates=[0.5, 1.0, 1.5], # multiscale + gap=500, + ) + ``` + +## Usage + +To train a model on the DOTA v1 dataset, you can utilize the following code snippets. Always refer to your model's documentation for a thorough list of available arguments. + +!!! Warning + + Please note that all images and associated annotations in the DOTAv1 dataset can be used for academic purposes, but commercial use is prohibited. Your understanding and respect for the dataset creators' wishes are greatly appreciated! + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Create a new YOLOv8n-OBB model from scratch + model = YOLO("yolov8n-obb.yaml") + + # Train the model on the DOTAv2 dataset + results = model.train(data="DOTAv1.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Train a new YOLOv8n-OBB model on the DOTAv2 dataset + yolo obb train data=DOTAv1.yaml model=yolov8n-obb.pt epochs=100 imgsz=640 + ``` + +## Sample Data and Annotations + +Having a glance at the dataset illustrates its depth: + +![Dataset sample image](https://captain-whu.github.io/DOTA/images/instances-DOTA.jpg) + +- **DOTA examples**: This snapshot underlines the complexity of aerial scenes and the significance of Oriented Bounding Box annotations, capturing objects in their natural orientation. + +The dataset's richness offers invaluable insights into object detection challenges exclusive to aerial imagery. + +## Citations and Acknowledgments + +For those leveraging DOTA in their endeavors, it's pertinent to cite the relevant research papers: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @article{9560031, + author={Ding, Jian and Xue, Nan and Xia, Gui-Song and Bai, Xiang and Yang, Wen and Yang, Michael and Belongie, Serge and Luo, Jiebo and Datcu, Mihai and Pelillo, Marcello and Zhang, Liangpei}, + journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, + title={Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges}, + year={2021}, + volume={}, + number={}, + pages={1-1}, + doi={10.1109/TPAMI.2021.3117983} + } + ``` + +A special note of gratitude to the team behind the DOTA datasets for their commendable effort in curating this dataset. For an exhaustive understanding of the dataset and its nuances, please visit the [official DOTA website](https://captain-whu.github.io/DOTA/index.html). + +## FAQ + +### What is the DOTA dataset and why is it important for object detection in aerial images? + +The [DOTA dataset](https://captain-whu.github.io/DOTA/index.html) is a specialized dataset focused on object detection in aerial images. It features Oriented Bounding Boxes (OBB), providing annotated images from diverse aerial scenes. DOTA's diversity in object orientation, scale, and shape across its 1.7M annotations and 18 categories makes it ideal for developing and evaluating models tailored for aerial imagery analysis, such as those used in surveillance, environmental monitoring, and disaster management. + +### How does the DOTA dataset handle different scales and orientations in images? + +DOTA utilizes Oriented Bounding Boxes (OBB) for annotation, which are represented by rotated rectangles encapsulating objects regardless of their orientation. This method ensures that objects, whether small or at different angles, are accurately captured. The dataset's multiscale images, ranging from 800 × 800 to 20,000 × 20,000 pixels, further allow for the detection of both small and large objects effectively. + +### How can I train a model using the DOTA dataset? + +To train a model on the DOTA dataset, you can use the following example with Ultralytics YOLO: + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Create a new YOLOv8n-OBB model from scratch + model = YOLO("yolov8n-obb.yaml") + + # Train the model on the DOTAv1 dataset + results = model.train(data="DOTAv1.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Train a new YOLOv8n-OBB model on the DOTAv1 dataset + yolo obb train data=DOTAv1.yaml model=yolov8n-obb.pt epochs=100 imgsz=640 + ``` + +For more details on how to split and preprocess the DOTA images, refer to the [split DOTA images section](#split-dota-images). + +### What are the differences between DOTA-v1.0, DOTA-v1.5, and DOTA-v2.0? + +- **DOTA-v1.0**: Includes 15 common categories across 2,806 images with 188,282 instances. The dataset is split into training, validation, and testing sets. + +- **DOTA-v1.5**: Builds upon DOTA-v1.0 by annotating very small instances (less than 10 pixels) and adding a new category, "container crane," totaling 403,318 instances. + +- **DOTA-v2.0**: Expands further with annotations from Google Earth and GF-2 Satellite, featuring 11,268 images and 1,793,658 instances. It includes new categories like "airport" and "helipad." + +For a detailed comparison and additional specifics, check the [dataset versions section](#dataset-versions). + +### How can I prepare high-resolution DOTA images for training? + +DOTA images, which can be very large, are split into smaller resolutions for manageable training. Here's a Python snippet to split images: + +!!! Example + + === "Python" + + ```python + from ultralytics.data.split_dota import split_test, split_trainval + + # split train and val set, with labels. + split_trainval( + data_root="path/to/DOTAv1.0/", + save_dir="path/to/DOTAv1.0-split/", + rates=[0.5, 1.0, 1.5], # multiscale + gap=500, + ) + # split test set, without labels. + split_test( + data_root="path/to/DOTAv1.0/", + save_dir="path/to/DOTAv1.0-split/", + rates=[0.5, 1.0, 1.5], # multiscale + gap=500, + ) + ``` + +This process facilitates better training efficiency and model performance. For detailed instructions, visit the [split DOTA images section](#split-dota-images). diff --git a/ultralytics/docs/en/datasets/obb/dota8.md b/ultralytics/docs/en/datasets/obb/dota8.md new file mode 100644 index 0000000000000000000000000000000000000000..ec111432417e385de7065beb690aa24489940b78 --- /dev/null +++ b/ultralytics/docs/en/datasets/obb/dota8.md @@ -0,0 +1,124 @@ +--- +comments: true +description: Explore the DOTA8 dataset - a small, versatile oriented object detection dataset ideal for testing and debugging object detection models using Ultralytics YOLOv8. +keywords: DOTA8 dataset, Ultralytics, YOLOv8, object detection, debugging, training models, oriented object detection, dataset YAML +--- + +# DOTA8 Dataset + +## Introduction + +[Ultralytics](https://ultralytics.com) DOTA8 is a small, but versatile oriented object detection dataset composed of the first 8 images of 8 images of the split DOTAv1 set, 4 for training and 4 for validation. This dataset is ideal for testing and debugging object detection models, or for experimenting with new detection approaches. With 8 images, it is small enough to be easily manageable, yet diverse enough to test training pipelines for errors and act as a sanity check before training larger datasets. + +This dataset is intended for use with Ultralytics [HUB](https://hub.ultralytics.com) and [YOLOv8](https://github.com/ultralytics/ultralytics). + +## Dataset YAML + +A YAML (Yet Another Markup Language) file is used to define the dataset configuration. It contains information about the dataset's paths, classes, and other relevant information. In the case of the DOTA8 dataset, the `dota8.yaml` file is maintained at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/dota8.yaml](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/dota8.yaml). + +!!! Example "ultralytics/cfg/datasets/dota8.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/dota8.yaml" + ``` + +## Usage + +To train a YOLOv8n-obb model on the DOTA8 dataset for 100 epochs with an image size of 640, you can use the following code snippets. For a comprehensive list of available arguments, refer to the model [Training](../../modes/train.md) page. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n-obb.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="dota8.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo obb train data=dota8.yaml model=yolov8n-obb.pt epochs=100 imgsz=640 + ``` + +## Sample Images and Annotations + +Here are some examples of images from the DOTA8 dataset, along with their corresponding annotations: + + + +- **Mosaiced Image**: This image demonstrates a training batch composed of mosaiced dataset images. Mosaicing is a technique used during training that combines multiple images into a single image to increase the variety of objects and scenes within each training batch. This helps improve the model's ability to generalize to different object sizes, aspect ratios, and contexts. + +The example showcases the variety and complexity of the images in the DOTA8 dataset and the benefits of using mosaicing during the training process. + +## Citations and Acknowledgments + +If you use the DOTA dataset in your research or development work, please cite the following paper: + +!!! Quote "" + + === "BibTeX" + + ```bibtex + @article{9560031, + author={Ding, Jian and Xue, Nan and Xia, Gui-Song and Bai, Xiang and Yang, Wen and Yang, Michael and Belongie, Serge and Luo, Jiebo and Datcu, Mihai and Pelillo, Marcello and Zhang, Liangpei}, + journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, + title={Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges}, + year={2021}, + volume={}, + number={}, + pages={1-1}, + doi={10.1109/TPAMI.2021.3117983} + } + ``` + +A special note of gratitude to the team behind the DOTA datasets for their commendable effort in curating this dataset. For an exhaustive understanding of the dataset and its nuances, please visit the [official DOTA website](https://captain-whu.github.io/DOTA/index.html). + +## FAQ + +### What is the DOTA8 dataset and how can it be used? + +The DOTA8 dataset is a small, versatile oriented object detection dataset made up of the first 8 images from the DOTAv1 split set, with 4 images designated for training and 4 for validation. It's ideal for testing and debugging object detection models like Ultralytics YOLOv8. Due to its manageable size and diversity, it helps in identifying pipeline errors and running sanity checks before deploying larger datasets. Learn more about object detection with [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics). + +### How do I train a YOLOv8 model using the DOTA8 dataset? + +To train a YOLOv8n-obb model on the DOTA8 dataset for 100 epochs with an image size of 640, you can use the following code snippets. For comprehensive argument options, refer to the model [Training](../../modes/train.md) page. + +!!! Example "Train Example" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n-obb.pt") # load a pretrained model (recommended for training) + + # Train the model + results = model.train(data="dota8.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Start training from a pretrained *.pt model + yolo obb train data=dota8.yaml model=yolov8n-obb.pt epochs=100 imgsz=640 + ``` + +### What are the key features of the DOTA dataset and where can I access the YAML file? + +The DOTA dataset is known for its large-scale benchmark and the challenges it presents for object detection in aerial images. The DOTA8 subset is a smaller, manageable dataset ideal for initial tests. You can access the `dota8.yaml` file, which contains paths, classes, and configuration details, at this [GitHub link](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/cfg/datasets/dota8.yaml). + +### How does mosaicing enhance model training with the DOTA8 dataset? + +Mosaicing combines multiple images into one during training, increasing the variety of objects and contexts within each batch. This improves a model's ability to generalize to different object sizes, aspect ratios, and scenes. This technique can be visually demonstrated through a training batch composed of mosaiced DOTA8 dataset images, helping in robust model development. Explore more about mosaicing and training techniques on our [Training](../../modes/train.md) page. + +### Why should I use Ultralytics YOLOv8 for object detection tasks? + +Ultralytics YOLOv8 provides state-of-the-art real-time object detection capabilities, including features like oriented bounding boxes (OBB), instance segmentation, and a highly versatile training pipeline. It's suitable for various applications and offers pretrained models for efficient fine-tuning. Explore further about the advantages and usage in the [Ultralytics YOLOv8 documentation](https://github.com/ultralytics/ultralytics). diff --git a/ultralytics/docs/en/datasets/obb/index.md b/ultralytics/docs/en/datasets/obb/index.md new file mode 100644 index 0000000000000000000000000000000000000000..b75cb502afadc5ba390656abe0bd5159b185fa4d --- /dev/null +++ b/ultralytics/docs/en/datasets/obb/index.md @@ -0,0 +1,144 @@ +--- +comments: true +description: Discover OBB dataset formats for Ultralytics YOLO models. Learn about their structure, application, and format conversions to enhance your object detection training. +keywords: Oriented Bounding Box, OBB Datasets, YOLO, Ultralytics, Object Detection, Dataset Formats +--- + +# Oriented Bounding Box (OBB) Datasets Overview + +Training a precise object detection model with oriented bounding boxes (OBB) requires a thorough dataset. This guide explains the various OBB dataset formats compatible with Ultralytics YOLO models, offering insights into their structure, application, and methods for format conversions. + +## Supported OBB Dataset Formats + +### YOLO OBB Format + +The YOLO OBB format designates bounding boxes by their four corner points with coordinates normalized between 0 and 1. It follows this format: + +```bash +class_index x1 y1 x2 y2 x3 y3 x4 y4 +``` + +Internally, YOLO processes losses and outputs in the `xywhr` format, which represents the bounding box's center point (xy), width, height, and rotation. + + + +An example of a `*.txt` label file for the above image, which contains an object of class `0` in OBB format, could look like: + +```bash +0 0.780811 0.743961 0.782371 0.74686 0.777691 0.752174 0.776131 0.749758 +``` + +## Usage + +To train a model using these OBB formats: + +!!! Example + + === "Python" + + ```python + from ultralytics import YOLO + + # Create a new YOLOv8n-OBB model from scratch + model = YOLO("yolov8n-obb.yaml") + + # Train the model on the DOTAv2 dataset + results = model.train(data="DOTAv1.yaml", epochs=100, imgsz=640) + ``` + + === "CLI" + + ```bash + # Train a new YOLOv8n-OBB model on the DOTAv2 dataset + yolo obb train data=DOTAv1.yaml model=yolov8n-obb.pt epochs=100 imgsz=640 + ``` + +## Supported Datasets + +Currently, the following datasets with Oriented Bounding Boxes are supported: + +- [DOTA-v2](dota-v2.md): DOTA (A Large-scale Dataset for Object Detection in Aerial Images) version 2, emphasizes detection from aerial perspectives and contains oriented bounding boxes with 1.7 million instances and 11,268 images. +- [DOTA8](dota8.md): A small, 8-image subset of the full DOTA dataset suitable for testing workflows and Continuous Integration (CI) checks of OBB training in the `ultralytics` repository. + +### Incorporating your own OBB dataset + +For those looking to introduce their own datasets with oriented bounding boxes, ensure compatibility with the "YOLO OBB format" mentioned above. Convert your annotations to this required format and detail the paths, classes, and class names in a corresponding YAML configuration file. + +## Convert Label Formats + +### DOTA Dataset Format to YOLO OBB Format + +Transitioning labels from the DOTA dataset format to the YOLO OBB format can be achieved with this script: + +!!! Example + + === "Python" + + ```python + from ultralytics.data.converter import convert_dota_to_yolo_obb + + convert_dota_to_yolo_obb("path/to/DOTA") + ``` + +This conversion mechanism is instrumental for datasets in the DOTA format, ensuring alignment with the Ultralytics YOLO OBB format. + +It's imperative to validate the compatibility of the dataset with your model and adhere to the necessary format conventions. Properly structured datasets are pivotal for training efficient object detection models with oriented bounding boxes. + +## FAQ + +### What are Oriented Bounding Boxes (OBB) and how are they used in Ultralytics YOLO models? + +Oriented Bounding Boxes (OBB) are a type of bounding box annotation where the box can be rotated to align more closely with the object being detected, rather than just being axis-aligned. This is particularly useful in aerial or satellite imagery where objects might not be aligned with the image axes. In Ultralytics YOLO models, OBBs are represented by their four corner points in the YOLO OBB format. This allows for more accurate object detection since the bounding boxes can rotate to fit the objects better. + +### How do I convert my existing DOTA dataset labels to YOLO OBB format for use with Ultralytics YOLOv8? + +You can convert DOTA dataset labels to YOLO OBB format using the `convert_dota_to_yolo_obb` function from Ultralytics. This conversion ensures compatibility with the Ultralytics YOLO models, enabling you to leverage the OBB capabilities for enhanced object detection. Here's a quick example: + +```python +from ultralytics.data.converter import convert_dota_to_yolo_obb + +convert_dota_to_yolo_obb("path/to/DOTA") +``` + +This script will reformat your DOTA annotations into a YOLO-compatible format. + +### How do I train a YOLOv8 model with oriented bounding boxes (OBB) on my dataset? + +Training a YOLOv8 model with OBBs involves ensuring your dataset is in the YOLO OBB format and then using the Ultralytics API to train the model. Here's an example in both Python and CLI: + +!!! Example + + === "Python" + + ```python + from ultralytics import YOLO + + # Create a new YOLOv8n-OBB model from scratch + model = YOLO("yolov8n-obb.yaml") + + # Train the model on the custom dataset + results = model.train(data="your_dataset.yaml", epochs=100, imgsz=640) + ``` + + + === "CLI" + + ```bash + # Train a new YOLOv8n-OBB model on the custom dataset + yolo obb train data=your_dataset.yaml model=yolov8n-obb.yaml epochs=100 imgsz=640 + ``` + +This ensures your model leverages the detailed OBB annotations for improved detection accuracy. + +### What datasets are currently supported for OBB training in Ultralytics YOLO models? + +Currently, Ultralytics supports the following datasets for OBB training: + +- [DOTA-v2](dota-v2.md): This dataset includes 1.7 million instances with oriented bounding boxes and 11,268 images, primarily focusing on aerial object detection. +- [DOTA8](dota8.md): A smaller, 8-image subset of the DOTA dataset used for testing and continuous integration (CI) checks. + +These datasets are tailored for scenarios where OBBs offer a significant advantage, such as aerial and satellite image analysis. + +### Can I use my own dataset with oriented bounding boxes for YOLOv8 training, and if so, how? + +Yes, you can use your own dataset with oriented bounding boxes for YOLOv8 training. Ensure your dataset annotations are converted to the YOLO OBB format, which involves defining bounding boxes by their four corner points. You can then create a YAML configuration file specifying the dataset paths, classes, and other necessary details. For more information on creating and configuring your datasets, refer to the [Supported Datasets](#supported-datasets) section. diff --git a/ultralytics/docs/en/datasets/pose/coco.md b/ultralytics/docs/en/datasets/pose/coco.md new file mode 100644 index 0000000000000000000000000000000000000000..25b24781707c27a7a7cdf892a92bac587152b478 --- /dev/null +++ b/ultralytics/docs/en/datasets/pose/coco.md @@ -0,0 +1,159 @@ +--- +comments: true +description: Explore the COCO-Pose dataset for advanced pose estimation. Learn about datasets, pretrained models, metrics, and applications for training with YOLO. +keywords: COCO-Pose, pose estimation, dataset, keypoints, COCO Keypoints 2017, YOLO, deep learning, computer vision +--- + +# COCO-Pose Dataset + +The [COCO-Pose](https://cocodataset.org/#keypoints-2017) dataset is a specialized version of the COCO (Common Objects in Context) dataset, designed for pose estimation tasks. It leverages the COCO Keypoints 2017 images and labels to enable the training of models like YOLO for pose estimation tasks. + +![Pose sample image](https://user-images.githubusercontent.com/26833433/277141128-cd62d09e-1eb0-4d20-9938-c55239a5cb76.jpg) + +## COCO-Pose Pretrained Models + +| Model | size
+
+
+
+ Watch: Train YOLOv8 Pose Model on Tiger-Pose Dataset Using Ultralytics HUB
+
+
+
+
+ Watch: Carparts Instance Segmentation Using Ultralytics HUB
+
+ +
+ +## Quickstart from Terminal + +Start your compute and open a Terminal: + ++ +
+ +### Create virtualenv + +Create your conda virtualenv and install pip in it: + +```bash +conda create --name yolov8env -y +conda activate yolov8env +conda install pip -y +``` + +Install the required dependencies: + +```bash +cd ultralytics +pip install -r requirements.txt +pip install ultralytics +pip install onnx>=1.12.0 +``` + +### Perform YOLOv8 tasks + +Predict: + +```bash +yolo predict model=yolov8n.pt source='https://ultralytics.com/images/bus.jpg' +``` + +Train a detection model for 10 epochs with an initial learning_rate of 0.01: + +```bash +yolo train data=coco8.yaml model=yolov8n.pt epochs=10 lr0=0.01 +``` + +You can find more [instructions to use the Ultralytics CLI here](../quickstart.md#use-ultralytics-with-cli). + +## Quickstart from a Notebook + +### Create a new IPython kernel + +Open the compute Terminal. + ++ +
+ +From your compute terminal, you need to create a new ipykernel that will be used by your notebook to manage your dependencies: + +```bash +conda create --name yolov8env -y +conda activate yolov8env +conda install pip -y +conda install ipykernel -y +python -m ipykernel install --user --name yolov8env --display-name "yolov8env" +``` + +Close your terminal and create a new notebook. From your Notebook, you can select the new kernel. + +Then you can open a Notebook cell and install the required dependencies: + +```bash +%%bash +source activate yolov8env +cd ultralytics +pip install -r requirements.txt +pip install ultralytics +pip install onnx>=1.12.0 +``` + +Note that we need to use the `source activate yolov8env` for all the %%bash cells, to make sure that the %%bash cell uses environment we want. + +Run some predictions using the [Ultralytics CLI](../quickstart.md#use-ultralytics-with-cli): + +```bash +%%bash +source activate yolov8env +yolo predict model=yolov8n.pt source='https://ultralytics.com/images/bus.jpg' +``` + +Or with the [Ultralytics Python interface](../quickstart.md#use-ultralytics-with-python), for example to train the model: + +```python +from ultralytics import YOLO + +# Load a model +model = YOLO("yolov8n.pt") # load an official YOLOv8n model + +# Use the model +model.train(data="coco8.yaml", epochs=3) # train the model +metrics = model.val() # evaluate model performance on the validation set +results = model("https://ultralytics.com/images/bus.jpg") # predict on an image +path = model.export(format="onnx") # export the model to ONNX format +``` + +You can use either the Ultralytics CLI or Python interface for running YOLOv8 tasks, as described in the terminal section above. + +By following these steps, you should be able to get YOLOv8 running quickly on AzureML for quick trials. For more advanced uses, you may refer to the full AzureML documentation linked at the beginning of this guide. + +## Explore More with AzureML + +This guide serves as an introduction to get you up and running with YOLOv8 on AzureML. However, it only scratches the surface of what AzureML can offer. To delve deeper and unlock the full potential of AzureML for your machine learning projects, consider exploring the following resources: + +- [Create a Data Asset](https://learn.microsoft.com/azure/machine-learning/how-to-create-data-assets): Learn how to set up and manage your data assets effectively within the AzureML environment. +- [Initiate an AzureML Job](https://learn.microsoft.com/azure/machine-learning/how-to-train-model): Get a comprehensive understanding of how to kickstart your machine learning training jobs on AzureML. +- [Register a Model](https://learn.microsoft.com/azure/machine-learning/how-to-manage-models): Familiarize yourself with model management practices including registration, versioning, and deployment. +- [Train YOLOv8 with AzureML Python SDK](https://medium.com/@ouphi/how-to-train-the-yolov8-model-with-azure-machine-learning-python-sdk-8268696be8ba): Explore a step-by-step guide on using the AzureML Python SDK to train your YOLOv8 models. +- [Train YOLOv8 with AzureML CLI](https://medium.com/@ouphi/how-to-train-the-yolov8-model-with-azureml-and-the-az-cli-73d3c870ba8e): Discover how to utilize the command-line interface for streamlined training and management of YOLOv8 models on AzureML. + +## FAQ + +### How do I run YOLOv8 on AzureML for model training? + +Running YOLOv8 on AzureML for model training involves several steps: + +1. **Create a Compute Instance**: From your AzureML workspace, navigate to Compute > Compute instances > New, and select the required instance. + +2. **Setup Environment**: Start your compute instance, open a terminal, and create a conda environment: + + ```bash + conda create --name yolov8env -y + conda activate yolov8env + conda install pip -y + pip install ultralytics onnx>=1.12.0 + ``` + +3. **Run YOLOv8 Tasks**: Use the Ultralytics CLI to train your model: + ```bash + yolo train data=coco8.yaml model=yolov8n.pt epochs=10 lr0=0.01 + ``` + +For more details, you can refer to the [instructions to use the Ultralytics CLI](../quickstart.md#use-ultralytics-with-cli). + +### What are the benefits of using AzureML for YOLOv8 training? + +AzureML provides a robust and efficient ecosystem for training YOLOv8 models: + +- **Scalability**: Easily scale your compute resources as your data and model complexity grows. +- **MLOps Integration**: Utilize features like versioning, monitoring, and auditing to streamline ML operations. +- **Collaboration**: Share and manage resources within teams, enhancing collaborative workflows. + +These advantages make AzureML an ideal platform for projects ranging from quick prototypes to large-scale deployments. For more tips, check out [AzureML Jobs](https://learn.microsoft.com/azure/machine-learning/how-to-train-model). + +### How do I troubleshoot common issues when running YOLOv8 on AzureML? + +Troubleshooting common issues with YOLOv8 on AzureML can involve the following steps: + +- **Dependency Issues**: Ensure all required packages are installed. Refer to the `requirements.txt` file for dependencies. +- **Environment Setup**: Verify that your conda environment is correctly activated before running commands. +- **Resource Allocation**: Make sure your compute instances have sufficient resources to handle the training workload. + +For additional guidance, review our [YOLO Common Issues](https://docs.ultralytics.com/guides/yolo-common-issues/) documentation. + +### Can I use both the Ultralytics CLI and Python interface on AzureML? + +Yes, AzureML allows you to use both the Ultralytics CLI and the Python interface seamlessly: + +- **CLI**: Ideal for quick tasks and running standard scripts directly from the terminal. + + ```bash + yolo predict model=yolov8n.pt source='https://ultralytics.com/images/bus.jpg' + ``` + +- **Python Interface**: Useful for more complex tasks requiring custom coding and integration within notebooks. + + ```python + from ultralytics import YOLO + + model = YOLO("yolov8n.pt") + model.train(data="coco8.yaml", epochs=3) + ``` + +Refer to the quickstart guides for more detailed instructions [here](../quickstart.md#use-ultralytics-with-cli) and [here](../quickstart.md#use-ultralytics-with-python). + +### What is the advantage of using Ultralytics YOLOv8 over other object detection models? + +Ultralytics YOLOv8 offers several unique advantages over competing object detection models: + +- **Speed**: Faster inference and training times compared to models like Faster R-CNN and SSD. +- **Accuracy**: High accuracy in detection tasks with features like anchor-free design and enhanced augmentation strategies. +- **Ease of Use**: Intuitive API and CLI for quick setup, making it accessible both to beginners and experts. + +To explore more about YOLOv8's features, visit the [Ultralytics YOLO](https://www.ultralytics.com/yolo) page for detailed insights. diff --git a/ultralytics/docs/en/guides/conda-quickstart.md b/ultralytics/docs/en/guides/conda-quickstart.md new file mode 100644 index 0000000000000000000000000000000000000000..32d267537aed61635fe51efcee640ec6d45a41c2 --- /dev/null +++ b/ultralytics/docs/en/guides/conda-quickstart.md @@ -0,0 +1,192 @@ +--- +comments: true +description: Learn to set up a Conda environment for Ultralytics projects. Follow our comprehensive guide for easy installation and initialization. +keywords: Ultralytics, Conda, setup, installation, environment, guide, machine learning, data science +--- + +# Conda Quickstart Guide for Ultralytics + ++ +
+ +This guide provides a comprehensive introduction to setting up a Conda environment for your Ultralytics projects. Conda is an open-source package and environment management system that offers an excellent alternative to pip for installing packages and dependencies. Its isolated environments make it particularly well-suited for data science and machine learning endeavors. For more details, visit the Ultralytics Conda package on [Anaconda](https://anaconda.org/conda-forge/ultralytics) and check out the Ultralytics feedstock repository for package updates on [GitHub](https://github.com/conda-forge/ultralytics-feedstock/). + +[![Conda Version](https://img.shields.io/conda/vn/conda-forge/ultralytics?logo=condaforge)](https://anaconda.org/conda-forge/ultralytics) +[![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/ultralytics.svg)](https://anaconda.org/conda-forge/ultralytics) +[![Conda Recipe](https://img.shields.io/badge/recipe-ultralytics-green.svg)](https://anaconda.org/conda-forge/ultralytics) +[![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/ultralytics.svg)](https://anaconda.org/conda-forge/ultralytics) + +## What You Will Learn + +- Setting up a Conda environment +- Installing Ultralytics via Conda +- Initializing Ultralytics in your environment +- Using Ultralytics Docker images with Conda + +--- + +## Prerequisites + +- You should have Anaconda or Miniconda installed on your system. If not, download and install it from [Anaconda](https://www.anaconda.com/) or [Miniconda](https://docs.conda.io/projects/miniconda/en/latest/). + +--- + +## Setting up a Conda Environment + +First, let's create a new Conda environment. Open your terminal and run the following command: + +```bash +conda create --name ultralytics-env python=3.8 -y +``` + +Activate the new environment: + +```bash +conda activate ultralytics-env +``` + +--- + +## Installing Ultralytics + +You can install the Ultralytics package from the conda-forge channel. Execute the following command: + +```bash +conda install -c conda-forge ultralytics +``` + +### Note on CUDA Environment + +If you're working in a CUDA-enabled environment, it's a good practice to install `ultralytics`, `pytorch`, and `pytorch-cuda` together to resolve any conflicts: + +```bash +conda install -c pytorch -c nvidia -c conda-forge pytorch torchvision pytorch-cuda=11.8 ultralytics +``` + +--- + +## Using Ultralytics + +With Ultralytics installed, you can now start using its robust features for object detection, instance segmentation, and more. For example, to predict an image, you can run: + +```python +from ultralytics import YOLO + +model = YOLO("yolov8n.pt") # initialize model +results = model("path/to/image.jpg") # perform inference +results[0].show() # display results for the first image +``` + +--- + +## Ultralytics Conda Docker Image + +If you prefer using Docker, Ultralytics offers Docker images with a Conda environment included. You can pull these images from [DockerHub](https://hub.docker.com/r/ultralytics/ultralytics). + +Pull the latest Ultralytics image: + +```bash +# Set image name as a variable +t=ultralytics/ultralytics:latest-conda + +# Pull the latest Ultralytics image from Docker Hub +sudo docker pull $t +``` + +Run the image: + +```bash +# Run the Ultralytics image in a container with GPU support +sudo docker run -it --ipc=host --gpus all $t # all GPUs +sudo docker run -it --ipc=host --gpus '"device=2,3"' $t # specify GPUs +``` + +## Speeding Up Installation with Libmamba + +If you're looking to [speed up the package installation](https://www.anaconda.com/blog/a-faster-conda-for-a-growing-community) process in Conda, you can opt to use `libmamba`, a fast, cross-platform, and dependency-aware package manager that serves as an alternative solver to Conda's default. + +### How to Enable Libmamba + +To enable `libmamba` as the solver for Conda, you can perform the following steps: + +1. First, install the `conda-libmamba-solver` package. This can be skipped if your Conda version is 4.11 or above, as `libmamba` is included by default. + + ```bash + conda install conda-libmamba-solver + ``` + +2. Next, configure Conda to use `libmamba` as the solver: + + ```bash + conda config --set solver libmamba + ``` + +And that's it! Your Conda installation will now use `libmamba` as the solver, which should result in a faster package installation process. + +--- + +Congratulations! You have successfully set up a Conda environment, installed the Ultralytics package, and are now ready to explore its rich functionalities. Feel free to dive deeper into the [Ultralytics documentation](../index.md) for more advanced tutorials and examples. + +## FAQ + +### What is the process for setting up a Conda environment for Ultralytics projects? + +Setting up a Conda environment for Ultralytics projects is straightforward and ensures smooth package management. First, create a new Conda environment using the following command: + +```bash +conda create --name ultralytics-env python=3.8 -y +``` + +Then, activate the new environment with: + +```bash +conda activate ultralytics-env +``` + +Finally, install Ultralytics from the conda-forge channel: + +```bash +conda install -c conda-forge ultralytics +``` + +### Why should I use Conda over pip for managing dependencies in Ultralytics projects? + +Conda is a robust package and environment management system that offers several advantages over pip. It manages dependencies efficiently and ensures that all necessary libraries are compatible. Conda's isolated environments prevent conflicts between packages, which is crucial in data science and machine learning projects. Additionally, Conda supports binary package distribution, speeding up the installation process. + +### Can I use Ultralytics YOLO in a CUDA-enabled environment for faster performance? + +Yes, you can enhance performance by utilizing a CUDA-enabled environment. Ensure that you install `ultralytics`, `pytorch`, and `pytorch-cuda` together to avoid conflicts: + +```bash +conda install -c pytorch -c nvidia -c conda-forge pytorch torchvision pytorch-cuda=11.8 ultralytics +``` + +This setup enables GPU acceleration, crucial for intensive tasks like deep learning model training and inference. For more information, visit the [Ultralytics installation guide](../quickstart.md). + +### What are the benefits of using Ultralytics Docker images with a Conda environment? + +Using Ultralytics Docker images ensures a consistent and reproducible environment, eliminating "it works on my machine" issues. These images include a pre-configured Conda environment, simplifying the setup process. You can pull and run the latest Ultralytics Docker image with the following commands: + +```bash +sudo docker pull ultralytics/ultralytics:latest-conda +sudo docker run -it --ipc=host --gpus all ultralytics/ultralytics:latest-conda +``` + +This approach is ideal for deploying applications in production or running complex workflows without manual configuration. Learn more about [Ultralytics Conda Docker Image](../quickstart.md). + +### How can I speed up Conda package installation in my Ultralytics environment? + +You can speed up the package installation process by using `libmamba`, a fast dependency solver for Conda. First, install the `conda-libmamba-solver` package: + +```bash +conda install conda-libmamba-solver +``` + +Then configure Conda to use `libmamba` as the solver: + +```bash +conda config --set solver libmamba +``` + +This setup provides faster and more efficient package management. For more tips on optimizing your environment, read about [libmamba installation](../quickstart.md). diff --git a/ultralytics/docs/en/guides/coral-edge-tpu-on-raspberry-pi.md b/ultralytics/docs/en/guides/coral-edge-tpu-on-raspberry-pi.md new file mode 100644 index 0000000000000000000000000000000000000000..43a6f4a540817ab207f8e9df33b026a241107e0e --- /dev/null +++ b/ultralytics/docs/en/guides/coral-edge-tpu-on-raspberry-pi.md @@ -0,0 +1,224 @@ +--- +comments: true +description: Learn how to boost your Raspberry Pi's ML performance using Coral Edge TPU with Ultralytics YOLOv8. Follow our detailed setup and installation guide. +keywords: Coral Edge TPU, Raspberry Pi, YOLOv8, Ultralytics, TensorFlow Lite, ML inference, machine learning, AI, installation guide, setup tutorial +--- + +# Coral Edge TPU on a Raspberry Pi with Ultralytics YOLOv8 🚀 + ++ +
+ +## What is a Coral Edge TPU? + +The Coral Edge TPU is a compact device that adds an Edge TPU coprocessor to your system. It enables low-power, high-performance ML inference for TensorFlow Lite models. Read more at the [Coral Edge TPU home page](https://coral.ai/products/accelerator). + +## Boost Raspberry Pi Model Performance with Coral Edge TPU + +Many people want to run their models on an embedded or mobile device such as a Raspberry Pi, since they are very power efficient and can be used in many different applications. However, the inference performance on these devices is usually poor even when using formats like [onnx](../integrations/onnx.md) or [openvino](../integrations/openvino.md). The Coral Edge TPU is a great solution to this problem, since it can be used with a Raspberry Pi and accelerate inference performance greatly. + +## Edge TPU on Raspberry Pi with TensorFlow Lite (New)⭐ + +The [existing guide](https://coral.ai/docs/accelerator/get-started/) by Coral on how to use the Edge TPU with a Raspberry Pi is outdated, and the current Coral Edge TPU runtime builds do not work with the current TensorFlow Lite runtime versions anymore. In addition to that, Google seems to have completely abandoned the Coral project, and there have not been any updates between 2021 and 2024. This guide will show you how to get the Edge TPU working with the latest versions of the TensorFlow Lite runtime and an updated Coral Edge TPU runtime on a Raspberry Pi single board computer (SBC). + +## Prerequisites + +- [Raspberry Pi 4B](https://www.raspberrypi.com/products/raspberry-pi-4-model-b/) (2GB or more recommended) or [Raspberry Pi 5](https://www.raspberrypi.com/products/raspberry-pi-5/) (Recommended) +- [Raspberry Pi OS](https://www.raspberrypi.com/software/) Bullseye/Bookworm (64-bit) with desktop (Recommended) +- [Coral USB Accelerator](https://coral.ai/products/accelerator/) +- A non-ARM based platform for exporting an Ultralytics PyTorch model + +## Installation Walkthrough + +This guide assumes that you already have a working Raspberry Pi OS install and have installed `ultralytics` and all dependencies. To get `ultralytics` installed, visit the [quickstart guide](../quickstart.md) to get setup before continuing here. + +### Installing the Edge TPU runtime + +First, we need to install the Edge TPU runtime. There are many different versions available, so you need to choose the right version for your operating system. + +| Raspberry Pi OS | High frequency mode | Version to download | +| --------------- | :-----------------: | ------------------------------------------ | +| Bullseye 32bit | No | `libedgetpu1-std_ ... .bullseye_armhf.deb` | +| Bullseye 64bit | No | `libedgetpu1-std_ ... .bullseye_arm64.deb` | +| Bullseye 32bit | Yes | `libedgetpu1-max_ ... .bullseye_armhf.deb` | +| Bullseye 64bit | Yes | `libedgetpu1-max_ ... .bullseye_arm64.deb` | +| Bookworm 32bit | No | `libedgetpu1-std_ ... .bookworm_armhf.deb` | +| Bookworm 64bit | No | `libedgetpu1-std_ ... .bookworm_arm64.deb` | +| Bookworm 32bit | Yes | `libedgetpu1-max_ ... .bookworm_armhf.deb` | +| Bookworm 64bit | Yes | `libedgetpu1-max_ ... .bookworm_arm64.deb` | + +[Download the latest version from here](https://github.com/feranick/libedgetpu/releases). + +After downloading the file, you can install it with the following command: + +```bash +sudo dpkg -i path/to/package.deb +``` + +After installing the runtime, you need to plug in your Coral Edge TPU into a USB 3.0 port on your Raspberry Pi. This is because, according to the official guide, a new `udev` rule needs to take effect after installation. + +???+ warning "Important" + + If you already have the Coral Edge TPU runtime installed, uninstall it using the following command. + + ```bash + # If you installed the standard version + sudo apt remove libedgetpu1-std + + # If you installed the high frequency version + sudo apt remove libedgetpu1-max + ``` + +## Export your model to a Edge TPU compatible model + +To use the Edge TPU, you need to convert your model into a compatible format. It is recommended that you run export on Google Colab, x86_64 Linux machine, using the official [Ultralytics Docker container](docker-quickstart.md), or using [Ultralytics HUB](../hub/quickstart.md), since the Edge TPU compiler is not available on ARM. See the [Export Mode](../modes/export.md) for the available arguments. + +!!! Exporting the model + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("path/to/model.pt") # Load an official model or custom model + + # Export the model + model.export(format="edgetpu") + ``` + + === "CLI" + + ```bash + yolo export model=path/to/model.pt format=edgetpu # Export an official model or custom model + ``` + +The exported model will be saved in the `+ +
+ +### Common Annotation Formats + +After selecting a type of annotation, it's important to choose the appropriate format for storing and sharing annotations. + +Commonly used formats include [COCO](../datasets/detect/coco.md), which supports various annotation types like object detection, keypoint detection, stuff segmentation, panoptic segmentation, and image captioning, stored in JSON. [Pascal VOC](../datasets/detect/voc.md) uses XML files and is popular for object detection tasks. YOLO, on the other hand, creates a .txt file for each image, containing annotations like object class, coordinates, height, and width, making it suitable for object detection. + +### Techniques of Annotation + +Now, assuming you've chosen a type of annotation and format, it's time to establish clear and objective labeling rules. These rules are like a roadmap for consistency and accuracy throughout the annotation process. Key aspects of these rules include: + +- **Clarity and Detail**: Make sure your instructions are clear. Use examples and illustrations to understand what's expected. +- **Consistency**: Keep your annotations uniform. Set standard criteria for annotating different types of data, so all annotations follow the same rules. +- **Reducing Bias**: Stay neutral. Train yourself to be objective and minimize personal biases to ensure fair annotations. +- **Efficiency**: Work smarter, not harder. Use tools and workflows that automate repetitive tasks, making the annotation process faster and more efficient. + +Regularly reviewing and updating your labeling rules will help keep your annotations accurate, consistent, and aligned with your project goals. + +### Popular Annotation Tools + +Let's say you are ready to annotate now. There are several open-source tools available to help streamline the data annotation process. Here are some useful open annotation tools: + +- **[Label Studio](https://github.com/HumanSignal/label-studio)**: A flexible tool that supports a wide range of annotation tasks and includes features for managing projects and quality control. +- **[CVAT](https://github.com/cvat-ai/cvat)**: A powerful tool that supports various annotation formats and customizable workflows, making it suitable for complex projects. +- **[Labelme](https://github.com/labelmeai/labelme)**: A simple and easy-to-use tool that allows for quick annotation of images with polygons, making it ideal for straightforward tasks. + ++ +
+ +These open-source tools are budget-friendly and provide a range of features to meet different annotation needs. + +### Some More Things to Consider Before Annotating Data + +Before you dive into annotating your data, there are a few more things to keep in mind. You should be aware of accuracy, precision, outliers, and quality control to avoid labeling your data in a counterproductive manner. + +#### Understanding Accuracy and Precision + +It's important to understand the difference between accuracy and precision and how it relates to annotation. Accuracy refers to how close the annotated data is to the true values. It helps us measure how closely the labels reflect real-world scenarios. Precision indicates the consistency of annotations. It checks if you are giving the same label to the same object or feature throughout the dataset. High accuracy and precision lead to better-trained models by reducing noise and improving the model's ability to generalize from the training data. + ++ +
+ +#### Identifying Outliers + +Outliers are data points that deviate quite a bit from other observations in the dataset. With respect to annotations, an outlier could be an incorrectly labeled image or an annotation that doesn't fit with the rest of the dataset. Outliers are concerning because they can distort the model's learning process, leading to inaccurate predictions and poor generalization. + +You can use various methods to detect and correct outliers: + +- **Statistical Techniques**: To detect outliers in numerical features like pixel values, bounding box coordinates, or object sizes, you can use methods such as box plots, histograms, or z-scores. +- **Visual Techniques**: To spot anomalies in categorical features like object classes, colors, or shapes, use visual methods like plotting images, labels, or heat maps. +- **Algorithmic Methods**: Use tools like clustering (e.g., K-means clustering, DBSCAN) and anomaly detection algorithms to identify outliers based on data distribution patterns. + +#### Quality Control of Annotated Data + +Just like other technical projects, quality control is a must for annotated data. It is a good practice to regularly check annotations to make sure they are accurate and consistent. This can be done in a few different ways: + +- Reviewing samples of annotated data +- Using automated tools to spot common errors +- Having another person double-check the annotations + +If you are working with multiple people, consistency between different annotators is important. Good inter-annotator agreement means that the guidelines are clear and everyone is following them the same way. It keeps everyone on the same page and the annotations consistent. + +While reviewing, if you find errors, correct them and update the guidelines to avoid future mistakes. Provide feedback to annotators and offer regular training to help reduce errors. Having a strong process for handling errors keeps your dataset accurate and reliable. + +## Share Your Thoughts with the Community + +Bouncing your ideas and queries off other computer vision enthusiasts can help accelerate your projects. Here are some great ways to learn, troubleshoot, and network: + +### Where to Find Help and Support + +- **GitHub Issues:** Visit the YOLOv8 GitHub repository and use the [Issues tab](https://github.com/ultralytics/ultralytics/issues) to raise questions, report bugs, and suggest features. The community and maintainers are there to help with any issues you face. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to connect with other users and developers, get support, share knowledge, and brainstorm ideas. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Refer to the [official YOLOv8 documentation](./index.md) for thorough guides and valuable insights on numerous computer vision tasks and projects. + +## Conclusion + +By following the best practices for collecting and annotating data, avoiding bias, and using the right tools and techniques, you can significantly improve your model's performance. Engaging with the community and using available resources will keep you informed and help you troubleshoot issues effectively. Remember, quality data is the foundation of a successful project, and the right strategies will help you build robust and reliable models. + +## FAQ + +### What is the best way to avoid bias in data collection for computer vision projects? + +Avoiding bias in data collection ensures that your computer vision model performs well across various scenarios. To minimize bias, consider collecting data from diverse sources to capture different perspectives and scenarios. Ensure balanced representation among all relevant groups, such as different ages, genders, and ethnicities. Regularly review and update your dataset to identify and address any emerging biases. Techniques such as oversampling underrepresented classes, data augmentation, and fairness-aware algorithms can also help mitigate bias. By employing these strategies, you maintain a robust and fair dataset that enhances your model's generalization capability. + +### How can I ensure high consistency and accuracy in data annotation? + +Ensuring high consistency and accuracy in data annotation involves establishing clear and objective labeling guidelines. Your instructions should be detailed, with examples and illustrations to clarify expectations. Consistency is achieved by setting standard criteria for annotating various data types, ensuring all annotations follow the same rules. To reduce personal biases, train annotators to stay neutral and objective. Regular reviews and updates of labeling rules help maintain accuracy and alignment with project goals. Using automated tools to check for consistency and getting feedback from other annotators also contribute to maintaining high-quality annotations. + +### How many images do I need for training Ultralytics YOLO models? + +For effective transfer learning and object detection with Ultralytics YOLO models, start with a minimum of a few hundred annotated objects per class. If training for just one class, begin with at least 100 annotated images and train for approximately 100 epochs. More complex tasks might require thousands of images per class to achieve high reliability and performance. Quality annotations are crucial, so ensure your data collection and annotation processes are rigorous and aligned with your project's specific goals. Explore detailed training strategies in the [YOLOv8 training guide](../modes/train.md). + +### What are some popular tools for data annotation? + +Several popular open-source tools can streamline the data annotation process: + +- **[Label Studio](https://github.com/HumanSignal/label-studio)**: A flexible tool supporting various annotation tasks, project management, and quality control features. +- **[CVAT](https://www.cvat.ai/)**: Offers multiple annotation formats and customizable workflows, making it suitable for complex projects. +- **[Labelme](https://github.com/labelmeai/labelme)**: Ideal for quick and straightforward image annotation with polygons. + +These tools can help enhance the efficiency and accuracy of your annotation workflows. For extensive feature lists and guides, refer to our [data annotation tools documentation](../datasets/index.md). + +### What types of data annotation are commonly used in computer vision? + +Different types of data annotation cater to various computer vision tasks: + +- **Bounding Boxes**: Used primarily for object detection, these are rectangular boxes around objects in an image. +- **Polygons**: Provide more precise object outlines suitable for instance segmentation tasks. +- **Masks**: Offer pixel-level detail, used in semantic segmentation to differentiate objects from the background. +- **Keypoints**: Identify specific points of interest within an image, useful for tasks like pose estimation and facial landmark detection. + +Selecting the appropriate annotation type depends on your project's requirements. Learn more about how to implement these annotations and their formats in our [data annotation guide](#what-is-data-annotation). diff --git a/ultralytics/docs/en/guides/deepstream-nvidia-jetson.md b/ultralytics/docs/en/guides/deepstream-nvidia-jetson.md new file mode 100644 index 0000000000000000000000000000000000000000..20103a6c2d5af9f952f992d3d0ecaa19f19dfb8c --- /dev/null +++ b/ultralytics/docs/en/guides/deepstream-nvidia-jetson.md @@ -0,0 +1,341 @@ +--- +comments: true +description: Learn how to deploy Ultralytics YOLOv8 on NVIDIA Jetson devices using TensorRT and DeepStream SDK. Explore performance benchmarks and maximize AI capabilities. +keywords: Ultralytics, YOLOv8, NVIDIA Jetson, JetPack, AI deployment, embedded systems, deep learning, TensorRT, DeepStream SDK, computer vision +--- + +# Ultralytics YOLOv8 on NVIDIA Jetson using DeepStream SDK and TensorRT + +This comprehensive guide provides a detailed walkthrough for deploying Ultralytics YOLOv8 on [NVIDIA Jetson](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/) devices using DeepStream SDK and TensorRT. Here we use TensorRT to maximize the inference performance on the Jetson platform. + + + +!!! Note + + This guide has been tested with both [Seeed Studio reComputer J4012](https://www.seeedstudio.com/reComputer-J4012-p-5586.html) which is based on NVIDIA Jetson Orin NX 16GB running JetPack release of [JP5.1.3](https://developer.nvidia.com/embedded/jetpack-sdk-513) and [Seeed Studio reComputer J1020 v2](https://www.seeedstudio.com/reComputer-J1020-v2-p-5498.html) which is based on NVIDIA Jetson Nano 4GB running JetPack release of [JP4.6.4](https://developer.nvidia.com/jetpack-sdk-464). It is expected to work across all the NVIDIA Jetson hardware lineup including latest and legacy. + +## What is NVIDIA DeepStream? + +[NVIDIA's DeepStream SDK](https://developer.nvidia.com/deepstream-sdk) is a complete streaming analytics toolkit based on GStreamer for AI-based multi-sensor processing, video, audio, and image understanding. It's ideal for vision AI developers, software partners, startups, and OEMs building IVA (Intelligent Video Analytics) apps and services. You can now create stream-processing pipelines that incorporate neural networks and other complex processing tasks like tracking, video encoding/decoding, and video rendering. These pipelines enable real-time analytics on video, image, and sensor data. DeepStream's multi-platform support gives you a faster, easier way to develop vision AI applications and services on-premise, at the edge, and in the cloud. + +## Prerequisites + +Before you start to follow this guide: + +- Visit our documentation, [Quick Start Guide: NVIDIA Jetson with Ultralytics YOLOv8](nvidia-jetson.md) to set up your NVIDIA Jetson device with Ultralytics YOLOv8 +- Install [DeepStream SDK](https://developer.nvidia.com/deepstream-getting-started) according to the JetPack version + + - For JetPack 4.6.4, install [DeepStream 6.0.1](https://docs.nvidia.com/metropolis/deepstream/6.0.1/dev-guide/text/DS_Quickstart.html) + - For JetPack 5.1.3, install [DeepStream 6.3](https://docs.nvidia.com/metropolis/deepstream/6.3/dev-guide/text/DS_Quickstart.html) + +!!! Tip + + In this guide we have used the Debian package method of installing DeepStream SDK to the Jetson device. You can also visit the [DeepStream SDK on Jetson (Archived)](https://developer.nvidia.com/embedded/deepstream-on-jetson-downloads-archived) to access legacy versions of DeepStream. + +## DeepStream Configuration for YOLOv8 + +Here we are using [marcoslucianops/DeepStream-Yolo](https://github.com/marcoslucianops/DeepStream-Yolo) GitHub repository which includes NVIDIA DeepStream SDK support for YOLO models. We appreciate the efforts of marcoslucianops for his contributions! + +1. Install dependencies + + ```bash + pip install cmake + pip install onnxsim + ``` + +2. Clone the following repository + + ```bash + git clone https://github.com/marcoslucianops/DeepStream-Yolo + cd DeepStream-Yolo + ``` + +3. Download Ultralytics YOLOv8 detection model (.pt) of your choice from [YOLOv8 releases](https://github.com/ultralytics/assets/releases). Here we use [yolov8s.pt](https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8s.pt). + + ```bash + wget https://github.com/ultralytics/assets/releases/download/v8.2.0/yolov8s.pt + ``` + + !!! Note + + You can also use a [custom trained YOLOv8 model](https://docs.ultralytics.com/modes/train/). + +4. Convert model to ONNX + + ```bash + python3 utils/export_yoloV8.py -w yolov8s.pt + ``` + + !!! Note "Pass the below arguments to the above command" + + For DeepStream 6.0.1, use opset 12 or lower. The default opset is 16. + + ```bash + --opset 12 + ``` + + To change the inference size (default: 640) + + ```bash + -s SIZE + --size SIZE + -s HEIGHT WIDTH + --size HEIGHT WIDTH + ``` + + Example for 1280: + + ```bash + -s 1280 + or + -s 1280 1280 + ``` + + To simplify the ONNX model (DeepStream >= 6.0) + + ```bash + --simplify + ``` + + To use dynamic batch-size (DeepStream >= 6.1) + + ```bash + --dynamic + ``` + + To use static batch-size (example for batch-size = 4) + + ```bash + --batch 4 + ``` + +5. Set the CUDA version according to the JetPack version installed + + For JetPack 4.6.4: + + ```bash + export CUDA_VER=10.2 + ``` + + For JetPack 5.1.3: + + ```bash + export CUDA_VER=11.4 + ``` + +6. Compile the library + + ```bash + make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo + ``` + +7. Edit the `config_infer_primary_yoloV8.txt` file according to your model (for YOLOv8s with 80 classes) + + ```bash + [property] + ... + onnx-file=yolov8s.onnx + ... + num-detected-classes=80 + ... + ``` + +8. Edit the `deepstream_app_config` file + + ```bash + ... + [primary-gie] + ... + config-file=config_infer_primary_yoloV8.txt + ``` + +9. You can also change the video source in `deepstream_app_config` file. Here a default video file is loaded + + ```bash + ... + [source0] + ... + uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 + ``` + +### Run Inference + +```bash +deepstream-app -c deepstream_app_config.txt +``` + +!!! Note + + It will take a long time to generate the TensorRT engine file before starting the inference. So please be patient. + + + +!!! Tip + + If you want to convert the model to FP16 precision, simply set `model-engine-file=model_b1_gpu0_fp16.engine` and `network-mode=2` inside `config_infer_primary_yoloV8.txt` + +## INT8 Calibration + +If you want to use INT8 precision for inference, you need to follow the steps below + +1. Set `OPENCV` environment variable + + ```bash + export OPENCV=1 + ``` + +2. Compile the library + + ```bash + make -C nvdsinfer_custom_impl_Yolo clean && make -C nvdsinfer_custom_impl_Yolo + ``` + +3. For COCO dataset, download the [val2017](http://images.cocodataset.org/zips/val2017.zip), extract, and move to `DeepStream-Yolo` folder + +4. Make a new directory for calibration images + + ```bash + mkdir calibration + ``` + +5. Run the following to select 1000 random images from COCO dataset to run calibration + + ```bash + for jpg in $(ls -1 val2017/*.jpg | sort -R | head -1000); do \ + cp ${jpg} calibration/; \ + done + ``` + + !!! Note + + NVIDIA recommends at least 500 images to get a good accuracy. On this example, 1000 images are chosen to get better accuracy (more images = more accuracy). You can set it from **head -1000**. For example, for 2000 images, **head -2000**. This process can take a long time. + +6. Create the `calibration.txt` file with all selected images + + ```bash + realpath calibration/*jpg > calibration.txt + ``` + +7. Set environment variables + + ```bash + export INT8_CALIB_IMG_PATH=calibration.txt + export INT8_CALIB_BATCH_SIZE=1 + ``` + + !!! Note + + Higher INT8_CALIB_BATCH_SIZE values will result in more accuracy and faster calibration speed. Set it according to you GPU memory. + +8. Update the `config_infer_primary_yoloV8.txt` file + + From + + ```bash + ... + model-engine-file=model_b1_gpu0_fp32.engine + #int8-calib-file=calib.table + ... + network-mode=0 + ... + ``` + + To + + ```bash + ... + model-engine-file=model_b1_gpu0_int8.engine + int8-calib-file=calib.table + ... + network-mode=1 + ... + ``` + +### Run Inference + +```bash +deepstream-app -c deepstream_app_config.txt +``` + +## MultiStream Setup + +To set up multiple streams under a single deepstream application, you can do the following changes to the `deepstream_app_config.txt` file + +1. Change the rows and columns to build a grid display according to the number of streams you want to have. For example, for 4 streams, we can add 2 rows and 2 columns. + + ```bash + [tiled-display] + rows=2 + columns=2 + ``` + +2. Set `num-sources=4` and add `uri` of all the 4 streams + + ```bash + [source0] + enable=1 + type=3 + uri=+ +
+ +Primary users include traffic management authorities and law enforcement, while secondary stakeholders are highway planners and the public benefiting from safer roads. Key requirements involve evaluating budget, time, and personnel, as well as addressing technical needs like high-resolution cameras and real-time data processing. Additionally, regulatory constraints on privacy and data security must be considered. + +### Setting Measurable Objectives + +Setting measurable objectives is key to the success of a computer vision project. These goals should be clear, achievable, and time-bound. + +For example, if you are developing a system to estimate vehicle speeds on a highway. You could consider the following measurable objectives: + +- To achieve at least 95% accuracy in speed detection within six months, using a dataset of 10,000 vehicle images. +- The system should be able to process real-time video feeds at 30 frames per second with minimal delay. + +By setting specific and quantifiable goals, you can effectively track progress, identify areas for improvement, and ensure the project stays on course. + +## The Connection Between The Problem Statement and The Computer Vision Tasks + +Your problem statement helps you conceptualize which computer vision task can solve your issue. + +For example, if your problem is monitoring vehicle speeds on a highway, the relevant computer vision task is object tracking. [Object tracking](../modes/track.md) is suitable because it allows the system to continuously follow each vehicle in the video feed, which is crucial for accurately calculating their speeds. + ++ +
+ +Other tasks, like [object detection](../tasks/detect.md), are not suitable as they don't provide continuous location or movement information. Once you've identified the appropriate computer vision task, it guides several critical aspects of your project, like model selection, dataset preparation, and model training approaches. + +## Which Comes First: Model Selection, Dataset Preparation, or Model Training Approach? + +The order of model selection, dataset preparation, and training approach depends on the specifics of your project. Here are a few tips to help you decide: + +- **Clear Understanding of the Problem**: If your problem and objectives are well-defined, start with model selection. Then, prepare your dataset and decide on the training approach based on the model's requirements. + + - **Example**: Start by selecting a model for a traffic monitoring system that estimates vehicle speeds. Choose an object tracking model, gather and annotate highway videos, and then train the model with techniques for real-time video processing. + +- **Unique or Limited Data**: If your project is constrained by unique or limited data, begin with dataset preparation. For instance, if you have a rare dataset of medical images, annotate and prepare the data first. Then, select a model that performs well on such data, followed by choosing a suitable training approach. + + - **Example**: Prepare the data first for a facial recognition system with a small dataset. Annotate it, then select a model that works well with limited data, such as a pre-trained model for transfer learning. Finally, decide on a training approach, including data augmentation, to expand the dataset. + +- **Need for Experimentation**: In projects where experimentation is crucial, start with the training approach. This is common in research projects where you might initially test different training techniques. Refine your model selection after identifying a promising method and prepare the dataset based on your findings. + - **Example**: In a project exploring new methods for detecting manufacturing defects, start with experimenting on a small data subset. Once you find a promising technique, select a model tailored to those findings and prepare a comprehensive dataset. + +## Common Discussion Points in the Community + +Next, let's look at a few common discussion points in the community regarding computer vision tasks and project planning. + +### What Are the Different Computer Vision Tasks? + +The most popular computer vision tasks include image classification, object detection, and image segmentation. + ++ +
+ +For a detailed explanation of various tasks, please take a look at the Ultralytics Docs page on [YOLOv8 Tasks](../tasks/index.md). + +### Can a Pre-trained Model Remember Classes It Knew Before Custom Training? + +No, pre-trained models don't "remember" classes in the traditional sense. They learn patterns from massive datasets, and during custom training (fine-tuning), these patterns are adjusted for your specific task. The model's capacity is limited, and focusing on new information can overwrite some previous learnings. + ++ +
+ +If you want to use the classes the model was pre-trained on, a practical approach is to use two models: one retains the original performance, and the other is fine-tuned for your specific task. This way, you can combine the outputs of both models. There are other options like freezing layers, using the pre-trained model as a feature extractor, and task-specific branching, but these are more complex solutions and require more expertise. + +### How Do Deployment Options Affect My Computer Vision Project? + +[Model deployment options](./model-deployment-options.md) critically impact the performance of your computer vision project. For instance, the deployment environment must handle the computational load of your model. Here are some practical examples: + +- **Edge Devices**: Deploying on edge devices like smartphones or IoT devices requires lightweight models due to their limited computational resources. Example technologies include [TensorFlow Lite](../integrations/tflite.md) and [ONNX Runtime](../integrations/onnx.md), which are optimized for such environments. +- **Cloud Servers**: Cloud deployments can handle more complex models with larger computational demands. Cloud platforms like [AWS](../integrations/amazon-sagemaker.md), Google Cloud, and Azure offer robust hardware options that can scale based on the project's needs. +- **On-Premise Servers**: For scenarios requiring high data privacy and security, deploying on-premise might be necessary. This involves significant upfront hardware investment but allows full control over the data and infrastructure. +- **Hybrid Solutions**: Some projects might benefit from a hybrid approach, where some processing is done on the edge, while more complex analyses are offloaded to the cloud. This can balance performance needs with cost and latency considerations. + +Each deployment option offers different benefits and challenges, and the choice depends on specific project requirements like performance, cost, and security. + +## Connecting with the Community + +Connecting with other computer vision enthusiasts can be incredibly helpful for your projects by providing support, solutions, and new ideas. Here are some great ways to learn, troubleshoot, and network: + +### Community Support Channels + +- **GitHub Issues:** Head over to the YOLOv8 GitHub repository. You can use the [Issues tab](https://github.com/ultralytics/ultralytics/issues) to raise questions, report bugs, and suggest features. The community and maintainers can assist with specific problems you encounter. +- **Ultralytics Discord Server:** Become part of the [Ultralytics Discord server](https://ultralytics.com/discord/). Connect with fellow users and developers, seek support, exchange knowledge, and discuss ideas. + +### Comprehensive Guides and Documentation + +- **Ultralytics YOLOv8 Documentation:** Explore the [official YOLOv8 documentation](./index.md) for in-depth guides and valuable tips on various computer vision tasks and projects. + +## Conclusion + +Defining a clear problem and setting measurable goals is key to a successful computer vision project. We've highlighted the importance of being clear and focused from the start. Having specific goals helps avoid oversight. Also, staying connected with others in the community through platforms like GitHub or Discord is important for learning and staying current. In short, good planning and engaging with the community is a huge part of successful computer vision projects. + +## FAQ + +### How do I define a clear problem statement for my Ultralytics computer vision project? + +To define a clear problem statement for your Ultralytics computer vision project, follow these steps: + +1. **Identify the Core Issue:** Pinpoint the specific challenge your project aims to solve. +2. **Determine the Scope:** Clearly outline the boundaries of your problem. +3. **Consider End Users and Stakeholders:** Identify who will be affected by your solution. +4. **Analyze Project Requirements and Constraints:** Assess available resources and any technical or regulatory limitations. + +Providing a well-defined problem statement ensures that the project remains focused and aligned with your objectives. For a detailed guide, refer to our [practical guide](#defining-a-clear-problem-statement). + +### Why should I use Ultralytics YOLOv8 for speed estimation in my computer vision project? + +Ultralytics YOLOv8 is ideal for speed estimation because of its real-time object tracking capabilities, high accuracy, and robust performance in detecting and monitoring vehicle speeds. It overcomes inefficiencies and inaccuracies of traditional radar systems by leveraging cutting-edge computer vision technology. Check out our blog on [speed estimation using YOLOv8](https://www.ultralytics.com/blog/ultralytics-yolov8-for-speed-estimation-in-computer-vision-projects) for more insights and practical examples. + +### How do I set effective measurable objectives for my computer vision project with Ultralytics YOLOv8? + +Set effective and measurable objectives using the SMART criteria: + +- **Specific:** Define clear and detailed goals. +- **Measurable:** Ensure objectives are quantifiable. +- **Achievable:** Set realistic targets within your capabilities. +- **Relevant:** Align objectives with your overall project goals. +- **Time-bound:** Set deadlines for each objective. + +For example, "Achieve 95% accuracy in speed detection within six months using a 10,000 vehicle image dataset." This approach helps track progress and identifies areas for improvement. Read more about [setting measurable objectives](#setting-measurable-objectives). + +### How do deployment options affect the performance of my Ultralytics YOLO models? + +Deployment options critically impact the performance of your Ultralytics YOLO models. Here are key options: + +- **Edge Devices:** Use lightweight models like TensorFlow Lite or ONNX Runtime for deployment on devices with limited resources. +- **Cloud Servers:** Utilize robust cloud platforms like AWS, Google Cloud, or Azure for handling complex models. +- **On-Premise Servers:** High data privacy and security needs may require on-premise deployments. +- **Hybrid Solutions:** Combine edge and cloud approaches for balanced performance and cost-efficiency. + +For more information, refer to our [detailed guide on model deployment options](./model-deployment-options.md). + +### What are the most common challenges in defining the problem for a computer vision project with Ultralytics? + +Common challenges include: + +- Vague or overly broad problem statements. +- Unrealistic objectives. +- Lack of stakeholder alignment. +- Insufficient understanding of technical constraints. +- Underestimating data requirements. + +Address these challenges through thorough initial research, clear communication with stakeholders, and iterative refinement of the problem statement and objectives. Learn more about these challenges in our [Computer Vision Project guide](steps-of-a-cv-project.md). diff --git a/ultralytics/docs/en/guides/distance-calculation.md b/ultralytics/docs/en/guides/distance-calculation.md new file mode 100644 index 0000000000000000000000000000000000000000..b756c6004f33876d614688ee8331ac5219406938 --- /dev/null +++ b/ultralytics/docs/en/guides/distance-calculation.md @@ -0,0 +1,138 @@ +--- +comments: true +description: Learn how to calculate distances between objects using Ultralytics YOLOv8 for accurate spatial positioning and scene understanding. +keywords: Ultralytics, YOLOv8, distance calculation, computer vision, object tracking, spatial positioning +--- + +# Distance Calculation using Ultralytics YOLOv8 + +## What is Distance Calculation? + +Measuring the gap between two objects is known as distance calculation within a specified space. In the case of [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics), the bounding box centroid is employed to calculate the distance for bounding boxes highlighted by the user. + +
+
+
+
+ Watch: Distance Calculation using Ultralytics YOLOv8
+
+ +
+ +This guide serves as a comprehensive introduction to setting up a Docker environment for your Ultralytics projects. [Docker](https://docker.com/) is a platform for developing, shipping, and running applications in containers. It is particularly beneficial for ensuring that the software will always run the same, regardless of where it's deployed. For more details, visit the Ultralytics Docker repository on [Docker Hub](https://hub.docker.com/r/ultralytics/ultralytics). + +[![Docker Image Version](https://img.shields.io/docker/v/ultralytics/ultralytics?sort=semver&logo=docker)](https://hub.docker.com/r/ultralytics/ultralytics) +[![Docker Pulls](https://img.shields.io/docker/pulls/ultralytics/ultralytics)](https://hub.docker.com/r/ultralytics/ultralytics) + +## What You Will Learn + +- Setting up Docker with NVIDIA support +- Installing Ultralytics Docker images +- Running Ultralytics in a Docker container with CPU or GPU support +- Using a Display Server with Docker to Show Ultralytics Detection Results +- Mounting local directories into the container + +--- + +## Prerequisites + +- Make sure Docker is installed on your system. If not, you can download and install it from [Docker's website](https://www.docker.com/products/docker-desktop). +- Ensure that your system has an NVIDIA GPU and NVIDIA drivers are installed. + +--- + +## Setting up Docker with NVIDIA Support + +First, verify that the NVIDIA drivers are properly installed by running: + +```bash +nvidia-smi +``` + +### Installing NVIDIA Docker Runtime + +Now, let's install the NVIDIA Docker runtime to enable GPU support in Docker containers: + +```bash +# Add NVIDIA package repositories +curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - +distribution=$(lsb_release -cs) +curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list + +# Install NVIDIA Docker runtime +sudo apt-get update +sudo apt-get install -y nvidia-docker2 + +# Restart Docker service to apply changes +sudo systemctl restart docker +``` + +### Verify NVIDIA Runtime with Docker + +Run `docker info | grep -i runtime` to ensure that `nvidia` appears in the list of runtimes: + +```bash +docker info | grep -i runtime +``` + +--- + +## Installing Ultralytics Docker Images + +Ultralytics offers several Docker images optimized for various platforms and use-cases: + +- **Dockerfile:** GPU image, ideal for training. +- **Dockerfile-arm64:** For ARM64 architecture, suitable for devices like [Raspberry Pi](raspberry-pi.md). +- **Dockerfile-cpu:** CPU-only version for inference and non-GPU environments. +- **Dockerfile-jetson:** Optimized for NVIDIA Jetson devices. +- **Dockerfile-python:** Minimal Python environment for lightweight applications. +- **Dockerfile-conda:** Includes [Miniconda3](https://docs.conda.io/projects/miniconda/en/latest/) and Ultralytics package installed via Conda. + +To pull the latest image: + +```bash +# Set image name as a variable +t=ultralytics/ultralytics:latest + +# Pull the latest Ultralytics image from Docker Hub +sudo docker pull $t +``` + +--- + +## Running Ultralytics in Docker Container + +Here's how to execute the Ultralytics Docker container: + +### Using only the CPU + +```bash +# Run with all GPUs +sudo docker run -it --ipc=host $t +``` + +### Using GPUs + +```bash +# Run with all GPUs +sudo docker run -it --ipc=host --gpus all $t + +# Run specifying which GPUs to use +sudo docker run -it --ipc=host --gpus '"device=2,3"' $t +``` + +The `-it` flag assigns a pseudo-TTY and keeps stdin open, allowing you to interact with the container. The `--ipc=host` flag enables sharing of host's IPC namespace, essential for sharing memory between processes. The `--gpus` flag allows the container to access the host's GPUs. + +## Running Ultralytics in Docker Container + +Here's how to execute the Ultralytics Docker container: + +### Using only the CPU + +```bash +# Run with all GPUs +sudo docker run -it --ipc=host $t +``` + +### Using GPUs + +```bash +# Run with all GPUs +sudo docker run -it --ipc=host --gpus all $t + +# Run specifying which GPUs to use +sudo docker run -it --ipc=host --gpus '"device=2,3"' $t +``` + +The `-it` flag assigns a pseudo-TTY and keeps stdin open, allowing you to interact with the container. The `--ipc=host` flag enables sharing of host's IPC namespace, essential for sharing memory between processes. The `--gpus` flag allows the container to access the host's GPUs. + +### Note on File Accessibility + +To work with files on your local machine within the container, you can use Docker volumes: + +```bash +# Mount a local directory into the container +sudo docker run -it --ipc=host --gpus all -v /path/on/host:/path/in/container $t +``` + +Replace `/path/on/host` with the directory path on your local machine and `/path/in/container` with the desired path inside the Docker container. + +## Run graphical user interface (GUI) applications in a Docker Container + +!!! danger "Highly Experimental - User Assumes All Risk" + + The following instructions are experimental. Sharing a X11 socket with a Docker container poses potential security risks. Therefore, it's recommended to test this solution only in a controlled environment. For more information, refer to these resources on how to use `xhost`[(1)](http://users.stat.umn.edu/~geyer/secure.html)[(2)](https://linux.die.net/man/1/xhost). + +Docker is primarily used to containerize background applications and CLI programs, but it can also run graphical programs. In the Linux world, two main graphic servers handle graphical display: [X11](https://www.x.org/wiki/) (also known as the X Window System) and [Wayland](https://wayland.freedesktop.org/). Before starting, it's essential to determine which graphics server you are currently using. Run this command to find out: + +```bash +env | grep -E -i 'x11|xorg|wayland' +``` + +Setup and configuration of an X11 or Wayland display server is outside the scope of this guide. If the above command returns nothing, then you'll need to start by getting either working for your system before continuing. + +### Running a Docker Container with a GUI + +!!! example + + ??? info "Use GPUs" + If you're using [GPUs](#using-gpus), you can add the `--gpus all` flag to the command. + + === "X11" + + If you're using X11, you can run the following command to allow the Docker container to access the X11 socket: + + ```bash + xhost +local:docker && docker run -e DISPLAY=$DISPLAY \ + -v /tmp/.X11-unix:/tmp/.X11-unix \ + -v ~/.Xauthority:/root/.Xauthority \ + -it --ipc=host $t + ``` + + This command sets the `DISPLAY` environment variable to the host's display, mounts the X11 socket, and maps the `.Xauthority` file to the container. The `xhost +local:docker` command allows the Docker container to access the X11 server. + + + === "Wayland" + + For Wayland, use the following command: + + ```bash + xhost +local:docker && docker run -e DISPLAY=$DISPLAY \ + -v $XDG_RUNTIME_DIR/$WAYLAND_DISPLAY:/tmp/$WAYLAND_DISPLAY \ + --net=host -it --ipc=host $t + ``` + + This command sets the `DISPLAY` environment variable to the host's display, mounts the Wayland socket, and allows the Docker container to access the Wayland server. + +### Using Docker with a GUI + +Now you can display graphical applications inside your Docker container. For example, you can run the following [CLI command](../usage/cli.md) to visualize the [predictions](../modes/predict.md) from a [YOLOv8 model](../models/yolov8.md): + +```bash +yolo predict model=yolov8n.pt show=True +``` + +??? info "Testing" + + A simple way to validate that the Docker group has access to the X11 server is to run a container with a GUI program like [`xclock`](https://www.x.org/archive/X11R6.8.1/doc/xclock.1.html) or [`xeyes`](https://www.x.org/releases/X11R7.5/doc/man/man1/xeyes.1.html). Alternatively, you can also install these programs in the Ultralytics Docker container to test the access to the X11 server of your GNU-Linux display server. If you run into any problems, consider setting the environment variable `-e QT_DEBUG_PLUGINS=1`. Setting this environment variable enables the output of debugging information, aiding in the troubleshooting process. + +### When finished with Docker GUI + +!!! warning "Revoke access" + + In both cases, don't forget to revoke access from the Docker group when you're done. + + ```bash + xhost -local:docker + ``` + +??? question "Want to view image results directly in the Terminal?" + + Refer to the following guide on [viewing the image results using a terminal](./view-results-in-terminal.md) + +--- + +Congratulations! You're now set up to use Ultralytics with Docker and ready to take advantage of its powerful capabilities. For alternate installation methods, feel free to explore the [Ultralytics quickstart documentation](../quickstart.md). + +## FAQ + +### How do I set up Ultralytics with Docker? + +To set up Ultralytics with Docker, first ensure that Docker is installed on your system. If you have an NVIDIA GPU, install the NVIDIA Docker runtime to enable GPU support. Then, pull the latest Ultralytics Docker image from Docker Hub using the following command: + +```bash +sudo docker pull ultralytics/ultralytics:latest +``` + +For detailed steps, refer to our [Docker Quickstart Guide](../quickstart.md). + +### What are the benefits of using Ultralytics Docker images for machine learning projects? + +Using Ultralytics Docker images ensures a consistent environment across different machines, replicating the same software and dependencies. This is particularly useful for collaborating across teams, running models on various hardware, and maintaining reproducibility. For GPU-based training, Ultralytics provides optimized Docker images such as `Dockerfile` for general GPU usage and `Dockerfile-jetson` for NVIDIA Jetson devices. Explore [Ultralytics Docker Hub](https://hub.docker.com/r/ultralytics/ultralytics) for more details. + +### How can I run Ultralytics YOLO in a Docker container with GPU support? + +First, ensure that the NVIDIA Docker runtime is installed and configured. Then, use the following command to run Ultralytics YOLO with GPU support: + +```bash +sudo docker run -it --ipc=host --gpus all ultralytics/ultralytics:latest +``` + +This command sets up a Docker container with GPU access. For additional details, see the [Docker Quickstart Guide](../quickstart.md). + +### How do I visualize YOLO prediction results in a Docker container with a display server? + +To visualize YOLO prediction results with a GUI in a Docker container, you need to allow Docker to access your display server. For systems running X11, the command is: + +```bash +xhost +local:docker && docker run -e DISPLAY=$DISPLAY \ +-v /tmp/.X11-unix:/tmp/.X11-unix \ +-v ~/.Xauthority:/root/.Xauthority \ +-it --ipc=host ultralytics/ultralytics:latest +``` + +For systems running Wayland, use: + +```bash +xhost +local:docker && docker run -e DISPLAY=$DISPLAY \ +-v $XDG_RUNTIME_DIR/$WAYLAND_DISPLAY:/tmp/$WAYLAND_DISPLAY \ +--net=host -it --ipc=host ultralytics/ultralytics:latest +``` + +More information can be found in the [Run graphical user interface (GUI) applications in a Docker Container](#run-graphical-user-interface-gui-applications-in-a-docker-container) section. + +### Can I mount local directories into the Ultralytics Docker container? + +Yes, you can mount local directories into the Ultralytics Docker container using the `-v` flag: + +```bash +sudo docker run -it --ipc=host --gpus all -v /path/on/host:/path/in/container ultralytics/ultralytics:latest +``` + +Replace `/path/on/host` with the directory on your local machine and `/path/in/container` with the desired path inside the container. This setup allows you to work with your local files within the container. For more information, refer to the relevant section on [mounting local directories](../usage/python.md). diff --git a/ultralytics/docs/en/guides/heatmaps.md b/ultralytics/docs/en/guides/heatmaps.md new file mode 100644 index 0000000000000000000000000000000000000000..efcf427bd93e1be60a16fc5b868f3fa9b5551c96 --- /dev/null +++ b/ultralytics/docs/en/guides/heatmaps.md @@ -0,0 +1,403 @@ +--- +comments: true +description: Transform complex data into insightful heatmaps using Ultralytics YOLOv8. Discover patterns, trends, and anomalies with vibrant visualizations. +keywords: Ultralytics, YOLOv8, heatmaps, data visualization, data analysis, complex data, patterns, trends, anomalies +--- + +# Advanced Data Visualization: Heatmaps using Ultralytics YOLOv8 🚀 + +## Introduction to Heatmaps + +A heatmap generated with [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics/) transforms complex data into a vibrant, color-coded matrix. This visual tool employs a spectrum of colors to represent varying data values, where warmer hues indicate higher intensities and cooler tones signify lower values. Heatmaps excel in visualizing intricate data patterns, correlations, and anomalies, offering an accessible and engaging approach to data interpretation across diverse domains. + +
+
+
+
+ Watch: Heatmaps using Ultralytics YOLOv8
+
+ +
+ +For a full list of augmentation hyperparameters used in YOLOv8 please refer to the [configurations page](../usage/cfg.md#augmentation-settings). + +### Genetic Evolution and Mutation + +Ultralytics YOLO uses genetic algorithms to optimize hyperparameters. Genetic algorithms are inspired by the mechanism of natural selection and genetics. + +- **Mutation**: In the context of Ultralytics YOLO, mutation helps in locally searching the hyperparameter space by applying small, random changes to existing hyperparameters, producing new candidates for evaluation. +- **Crossover**: Although crossover is a popular genetic algorithm technique, it is not currently used in Ultralytics YOLO for hyperparameter tuning. The focus is mainly on mutation for generating new hyperparameter sets. + +## Preparing for Hyperparameter Tuning + +Before you begin the tuning process, it's important to: + +1. **Identify the Metrics**: Determine the metrics you will use to evaluate the model's performance. This could be AP50, F1-score, or others. +2. **Set the Tuning Budget**: Define how much computational resources you're willing to allocate. Hyperparameter tuning can be computationally intensive. + +## Steps Involved + +### Initialize Hyperparameters + +Start with a reasonable set of initial hyperparameters. This could either be the default hyperparameters set by Ultralytics YOLO or something based on your domain knowledge or previous experiments. + +### Mutate Hyperparameters + +Use the `_mutate` method to produce a new set of hyperparameters based on the existing set. + +### Train Model + +Training is performed using the mutated set of hyperparameters. The training performance is then assessed. + +### Evaluate Model + +Use metrics like AP50, F1-score, or custom metrics to evaluate the model's performance. + +### Log Results + +It's crucial to log both the performance metrics and the corresponding hyperparameters for future reference. + +### Repeat + +The process is repeated until either the set number of iterations is reached or the performance metric is satisfactory. + +## Usage Example + +Here's how to use the `model.tune()` method to utilize the `Tuner` class for hyperparameter tuning of YOLOv8n on COCO8 for 30 epochs with an AdamW optimizer and skipping plotting, checkpointing and validation other than on final epoch for faster Tuning. + +!!! Example + + === "Python" + + ```python + from ultralytics import YOLO + + # Initialize the YOLO model + model = YOLO("yolov8n.pt") + + # Tune hyperparameters on COCO8 for 30 epochs + model.tune(data="coco8.yaml", epochs=30, iterations=300, optimizer="AdamW", plots=False, save=False, val=False) + ``` + +## Results + +After you've successfully completed the hyperparameter tuning process, you will obtain several files and directories that encapsulate the results of the tuning. The following describes each: + +### File Structure + +Here's what the directory structure of the results will look like. Training directories like `train1/` contain individual tuning iterations, i.e. one model trained with one set of hyperparameters. The `tune/` directory contains tuning results from all the individual model trainings: + +```plaintext +runs/ +└── detect/ + ├── train1/ + ├── train2/ + ├── ... + └── tune/ + ├── best_hyperparameters.yaml + ├── best_fitness.png + ├── tune_results.csv + ├── tune_scatter_plots.png + └── weights/ + ├── last.pt + └── best.pt +``` + +### File Descriptions + +#### best_hyperparameters.yaml + +This YAML file contains the best-performing hyperparameters found during the tuning process. You can use this file to initialize future trainings with these optimized settings. + +- **Format**: YAML +- **Usage**: Hyperparameter results +- **Example**: + + ```yaml + # 558/900 iterations complete ✅ (45536.81s) + # Results saved to /usr/src/ultralytics/runs/detect/tune + # Best fitness=0.64297 observed at iteration 498 + # Best fitness metrics are {'metrics/precision(B)': 0.87247, 'metrics/recall(B)': 0.71387, 'metrics/mAP50(B)': 0.79106, 'metrics/mAP50-95(B)': 0.62651, 'val/box_loss': 2.79884, 'val/cls_loss': 2.72386, 'val/dfl_loss': 0.68503, 'fitness': 0.64297} + # Best fitness model is /usr/src/ultralytics/runs/detect/train498 + # Best fitness hyperparameters are printed below. + + lr0: 0.00269 + lrf: 0.00288 + momentum: 0.73375 + weight_decay: 0.00015 + warmup_epochs: 1.22935 + warmup_momentum: 0.1525 + box: 18.27875 + cls: 1.32899 + dfl: 0.56016 + hsv_h: 0.01148 + hsv_s: 0.53554 + hsv_v: 0.13636 + degrees: 0.0 + translate: 0.12431 + scale: 0.07643 + shear: 0.0 + perspective: 0.0 + flipud: 0.0 + fliplr: 0.08631 + mosaic: 0.42551 + mixup: 0.0 + copy_paste: 0.0 + ``` + +#### best_fitness.png + +This is a plot displaying fitness (typically a performance metric like AP50) against the number of iterations. It helps you visualize how well the genetic algorithm performed over time. + +- **Format**: PNG +- **Usage**: Performance visualization + ++ +
+ +#### tune_results.csv + +A CSV file containing detailed results of each iteration during the tuning. Each row in the file represents one iteration, and it includes metrics like fitness score, precision, recall, as well as the hyperparameters used. + +- **Format**: CSV +- **Usage**: Per-iteration results tracking. +- **Example**: + ```csv + fitness,lr0,lrf,momentum,weight_decay,warmup_epochs,warmup_momentum,box,cls,dfl,hsv_h,hsv_s,hsv_v,degrees,translate,scale,shear,perspective,flipud,fliplr,mosaic,mixup,copy_paste + 0.05021,0.01,0.01,0.937,0.0005,3.0,0.8,7.5,0.5,1.5,0.015,0.7,0.4,0.0,0.1,0.5,0.0,0.0,0.0,0.5,1.0,0.0,0.0 + 0.07217,0.01003,0.00967,0.93897,0.00049,2.79757,0.81075,7.5,0.50746,1.44826,0.01503,0.72948,0.40658,0.0,0.0987,0.4922,0.0,0.0,0.0,0.49729,1.0,0.0,0.0 + 0.06584,0.01003,0.00855,0.91009,0.00073,3.42176,0.95,8.64301,0.54594,1.72261,0.01503,0.59179,0.40658,0.0,0.0987,0.46955,0.0,0.0,0.0,0.49729,0.80187,0.0,0.0 + ``` + +#### tune_scatter_plots.png + +This file contains scatter plots generated from `tune_results.csv`, helping you visualize relationships between different hyperparameters and performance metrics. Note that hyperparameters initialized to 0 will not be tuned, such as `degrees` and `shear` below. + +- **Format**: PNG +- **Usage**: Exploratory data analysis + ++ +
+ +#### weights/ + +This directory contains the saved PyTorch models for the last and the best iterations during the hyperparameter tuning process. + +- **`last.pt`**: The last.pt are the weights from the last epoch of training. +- **`best.pt`**: The best.pt weights for the iteration that achieved the best fitness score. + +Using these results, you can make more informed decisions for your future model trainings and analyses. Feel free to consult these artifacts to understand how well your model performed and how you might improve it further. + +## Conclusion + +The hyperparameter tuning process in Ultralytics YOLO is simplified yet powerful, thanks to its genetic algorithm-based approach focused on mutation. Following the steps outlined in this guide will assist you in systematically tuning your model to achieve better performance. + +### Further Reading + +1. [Hyperparameter Optimization in Wikipedia](https://en.wikipedia.org/wiki/Hyperparameter_optimization) +2. [YOLOv5 Hyperparameter Evolution Guide](../yolov5/tutorials/hyperparameter_evolution.md) +3. [Efficient Hyperparameter Tuning with Ray Tune and YOLOv8](../integrations/ray-tune.md) + +For deeper insights, you can explore the `Tuner` class source code and accompanying documentation. Should you have any questions, feature requests, or need further assistance, feel free to reach out to us on [GitHub](https://github.com/ultralytics/ultralytics/issues/new/choose) or [Discord](https://ultralytics.com/discord). + +## FAQ + +### How do I optimize the learning rate for Ultralytics YOLO during hyperparameter tuning? + +To optimize the learning rate for Ultralytics YOLO, start by setting an initial learning rate using the `lr0` parameter. Common values range from `0.001` to `0.01`. During the hyperparameter tuning process, this value will be mutated to find the optimal setting. You can utilize the `model.tune()` method to automate this process. For example: + +!!! Example + + === "Python" + + ```python + from ultralytics import YOLO + + # Initialize the YOLO model + model = YOLO("yolov8n.pt") + + # Tune hyperparameters on COCO8 for 30 epochs + model.tune(data="coco8.yaml", epochs=30, iterations=300, optimizer="AdamW", plots=False, save=False, val=False) + ``` + +For more details, check the [Ultralytics YOLO configuration page](../usage/cfg.md#augmentation-settings). + +### What are the benefits of using genetic algorithms for hyperparameter tuning in YOLOv8? + +Genetic algorithms in Ultralytics YOLOv8 provide a robust method for exploring the hyperparameter space, leading to highly optimized model performance. Key benefits include: + +- **Efficient Search**: Genetic algorithms like mutation can quickly explore a large set of hyperparameters. +- **Avoiding Local Minima**: By introducing randomness, they help in avoiding local minima, ensuring better global optimization. +- **Performance Metrics**: They adapt based on performance metrics such as AP50 and F1-score. + +To see how genetic algorithms can optimize hyperparameters, check out the [hyperparameter evolution guide](../yolov5/tutorials/hyperparameter_evolution.md). + +### How long does the hyperparameter tuning process take for Ultralytics YOLO? + +The time required for hyperparameter tuning with Ultralytics YOLO largely depends on several factors such as the size of the dataset, the complexity of the model architecture, the number of iterations, and the computational resources available. For instance, tuning YOLOv8n on a dataset like COCO8 for 30 epochs might take several hours to days, depending on the hardware. + +To effectively manage tuning time, define a clear tuning budget beforehand ([internal section link](#preparing-for-hyperparameter-tuning)). This helps in balancing resource allocation and optimization goals. + +### What metrics should I use to evaluate model performance during hyperparameter tuning in YOLO? + +When evaluating model performance during hyperparameter tuning in YOLO, you can use several key metrics: + +- **AP50**: The average precision at IoU threshold of 0.50. +- **F1-Score**: The harmonic mean of precision and recall. +- **Precision and Recall**: Individual metrics indicating the model's accuracy in identifying true positives versus false positives and false negatives. + +These metrics help you understand different aspects of your model's performance. Refer to the [Ultralytics YOLO performance metrics](../guides/yolo-performance-metrics.md) guide for a comprehensive overview. + +### Can I use Ultralytics HUB for hyperparameter tuning of YOLO models? + +Yes, you can use Ultralytics HUB for hyperparameter tuning of YOLO models. The HUB offers a no-code platform to easily upload datasets, train models, and perform hyperparameter tuning efficiently. It provides real-time tracking and visualization of tuning progress and results. + +Explore more about using Ultralytics HUB for hyperparameter tuning in the [Ultralytics HUB Cloud Training](../hub/cloud-training.md) documentation. diff --git a/ultralytics/docs/en/guides/index.md b/ultralytics/docs/en/guides/index.md new file mode 100644 index 0000000000000000000000000000000000000000..f05ee5950cf45e751be7dd32a5c2c827242434d3 --- /dev/null +++ b/ultralytics/docs/en/guides/index.md @@ -0,0 +1,104 @@ +--- +comments: true +description: Master YOLO with Ultralytics tutorials covering training, deployment and optimization. Find solutions, improve metrics, and deploy with ease!. +keywords: Ultralytics, YOLO, tutorials, guides, object detection, deep learning, PyTorch, training, deployment, optimization, computer vision +--- + +# Comprehensive Tutorials to Ultralytics YOLO + +Welcome to the Ultralytics' YOLO 🚀 Guides! Our comprehensive tutorials cover various aspects of the YOLO object detection model, ranging from training and prediction to deployment. Built on PyTorch, YOLO stands out for its exceptional speed and accuracy in real-time object detection tasks. + +Whether you're a beginner or an expert in deep learning, our tutorials offer valuable insights into the implementation and optimization of YOLO for your computer vision projects. Let's dive in! + +
+
+
+
+ Watch: Ultralytics YOLOv8 Guides Overview
+
+
+
+
+ Watch: Instance Segmentation with Object Tracking using Ultralytics YOLOv8
+
+ +
+ +## Recipe Walk Through + +1. See the [Ultralytics Quickstart Installation section](../quickstart.md) for a quick walkthrough on installing the required libraries. + + *** + +2. Load a model and run `predict()` method on a source. + + ```python + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n-seg.pt") + + # Run inference + results = model.predict() + ``` + + !!! question "No Prediction Arguments?" + + Without specifying a source, the example images from the library will be used: + + ``` + 'ultralytics/assets/bus.jpg' + 'ultralytics/assets/zidane.jpg' + ``` + + This is helpful for rapid testing with the `predict()` method. + + For additional information about Segmentation Models, visit the [Segment Task](../tasks/segment.md#models) page. To learn more about `predict()` method, see [Predict Mode](../modes/predict.md) section of the Documentation. + + *** + +3. Now iterate over the results and the contours. For workflows that want to save an image to file, the source image `base-name` and the detection `class-label` are retrieved for later use (optional). + + ```{ .py .annotate } + from pathlib import Path + + import numpy as np + + # (2) Iterate detection results (helpful for multiple images) + for r in res: + img = np.copy(r.orig_img) + img_name = Path(r.path).stem # source image base-name + + # Iterate each object contour (multiple detections) + for ci, c in enumerate(r): + # (1) Get detection class name + label = c.names[c.boxes.cls.tolist().pop()] + ``` + + 1. To learn more about working with detection results, see [Boxes Section for Predict Mode](../modes/predict.md#boxes). + 2. To learn more about `predict()` results see [Working with Results for Predict Mode](../modes/predict.md#working-with-results) + + ??? info "For-Loop" + + A single image will only iterate the first loop once. A single image with only a single detection will iterate each loop _only_ once. + + *** + +4. Start with generating a binary mask from the source image and then draw a filled contour onto the mask. This will allow the object to be isolated from the other parts of the image. An example from `bus.jpg` for one of the detected `person` class objects is shown on the right. + + ![Binary Mask Image](https://github.com/ultralytics/ultralytics/assets/62214284/59bce684-fdda-4b17-8104-0b4b51149aca){ width="240", align="right" } + + ```{ .py .annotate } + import cv2 + + # Create binary mask + b_mask = np.zeros(img.shape[:2], np.uint8) + + # (1) Extract contour result + contour = c.masks.xy.pop() + # (2) Changing the type + contour = contour.astype(np.int32) + # (3) Reshaping + contour = contour.reshape(-1, 1, 2) + + + # Draw contour onto mask + _ = cv2.drawContours(b_mask, [contour], -1, (255, 255, 255), cv2.FILLED) + ``` + + 1. For more info on `c.masks.xy` see [Masks Section from Predict Mode](../modes/predict.md#masks). + + 2. Here the values are cast into `np.int32` for compatibility with `drawContours()` function from OpenCV. + + 3. The OpenCV `drawContours()` function expects contours to have a shape of `[N, 1, 2]` expand section below for more details. + +contour
variable.+ + - `c.masks.xy` :: Provides the coordinates of the mask contour points in the format `(x, y)`. For more details, refer to the [Masks Section from Predict Mode](../modes/predict.md#masks). + + - `.pop()` :: As `masks.xy` is a list containing a single element, this element is extracted using the `pop()` method. + + - `.astype(np.int32)` :: Using `masks.xy` will return with a data type of `float32`, but this won't be compatible with the OpenCV `drawContours()` function, so this will change the data type to `int32` for compatibility. + + - `.reshape(-1, 1, 2)` :: Reformats the data into the required shape of `[N, 1, 2]` where `N` is the number of contour points, with each point represented by a single entry `1`, and the entry is composed of `2` values. The `-1` denotes that the number of values along this dimension is flexible. + +
drawContours()
configuration.+ + - Encapsulating the `contour` variable within square brackets, `[contour]`, was found to effectively generate the desired contour mask during testing. + + - The value `-1` specified for the `drawContours()` parameter instructs the function to draw all contours present in the image. + + - The `tuple` `(255, 255, 255)` represents the color white, which is the desired color for drawing the contour in this binary mask. + + - The addition of `cv2.FILLED` will color all pixels enclosed by the contour boundary the same, in this case, all enclosed pixels will be white. + + - See [OpenCV Documentation on `drawContours()`](https://docs.opencv.org/4.8.0/d6/d6e/group__imgproc__draw.html#ga746c0625f1781f1ffc9056259103edbc) for more information. + +
+ +
+ +Whether your project involves the Fruit Detection dataset or a custom data source, this tutorial aims to help you comprehend and apply K-Fold Cross Validation to bolster the reliability and robustness of your machine learning models. While we're applying `k=5` folds for this tutorial, keep in mind that the optimal number of folds can vary depending on your dataset and the specifics of your project. + +Without further ado, let's dive in! + +## Setup + +- Your annotations should be in the [YOLO detection format](../datasets/detect/index.md). + +- This guide assumes that annotation files are locally available. + +- For our demonstration, we use the [Fruit Detection](https://www.kaggle.com/datasets/lakshaytyagi01/fruit-detection/code) dataset. + - This dataset contains a total of 8479 images. + - It includes 6 class labels, each with its total instance counts listed below. + +| Class Label | Instance Count | +| :---------- | :------------: | +| Apple | 7049 | +| Grapes | 7202 | +| Pineapple | 1613 | +| Orange | 15549 | +| Banana | 3536 | +| Watermelon | 1976 | + +- Necessary Python packages include: + + - `ultralytics` + - `sklearn` + - `pandas` + - `pyyaml` + +- This tutorial operates with `k=5` folds. However, you should determine the best number of folds for your specific dataset. + +1. Initiate a new Python virtual environment (`venv`) for your project and activate it. Use `pip` (or your preferred package manager) to install: + + - The Ultralytics library: `pip install -U ultralytics`. Alternatively, you can clone the official [repo](https://github.com/ultralytics/ultralytics). + - Scikit-learn, pandas, and PyYAML: `pip install -U scikit-learn pandas pyyaml`. + +2. Verify that your annotations are in the [YOLO detection format](../datasets/detect/index.md). + + - For this tutorial, all annotation files are found in the `Fruit-Detection/labels` directory. + +## Generating Feature Vectors for Object Detection Dataset + +1. Start by creating a new `example.py` Python file for the steps below. + +2. Proceed to retrieve all label files for your dataset. + + ```python + from pathlib import Path + + dataset_path = Path("./Fruit-detection") # replace with 'path/to/dataset' for your custom data + labels = sorted(dataset_path.rglob("*labels/*.txt")) # all data in 'labels' + ``` + +3. Now, read the contents of the dataset YAML file and extract the indices of the class labels. + + ```python + yaml_file = "path/to/data.yaml" # your data YAML with data directories and names dictionary + with open(yaml_file, "r", encoding="utf8") as y: + classes = yaml.safe_load(y)["names"] + cls_idx = sorted(classes.keys()) + ``` + +4. Initialize an empty `pandas` DataFrame. + + ```python + import pandas as pd + + indx = [label.stem for label in labels] # uses base filename as ID (no extension) + labels_df = pd.DataFrame([], columns=cls_idx, index=indx) + ``` + +5. Count the instances of each class-label present in the annotation files. + + ```python + from collections import Counter + + for label in labels: + lbl_counter = Counter() + + with open(label, "r") as lf: + lines = lf.readlines() + + for line in lines: + # classes for YOLO label uses integer at first position of each line + lbl_counter[int(line.split(" ")[0])] += 1 + + labels_df.loc[label.stem] = lbl_counter + + labels_df = labels_df.fillna(0.0) # replace `nan` values with `0.0` + ``` + +6. The following is a sample view of the populated DataFrame: + + ```pandas + 0 1 2 3 4 5 + '0000a16e4b057580_jpg.rf.00ab48988370f64f5ca8ea4...' 0.0 0.0 0.0 0.0 0.0 7.0 + '0000a16e4b057580_jpg.rf.7e6dce029fb67f01eb19aa7...' 0.0 0.0 0.0 0.0 0.0 7.0 + '0000a16e4b057580_jpg.rf.bc4d31cdcbe229dd022957a...' 0.0 0.0 0.0 0.0 0.0 7.0 + '00020ebf74c4881c_jpg.rf.508192a0a97aa6c4a3b6882...' 0.0 0.0 0.0 1.0 0.0 0.0 + '00020ebf74c4881c_jpg.rf.5af192a2254c8ecc4188a25...' 0.0 0.0 0.0 1.0 0.0 0.0 + ... ... ... ... ... ... ... + 'ff4cd45896de38be_jpg.rf.c4b5e967ca10c7ced3b9e97...' 0.0 0.0 0.0 0.0 0.0 2.0 + 'ff4cd45896de38be_jpg.rf.ea4c1d37d2884b3e3cbce08...' 0.0 0.0 0.0 0.0 0.0 2.0 + 'ff5fd9c3c624b7dc_jpg.rf.bb519feaa36fc4bf630a033...' 1.0 0.0 0.0 0.0 0.0 0.0 + 'ff5fd9c3c624b7dc_jpg.rf.f0751c9c3aa4519ea3c9d6a...' 1.0 0.0 0.0 0.0 0.0 0.0 + 'fffe28b31f2a70d4_jpg.rf.7ea16bd637ba0711c53b540...' 0.0 6.0 0.0 0.0 0.0 0.0 + ``` + +The rows index the label files, each corresponding to an image in your dataset, and the columns correspond to your class-label indices. Each row represents a pseudo feature-vector, with the count of each class-label present in your dataset. This data structure enables the application of K-Fold Cross Validation to an object detection dataset. + +## K-Fold Dataset Split + +1. Now we will use the `KFold` class from `sklearn.model_selection` to generate `k` splits of the dataset. + + - Important: + - Setting `shuffle=True` ensures a randomized distribution of classes in your splits. + - By setting `random_state=M` where `M` is a chosen integer, you can obtain repeatable results. + + ```python + from sklearn.model_selection import KFold + + ksplit = 5 + kf = KFold(n_splits=ksplit, shuffle=True, random_state=20) # setting random_state for repeatable results + + kfolds = list(kf.split(labels_df)) + ``` + +2. The dataset has now been split into `k` folds, each having a list of `train` and `val` indices. We will construct a DataFrame to display these results more clearly. + + ```python + folds = [f"split_{n}" for n in range(1, ksplit + 1)] + folds_df = pd.DataFrame(index=indx, columns=folds) + + for idx, (train, val) in enumerate(kfolds, start=1): + folds_df[f"split_{idx}"].loc[labels_df.iloc[train].index] = "train" + folds_df[f"split_{idx}"].loc[labels_df.iloc[val].index] = "val" + ``` + +3. Now we will calculate the distribution of class labels for each fold as a ratio of the classes present in `val` to those present in `train`. + + ```python + fold_lbl_distrb = pd.DataFrame(index=folds, columns=cls_idx) + + for n, (train_indices, val_indices) in enumerate(kfolds, start=1): + train_totals = labels_df.iloc[train_indices].sum() + val_totals = labels_df.iloc[val_indices].sum() + + # To avoid division by zero, we add a small value (1E-7) to the denominator + ratio = val_totals / (train_totals + 1e-7) + fold_lbl_distrb.loc[f"split_{n}"] = ratio + ``` + + The ideal scenario is for all class ratios to be reasonably similar for each split and across classes. This, however, will be subject to the specifics of your dataset. + +4. Next, we create the directories and dataset YAML files for each split. + + ```python + import datetime + + supported_extensions = [".jpg", ".jpeg", ".png"] + + # Initialize an empty list to store image file paths + images = [] + + # Loop through supported extensions and gather image files + for ext in supported_extensions: + images.extend(sorted((dataset_path / "images").rglob(f"*{ext}"))) + + # Create the necessary directories and dataset YAML files (unchanged) + save_path = Path(dataset_path / f"{datetime.date.today().isoformat()}_{ksplit}-Fold_Cross-val") + save_path.mkdir(parents=True, exist_ok=True) + ds_yamls = [] + + for split in folds_df.columns: + # Create directories + split_dir = save_path / split + split_dir.mkdir(parents=True, exist_ok=True) + (split_dir / "train" / "images").mkdir(parents=True, exist_ok=True) + (split_dir / "train" / "labels").mkdir(parents=True, exist_ok=True) + (split_dir / "val" / "images").mkdir(parents=True, exist_ok=True) + (split_dir / "val" / "labels").mkdir(parents=True, exist_ok=True) + + # Create dataset YAML files + dataset_yaml = split_dir / f"{split}_dataset.yaml" + ds_yamls.append(dataset_yaml) + + with open(dataset_yaml, "w") as ds_y: + yaml.safe_dump( + { + "path": split_dir.as_posix(), + "train": "train", + "val": "val", + "names": classes, + }, + ds_y, + ) + ``` + +5. Lastly, copy images and labels into the respective directory ('train' or 'val') for each split. + + - **NOTE:** The time required for this portion of the code will vary based on the size of your dataset and your system hardware. + + ```python + import shutil + + for image, label in zip(images, labels): + for split, k_split in folds_df.loc[image.stem].items(): + # Destination directory + img_to_path = save_path / split / k_split / "images" + lbl_to_path = save_path / split / k_split / "labels" + + # Copy image and label files to new directory (SamefileError if file already exists) + shutil.copy(image, img_to_path / image.name) + shutil.copy(label, lbl_to_path / label.name) + ``` + +## Save Records (Optional) + +Optionally, you can save the records of the K-Fold split and label distribution DataFrames as CSV files for future reference. + +```python +folds_df.to_csv(save_path / "kfold_datasplit.csv") +fold_lbl_distrb.to_csv(save_path / "kfold_label_distribution.csv") +``` + +## Train YOLO using K-Fold Data Splits + +1. First, load the YOLO model. + + ```python + from ultralytics import YOLO + + weights_path = "path/to/weights.pt" + model = YOLO(weights_path, task="detect") + ``` + +2. Next, iterate over the dataset YAML files to run training. The results will be saved to a directory specified by the `project` and `name` arguments. By default, this directory is 'exp/runs#' where # is an integer index. + + ```python + results = {} + + # Define your additional arguments here + batch = 16 + project = "kfold_demo" + epochs = 100 + + for k in range(ksplit): + dataset_yaml = ds_yamls[k] + model.train(data=dataset_yaml, epochs=epochs, batch=batch, project=project) # include any train arguments + results[k] = model.metrics # save output metrics for further analysis + ``` + +## Conclusion + +In this guide, we have explored the process of using K-Fold cross-validation for training the YOLO object detection model. We learned how to split our dataset into K partitions, ensuring a balanced class distribution across the different folds. + +We also explored the procedure for creating report DataFrames to visualize the data splits and label distributions across these splits, providing us a clear insight into the structure of our training and validation sets. + +Optionally, we saved our records for future reference, which could be particularly useful in large-scale projects or when troubleshooting model performance. + +Finally, we implemented the actual model training using each split in a loop, saving our training results for further analysis and comparison. + +This technique of K-Fold cross-validation is a robust way of making the most out of your available data, and it helps to ensure that your model performance is reliable and consistent across different data subsets. This results in a more generalizable and reliable model that is less likely to overfit to specific data patterns. + +Remember that although we used YOLO in this guide, these steps are mostly transferable to other machine learning models. Understanding these steps allows you to apply cross-validation effectively in your own machine learning projects. Happy coding! + +## FAQ + +### What is K-Fold Cross Validation and why is it useful in object detection? + +K-Fold Cross Validation is a technique where the dataset is divided into 'k' subsets (folds) to evaluate model performance more reliably. Each fold serves as both training and validation data. In the context of object detection, using K-Fold Cross Validation helps to ensure your Ultralytics YOLO model's performance is robust and generalizable across different data splits, enhancing its reliability. For detailed instructions on setting up K-Fold Cross Validation with Ultralytics YOLO, refer to [K-Fold Cross Validation with Ultralytics](#introduction). + +### How do I implement K-Fold Cross Validation using Ultralytics YOLO? + +To implement K-Fold Cross Validation with Ultralytics YOLO, you need to follow these steps: + +1. Verify annotations are in the [YOLO detection format](../datasets/detect/index.md). +2. Use Python libraries like `sklearn`, `pandas`, and `pyyaml`. +3. Create feature vectors from your dataset. +4. Split your dataset using `KFold` from `sklearn.model_selection`. +5. Train the YOLO model on each split. + +For a comprehensive guide, see the [K-Fold Dataset Split](#k-fold-dataset-split) section in our documentation. + +### Why should I use Ultralytics YOLO for object detection? + +Ultralytics YOLO offers state-of-the-art, real-time object detection with high accuracy and efficiency. It's versatile, supporting multiple computer vision tasks such as detection, segmentation, and classification. Additionally, it integrates seamlessly with tools like Ultralytics HUB for no-code model training and deployment. For more details, explore the benefits and features on our [Ultralytics YOLO page](https://www.ultralytics.com/yolo). + +### How can I ensure my annotations are in the correct format for Ultralytics YOLO? + +Your annotations should follow the YOLO detection format. Each annotation file must list the object class, alongside its bounding box coordinates in the image. The YOLO format ensures streamlined and standardized data processing for training object detection models. For more information on proper annotation formatting, visit the [YOLO detection format guide](../datasets/detect/index.md). + +### Can I use K-Fold Cross Validation with custom datasets other than Fruit Detection? + +Yes, you can use K-Fold Cross Validation with any custom dataset as long as the annotations are in the YOLO detection format. Replace the dataset paths and class labels with those specific to your custom dataset. This flexibility ensures that any object detection project can benefit from robust model evaluation using K-Fold Cross Validation. For a practical example, review our [Generating Feature Vectors](#generating-feature-vectors-for-object-detection-dataset) section. diff --git a/ultralytics/docs/en/guides/model-deployment-options.md b/ultralytics/docs/en/guides/model-deployment-options.md new file mode 100644 index 0000000000000000000000000000000000000000..874bab4643a2fdec1133a57c51f9109d28b248c0 --- /dev/null +++ b/ultralytics/docs/en/guides/model-deployment-options.md @@ -0,0 +1,370 @@ +--- +comments: true +description: Learn about YOLOv8's diverse deployment options to maximize your model's performance. Explore PyTorch, TensorRT, OpenVINO, TF Lite, and more!. +keywords: YOLOv8, deployment options, export formats, PyTorch, TensorRT, OpenVINO, TF Lite, machine learning, model deployment +--- + +# Understanding YOLOv8's Deployment Options + +## Introduction + +You've come a long way on your journey with YOLOv8. You've diligently collected data, meticulously annotated it, and put in the hours to train and rigorously evaluate your custom YOLOv8 model. Now, it's time to put your model to work for your specific application, use case, or project. But there's a critical decision that stands before you: how to export and deploy your model effectively. + +This guide walks you through YOLOv8's deployment options and the essential factors to consider to choose the right option for your project. + +## How to Select the Right Deployment Option for Your YOLOv8 Model + +When it's time to deploy your YOLOv8 model, selecting a suitable export format is very important. As outlined in the [Ultralytics YOLOv8 Modes documentation](../modes/export.md#usage-examples), the model.export() function allows for converting your trained model into a variety of formats tailored to diverse environments and performance requirements. + +The ideal format depends on your model's intended operational context, balancing speed, hardware constraints, and ease of integration. In the following section, we'll take a closer look at each export option, understanding when to choose each one. + +### YOLOv8's Deployment Options + +Let's walk through the different YOLOv8 deployment options. For a detailed walkthrough of the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +#### PyTorch + +PyTorch is an open-source machine learning library widely used for applications in deep learning and artificial intelligence. It provides a high level of flexibility and speed, which has made it a favorite among researchers and developers. + +- **Performance Benchmarks**: PyTorch is known for its ease of use and flexibility, which may result in a slight trade-off in raw performance when compared to other frameworks that are more specialized and optimized. + +- **Compatibility and Integration**: Offers excellent compatibility with various data science and machine learning libraries in Python. + +- **Community Support and Ecosystem**: One of the most vibrant communities, with extensive resources for learning and troubleshooting. + +- **Case Studies**: Commonly used in research prototypes, many academic papers reference models deployed in PyTorch. + +- **Maintenance and Updates**: Regular updates with active development and support for new features. + +- **Security Considerations**: Regular patches for security issues, but security is largely dependent on the overall environment it's deployed in. + +- **Hardware Acceleration**: Supports CUDA for GPU acceleration, essential for speeding up model training and inference. + +#### TorchScript + +TorchScript extends PyTorch's capabilities by allowing the exportation of models to be run in a C++ runtime environment. This makes it suitable for production environments where Python is unavailable. + +- **Performance Benchmarks**: Can offer improved performance over native PyTorch, especially in production environments. + +- **Compatibility and Integration**: Designed for seamless transition from PyTorch to C++ production environments, though some advanced features might not translate perfectly. + +- **Community Support and Ecosystem**: Benefits from PyTorch's large community but has a narrower scope of specialized developers. + +- **Case Studies**: Widely used in industry settings where Python's performance overhead is a bottleneck. + +- **Maintenance and Updates**: Maintained alongside PyTorch with consistent updates. + +- **Security Considerations**: Offers improved security by enabling the running of models in environments without full Python installations. + +- **Hardware Acceleration**: Inherits PyTorch's CUDA support, ensuring efficient GPU utilization. + +#### ONNX + +The Open Neural Network Exchange (ONNX) is a format that allows for model interoperability across different frameworks, which can be critical when deploying to various platforms. + +- **Performance Benchmarks**: ONNX models may experience a variable performance depending on the specific runtime they are deployed on. + +- **Compatibility and Integration**: High interoperability across multiple platforms and hardware due to its framework-agnostic nature. + +- **Community Support and Ecosystem**: Supported by many organizations, leading to a broad ecosystem and a variety of tools for optimization. + +- **Case Studies**: Frequently used to move models between different machine learning frameworks, demonstrating its flexibility. + +- **Maintenance and Updates**: As an open standard, ONNX is regularly updated to support new operations and models. + +- **Security Considerations**: As with any cross-platform tool, it's essential to ensure secure practices in the conversion and deployment pipeline. + +- **Hardware Acceleration**: With ONNX Runtime, models can leverage various hardware optimizations. + +#### OpenVINO + +OpenVINO is an Intel toolkit designed to facilitate the deployment of deep learning models across Intel hardware, enhancing performance and speed. + +- **Performance Benchmarks**: Specifically optimized for Intel CPUs, GPUs, and VPUs, offering significant performance boosts on compatible hardware. + +- **Compatibility and Integration**: Works best within the Intel ecosystem but also supports a range of other platforms. + +- **Community Support and Ecosystem**: Backed by Intel, with a solid user base especially in the computer vision domain. + +- **Case Studies**: Often utilized in IoT and edge computing scenarios where Intel hardware is prevalent. + +- **Maintenance and Updates**: Intel regularly updates OpenVINO to support the latest deep learning models and Intel hardware. + +- **Security Considerations**: Provides robust security features suitable for deployment in sensitive applications. + +- **Hardware Acceleration**: Tailored for acceleration on Intel hardware, leveraging dedicated instruction sets and hardware features. + +For more details on deployment using OpenVINO, refer to the Ultralytics Integration documentation: [Intel OpenVINO Export](../integrations/openvino.md). + +#### TensorRT + +TensorRT is a high-performance deep learning inference optimizer and runtime from NVIDIA, ideal for applications needing speed and efficiency. + +- **Performance Benchmarks**: Delivers top-tier performance on NVIDIA GPUs with support for high-speed inference. + +- **Compatibility and Integration**: Best suited for NVIDIA hardware, with limited support outside this environment. + +- **Community Support and Ecosystem**: Strong support network through NVIDIA's developer forums and documentation. + +- **Case Studies**: Widely adopted in industries requiring real-time inference on video and image data. + +- **Maintenance and Updates**: NVIDIA maintains TensorRT with frequent updates to enhance performance and support new GPU architectures. + +- **Security Considerations**: Like many NVIDIA products, it has a strong emphasis on security, but specifics depend on the deployment environment. + +- **Hardware Acceleration**: Exclusively designed for NVIDIA GPUs, providing deep optimization and acceleration. + +#### CoreML + +CoreML is Apple's machine learning framework, optimized for on-device performance in the Apple ecosystem, including iOS, macOS, watchOS, and tvOS. + +- **Performance Benchmarks**: Optimized for on-device performance on Apple hardware with minimal battery usage. + +- **Compatibility and Integration**: Exclusively for Apple's ecosystem, providing a streamlined workflow for iOS and macOS applications. + +- **Community Support and Ecosystem**: Strong support from Apple and a dedicated developer community, with extensive documentation and tools. + +- **Case Studies**: Commonly used in applications that require on-device machine learning capabilities on Apple products. + +- **Maintenance and Updates**: Regularly updated by Apple to support the latest machine learning advancements and Apple hardware. + +- **Security Considerations**: Benefits from Apple's focus on user privacy and data security. + +- **Hardware Acceleration**: Takes full advantage of Apple's neural engine and GPU for accelerated machine learning tasks. + +#### TF SavedModel + +TF SavedModel is TensorFlow's format for saving and serving machine learning models, particularly suited for scalable server environments. + +- **Performance Benchmarks**: Offers scalable performance in server environments, especially when used with TensorFlow Serving. + +- **Compatibility and Integration**: Wide compatibility across TensorFlow's ecosystem, including cloud and enterprise server deployments. + +- **Community Support and Ecosystem**: Large community support due to TensorFlow's popularity, with a vast array of tools for deployment and optimization. + +- **Case Studies**: Extensively used in production environments for serving deep learning models at scale. + +- **Maintenance and Updates**: Supported by Google and the TensorFlow community, ensuring regular updates and new features. + +- **Security Considerations**: Deployment using TensorFlow Serving includes robust security features for enterprise-grade applications. + +- **Hardware Acceleration**: Supports various hardware accelerations through TensorFlow's backends. + +#### TF GraphDef + +TF GraphDef is a TensorFlow format that represents the model as a graph, which is beneficial for environments where a static computation graph is required. + +- **Performance Benchmarks**: Provides stable performance for static computation graphs, with a focus on consistency and reliability. + +- **Compatibility and Integration**: Easily integrates within TensorFlow's infrastructure but less flexible compared to SavedModel. + +- **Community Support and Ecosystem**: Good support from TensorFlow's ecosystem, with many resources available for optimizing static graphs. + +- **Case Studies**: Useful in scenarios where a static graph is necessary, such as in certain embedded systems. + +- **Maintenance and Updates**: Regular updates alongside TensorFlow's core updates. + +- **Security Considerations**: Ensures safe deployment with TensorFlow's established security practices. + +- **Hardware Acceleration**: Can utilize TensorFlow's hardware acceleration options, though not as flexible as SavedModel. + +#### TF Lite + +TF Lite is TensorFlow's solution for mobile and embedded device machine learning, providing a lightweight library for on-device inference. + +- **Performance Benchmarks**: Designed for speed and efficiency on mobile and embedded devices. + +- **Compatibility and Integration**: Can be used on a wide range of devices due to its lightweight nature. + +- **Community Support and Ecosystem**: Backed by Google, it has a robust community and a growing number of resources for developers. + +- **Case Studies**: Popular in mobile applications that require on-device inference with minimal footprint. + +- **Maintenance and Updates**: Regularly updated to include the latest features and optimizations for mobile devices. + +- **Security Considerations**: Provides a secure environment for running models on end-user devices. + +- **Hardware Acceleration**: Supports a variety of hardware acceleration options, including GPU and DSP. + +#### TF Edge TPU + +TF Edge TPU is designed for high-speed, efficient computing on Google's Edge TPU hardware, perfect for IoT devices requiring real-time processing. + +- **Performance Benchmarks**: Specifically optimized for high-speed, efficient computing on Google's Edge TPU hardware. + +- **Compatibility and Integration**: Works exclusively with TensorFlow Lite models on Edge TPU devices. + +- **Community Support and Ecosystem**: Growing support with resources provided by Google and third-party developers. + +- **Case Studies**: Used in IoT devices and applications that require real-time processing with low latency. + +- **Maintenance and Updates**: Continually improved upon to leverage the capabilities of new Edge TPU hardware releases. + +- **Security Considerations**: Integrates with Google's robust security for IoT and edge devices. + +- **Hardware Acceleration**: Custom-designed to take full advantage of Google Coral devices. + +#### TF.js + +TensorFlow.js (TF.js) is a library that brings machine learning capabilities directly to the browser, offering a new realm of possibilities for web developers and users alike. It allows for the integration of machine learning models in web applications without the need for back-end infrastructure. + +- **Performance Benchmarks**: Enables machine learning directly in the browser with reasonable performance, depending on the client device. + +- **Compatibility and Integration**: High compatibility with web technologies, allowing for easy integration into web applications. + +- **Community Support and Ecosystem**: Support from a community of web and Node.js developers, with a variety of tools for deploying ML models in browsers. + +- **Case Studies**: Ideal for interactive web applications that benefit from client-side machine learning without the need for server-side processing. + +- **Maintenance and Updates**: Maintained by the TensorFlow team with contributions from the open-source community. + +- **Security Considerations**: Runs within the browser's secure context, utilizing the security model of the web platform. + +- **Hardware Acceleration**: Performance can be enhanced with web-based APIs that access hardware acceleration like WebGL. + +#### PaddlePaddle + +PaddlePaddle is an open-source deep learning framework developed by Baidu. It is designed to be both efficient for researchers and easy to use for developers. It's particularly popular in China and offers specialized support for Chinese language processing. + +- **Performance Benchmarks**: Offers competitive performance with a focus on ease of use and scalability. + +- **Compatibility and Integration**: Well-integrated within Baidu's ecosystem and supports a wide range of applications. + +- **Community Support and Ecosystem**: While the community is smaller globally, it's rapidly growing, especially in China. + +- **Case Studies**: Commonly used in Chinese markets and by developers looking for alternatives to other major frameworks. + +- **Maintenance and Updates**: Regularly updated with a focus on serving Chinese language AI applications and services. + +- **Security Considerations**: Emphasizes data privacy and security, catering to Chinese data governance standards. + +- **Hardware Acceleration**: Supports various hardware accelerations, including Baidu's own Kunlun chips. + +#### NCNN + +NCNN is a high-performance neural network inference framework optimized for the mobile platform. It stands out for its lightweight nature and efficiency, making it particularly well-suited for mobile and embedded devices where resources are limited. + +- **Performance Benchmarks**: Highly optimized for mobile platforms, offering efficient inference on ARM-based devices. + +- **Compatibility and Integration**: Suitable for applications on mobile phones and embedded systems with ARM architecture. + +- **Community Support and Ecosystem**: Supported by a niche but active community focused on mobile and embedded ML applications. + +- **Case Studies**: Favoured for mobile applications where efficiency and speed are critical on Android and other ARM-based systems. + +- **Maintenance and Updates**: Continuously improved to maintain high performance on a range of ARM devices. + +- **Security Considerations**: Focuses on running locally on the device, leveraging the inherent security of on-device processing. + +- **Hardware Acceleration**: Tailored for ARM CPUs and GPUs, with specific optimizations for these architectures. + +## Comparative Analysis of YOLOv8 Deployment Options + +The following table provides a snapshot of the various deployment options available for YOLOv8 models, helping you to assess which may best fit your project needs based on several critical criteria. For an in-depth look at each deployment option's format, please see the [Ultralytics documentation page on export formats](../modes/export.md#export-formats). + +| Deployment Option | Performance Benchmarks | Compatibility and Integration | Community Support and Ecosystem | Case Studies | Maintenance and Updates | Security Considerations | Hardware Acceleration | +| ----------------- | ----------------------------------------------- | ---------------------------------------------- | --------------------------------------------- | ------------------------------------------ | ------------------------------------------- | ------------------------------------------------- | ---------------------------------- | +| PyTorch | Good flexibility; may trade off raw performance | Excellent with Python libraries | Extensive resources and community | Research and prototypes | Regular, active development | Dependent on deployment environment | CUDA support for GPU acceleration | +| TorchScript | Better for production than PyTorch | Smooth transition from PyTorch to C++ | Specialized but narrower than PyTorch | Industry where Python is a bottleneck | Consistent updates with PyTorch | Improved security without full Python | Inherits CUDA support from PyTorch | +| ONNX | Variable depending on runtime | High across different frameworks | Broad ecosystem, supported by many orgs | Flexibility across ML frameworks | Regular updates for new operations | Ensure secure conversion and deployment practices | Various hardware optimizations | +| OpenVINO | Optimized for Intel hardware | Best within Intel ecosystem | Solid in computer vision domain | IoT and edge with Intel hardware | Regular updates for Intel hardware | Robust features for sensitive applications | Tailored for Intel hardware | +| TensorRT | Top-tier on NVIDIA GPUs | Best for NVIDIA hardware | Strong network through NVIDIA | Real-time video and image inference | Frequent updates for new GPUs | Emphasis on security | Designed for NVIDIA GPUs | +| CoreML | Optimized for on-device Apple hardware | Exclusive to Apple ecosystem | Strong Apple and developer support | On-device ML on Apple products | Regular Apple updates | Focus on privacy and security | Apple neural engine and GPU | +| TF SavedModel | Scalable in server environments | Wide compatibility in TensorFlow ecosystem | Large support due to TensorFlow popularity | Serving models at scale | Regular updates by Google and community | Robust features for enterprise | Various hardware accelerations | +| TF GraphDef | Stable for static computation graphs | Integrates well with TensorFlow infrastructure | Resources for optimizing static graphs | Scenarios requiring static graphs | Updates alongside TensorFlow core | Established TensorFlow security practices | TensorFlow acceleration options | +| TF Lite | Speed and efficiency on mobile/embedded | Wide range of device support | Robust community, Google backed | Mobile applications with minimal footprint | Latest features for mobile | Secure environment on end-user devices | GPU and DSP among others | +| TF Edge TPU | Optimized for Google's Edge TPU hardware | Exclusive to Edge TPU devices | Growing with Google and third-party resources | IoT devices requiring real-time processing | Improvements for new Edge TPU hardware | Google's robust IoT security | Custom-designed for Google Coral | +| TF.js | Reasonable in-browser performance | High with web technologies | Web and Node.js developers support | Interactive web applications | TensorFlow team and community contributions | Web platform security model | Enhanced with WebGL and other APIs | +| PaddlePaddle | Competitive, easy to use and scalable | Baidu ecosystem, wide application support | Rapidly growing, especially in China | Chinese market and language processing | Focus on Chinese AI applications | Emphasizes data privacy and security | Including Baidu's Kunlun chips | +| NCNN | Optimized for mobile ARM-based devices | Mobile and embedded ARM systems | Niche but active mobile/embedded ML community | Android and ARM systems efficiency | High performance maintenance on ARM | On-device security advantages | ARM CPUs and GPUs optimizations | + +This comparative analysis gives you a high-level overview. For deployment, it's essential to consider the specific requirements and constraints of your project, and consult the detailed documentation and resources available for each option. + +## Community and Support + +When you're getting started with YOLOv8, having a helpful community and support can make a significant impact. Here's how to connect with others who share your interests and get the assistance you need. + +### Engage with the Broader Community + +- **GitHub Discussions:** The YOLOv8 repository on GitHub has a "Discussions" section where you can ask questions, report issues, and suggest improvements. + +- **Ultralytics Discord Server:** Ultralytics has a [Discord server](https://ultralytics.com/discord/) where you can interact with other users and developers. + +### Official Documentation and Resources + +- **Ultralytics YOLOv8 Docs:** The [official documentation](../index.md) provides a comprehensive overview of YOLOv8, along with guides on installation, usage, and troubleshooting. + +These resources will help you tackle challenges and stay updated on the latest trends and best practices in the YOLOv8 community. + +## Conclusion + +In this guide, we've explored the different deployment options for YOLOv8. We've also discussed the important factors to consider when making your choice. These options allow you to customize your model for various environments and performance requirements, making it suitable for real-world applications. + +Don't forget that the YOLOv8 and Ultralytics community is a valuable source of help. Connect with other developers and experts to learn unique tips and solutions you might not find in regular documentation. Keep seeking knowledge, exploring new ideas, and sharing your experiences. + +Happy deploying! + +## FAQ + +### What are the deployment options available for YOLOv8 on different hardware platforms? + +Ultralytics YOLOv8 supports various deployment formats, each designed for specific environments and hardware platforms. Key formats include: + +- **PyTorch** for research and prototyping, with excellent Python integration. +- **TorchScript** for production environments where Python is unavailable. +- **ONNX** for cross-platform compatibility and hardware acceleration. +- **OpenVINO** for optimized performance on Intel hardware. +- **TensorRT** for high-speed inference on NVIDIA GPUs. + +Each format has unique advantages. For a detailed walkthrough, see our [export process documentation](../modes/export.md#usage-examples). + +### How do I improve the inference speed of my YOLOv8 model on an Intel CPU? + +To enhance inference speed on Intel CPUs, you can deploy your YOLOv8 model using Intel's OpenVINO toolkit. OpenVINO offers significant performance boosts by optimizing models to leverage Intel hardware efficiently. + +1. Convert your YOLOv8 model to the OpenVINO format using the `model.export()` function. +2. Follow the detailed setup guide in the [Intel OpenVINO Export documentation](../integrations/openvino.md). + +For more insights, check out our [blog post](https://www.ultralytics.com/blog/achieve-faster-inference-speeds-ultralytics-yolov8-openvino). + +### Can I deploy YOLOv8 models on mobile devices? + +Yes, YOLOv8 models can be deployed on mobile devices using TensorFlow Lite (TF Lite) for both Android and iOS platforms. TF Lite is designed for mobile and embedded devices, providing efficient on-device inference. + +!!! Example + + === "Python" + + ```python + # Export command for TFLite format + model.export(format="tflite") + ``` + + === "CLI" + + ```bash + # CLI command for TFLite export + yolo export --format tflite + ``` + +For more details on deploying models to mobile, refer to our [TF Lite integration guide](../integrations/tflite.md). + +### What factors should I consider when choosing a deployment format for my YOLOv8 model? + +When choosing a deployment format for YOLOv8, consider the following factors: + +- **Performance**: Some formats like TensorRT provide exceptional speeds on NVIDIA GPUs, while OpenVINO is optimized for Intel hardware. +- **Compatibility**: ONNX offers broad compatibility across different platforms. +- **Ease of Integration**: Formats like CoreML or TF Lite are tailored for specific ecosystems like iOS and Android, respectively. +- **Community Support**: Formats like PyTorch and TensorFlow have extensive community resources and support. + +For a comparative analysis, refer to our [export formats documentation](../modes/export.md#export-formats). + +### How can I deploy YOLOv8 models in a web application? + +To deploy YOLOv8 models in a web application, you can use TensorFlow.js (TF.js), which allows for running machine learning models directly in the browser. This approach eliminates the need for backend infrastructure and provides real-time performance. + +1. Export the YOLOv8 model to the TF.js format. +2. Integrate the exported model into your web application. + +For step-by-step instructions, refer to our guide on [TensorFlow.js integration](../integrations/tfjs.md). diff --git a/ultralytics/docs/en/guides/model-deployment-practices.md b/ultralytics/docs/en/guides/model-deployment-practices.md new file mode 100644 index 0000000000000000000000000000000000000000..9d35d266bf083ee227849944aab8c5c1b42cb112 --- /dev/null +++ b/ultralytics/docs/en/guides/model-deployment-practices.md @@ -0,0 +1,159 @@ +--- +comments: true +description: Learn essential tips, insights, and best practices for deploying computer vision models with a focus on efficiency, optimization, troubleshooting, and maintaining security. +keywords: Model Deployment, Machine Learning Model Deployment, ML Model Deployment, AI Model Deployment, How to Deploy a Machine Learning Model, How to Deploy ML Models +--- + +# Best Practices for Model Deployment + +## Introduction + +Model deployment is the [step in a computer vision project](./steps-of-a-cv-project.md) that brings a model from the development phase into a real-world application. There are various [model deployment options](./model-deployment-options.md): cloud deployment offers scalability and ease of access, edge deployment reduces latency by bringing the model closer to the data source, and local deployment ensures privacy and control. Choosing the right strategy depends on your application's needs, balancing speed, security, and scalability. + +It's also important to follow best practices when deploying a model because deployment can significantly impact the effectiveness and reliability of the model's performance. In this guide, we'll focus on how to make sure that your model deployment is smooth, efficient, and secure. + +## Model Deployment Options + +Often times, once a model is [trained](./model-training-tips.md), [evaluated](./model-evaluation-insights.md), and [tested](./model-testing.md), it needs to be converted into specific formats to be deployed effectively in various environments, such as cloud, edge, or local devices. + +With respect to YOLOv8, you can [export your model](../modes/export.md) to different formats. For example, when you need to transfer your model between different frameworks, ONNX is an excellent tool and [exporting to YOLOv8 to ONNX](../integrations/onnx.md) is easy. You can check out more options about integrating your model into different environments smoothly and effectively [here](../integrations/index.md). + +### Choosing a Deployment Environment + +Choosing where to deploy your computer vision model depends on multiple factors. Different environments have unique benefits and challenges, so it's essential to pick the one that best fits your needs. + +#### Cloud Deployment + +Cloud deployment is great for applications that need to scale up quickly and handle large amounts of data. Platforms like AWS, [Google Cloud](../yolov5/environments/google_cloud_quickstart_tutorial.md), and Azure make it easy to manage your models from training to deployment. They offer services like [AWS SageMaker](../integrations/amazon-sagemaker.md), Google AI Platform, and [Azure Machine Learning](./azureml-quickstart.md) to help you throughout the process. + +However, using the cloud can be expensive, especially with high data usage, and you might face latency issues if your users are far from the data centers. To manage costs and performance, it's important to optimize resource use and ensure compliance with data privacy rules. + +#### Edge Deployment + +Edge deployment works well for applications needing real-time responses and low latency, particularly in places with limited or no internet access. Deploying models on edge devices like smartphones or IoT gadgets ensures fast processing and keeps data local, which enhances privacy. Deploying on edge also saves bandwidth due to reduced data sent to the cloud. + +However, edge devices often have limited processing power, so you'll need to optimize your models. Tools like [TensorFlow Lite](../integrations/tflite.md) and [NVIDIA Jetson](./nvidia-jetson.md) can help. Despite the benefits, maintaining and updating many devices can be challenging. + +#### Local Deployment + +Local Deployment is best when data privacy is critical or when there's unreliable or no internet access. Running models on local servers or desktops gives you full control and keeps your data secure. It can also reduce latency if the server is near the user. + +However, scaling locally can be tough, and maintenance can be time-consuming. Using tools like [Docker](./docker-quickstart.md) for containerization and Kubernetes for management can help make local deployments more efficient. Regular updates and maintenance are necessary to keep everything running smoothly. + +## Model Optimization Techniques + +Optimizing your computer vision model helps it runs efficiently, especially when deploying in environments with limited resources like edge devices. Here are some key techniques for optimizing your model. + +### Model Pruning + +Pruning reduces the size of the model by removing weights that contribute little to the final output. It makes the model smaller and faster without significantly affecting accuracy. Pruning involves identifying and eliminating unnecessary parameters, resulting in a lighter model that requires less computational power. It is particularly useful for deploying models on devices with limited resources. + ++ +
+ +### Model Quantization + +Quantization converts the model's weights and activations from high precision (like 32-bit floats) to lower precision (like 8-bit integers). By reducing the model size, it speeds up inference. Quantization-aware training (QAT) is a method where the model is trained with quantization in mind, preserving accuracy better than post-training quantization. By handling quantization during the training phase, the model learns to adjust to lower precision, maintaining performance while reducing computational demands. + ++ +
+ +### Knowledge Distillation + +Knowledge distillation involves training a smaller, simpler model (the student) to mimic the outputs of a larger, more complex model (the teacher). The student model learns to approximate the teacher's predictions, resulting in a compact model that retains much of the teacher's accuracy. This technique is beneficial for creating efficient models suitable for deployment on edge devices with constrained resources. + ++ +
+ +## Troubleshooting Deployment Issues + +You may face challenges while deploying your computer vision models, but understanding common problems and solutions can make the process smoother. Here are some general troubleshooting tips and best practices to help you navigate deployment issues. + +### Your Model is Less Accurate After Deployment + +Experiencing a drop in your model's accuracy after deployment can be frustrating. This issue can stem from various factors. Here are some steps to help you identify and resolve the problem: + +- **Check Data Consistency:** Check that the data your model is processing post-deployment is consistent with the data it was trained on. Differences in data distribution, quality, or format can significantly impact performance. +- **Validate Preprocessing Steps:** Verify that all preprocessing steps applied during training are also applied consistently during deployment. This includes resizing images, normalizing pixel values, and other data transformations. +- **Evaluate the Model's Environment:** Ensure that the hardware and software configurations used during deployment match those used during training. Differences in libraries, versions, and hardware capabilities can introduce discrepancies. +- **Monitor Model Inference:** Log inputs and outputs at various stages of the inference pipeline to detect any anomalies. It can help identify issues like data corruption or improper handling of model outputs. +- **Review Model Export and Conversion:** Re-export the model and make sure that the conversion process maintains the integrity of the model weights and architecture. +- **Test with a Controlled Dataset:** Deploy the model in a test environment with a dataset you control and compare the results with the training phase. You can identify if the issue is with the deployment environment or the data. + +When deploying YOLOv8, several factors can affect model accuracy. Converting models to formats like [TensorRT](../integrations/tensorrt.md) involves optimizations such as weight quantization and layer fusion, which can cause minor precision losses. Using FP16 (half-precision) instead of FP32 (full-precision) can speed up inference but may introduce numerical precision errors. Also, hardware constraints, like those on the [Jetson Nano](./nvidia-jetson.md), with lower CUDA core counts and reduced memory bandwidth, can impact performance. + +### Inferences Are Taking Longer Than You Expected + +When deploying machine learning models, it's important that they run efficiently. If inferences are taking longer than expected, it can affect the user experience and the effectiveness of your application. Here are some steps to help you identify and resolve the problem: + +- **Implement Warm-Up Runs**: Initial runs often include setup overhead, which can skew latency measurements. Perform a few warm-up inferences before measuring latency. Excluding these initial runs provides a more accurate measurement of the model's performance. +- **Optimize the Inference Engine:** Double-check that the inference engine is fully optimized for your specific GPU architecture. Use the latest drivers and software versions tailored to your hardware to ensure maximum performance and compatibility. +- **Use Asynchronous Processing:** Asynchronous processing can help manage workloads more efficiently. Use asynchronous processing techniques to handle multiple inferences concurrently, which can help distribute the load and reduce wait times. +- **Profile the Inference Pipeline:** Identifying bottlenecks in the inference pipeline can help pinpoint the source of delays. Use profiling tools to analyze each step of the inference process, identifying and addressing any stages that cause significant delays, such as inefficient layers or data transfer issues. +- **Use Appropriate Precision:** Using higher precision than necessary can slow down inference times. Experiment with using lower precision, such as FP16 (half-precision), instead of FP32 (full-precision). While FP16 can reduce inference time, also keep in mind that it can impact model accuracy. + +If you are facing this issue while deploying YOLOv8, consider that YOLOv8 offers [various model sizes](../models/yolov8.md), such as YOLOv8n (nano) for devices with lower memory capacity and YOLOv8x (extra-large) for more powerful GPUs. Choosing the right model variant for your hardware can help balance memory usage and processing time. + +Also keep in mind that the size of the input images directly impacts memory usage and processing time. Lower resolutions reduce memory usage and speed up inference, while higher resolutions improve accuracy but require more memory and processing power. + +## Security Considerations in Model Deployment + +Another important aspect of deployment is security. The security of your deployed models is critical to protect sensitive data and intellectual property. Here are some best practices you can follow related to secure model deployment. + +### Secure Data Transmission + +Making sure data sent between clients and servers is secure is very important to prevent it from being intercepted or accessed by unauthorized parties. You can use encryption protocols like TLS (Transport Layer Security) to encrypt data while it's being transmitted. Even if someone intercepts the data, they won't be able to read it. You can also use end-to-end encryption that protects the data all the way from the source to the destination, so no one in between can access it. + +### Access Controls + +It's essential to control who can access your model and its data to prevent unauthorized use. Use strong authentication methods to verify the identity of users or systems trying to access the model, and consider adding extra security with multi-factor authentication (MFA). Set up role-based access control (RBAC) to assign permissions based on user roles so that people only have access to what they need. Keep detailed audit logs to track all access and changes to the model and its data, and regularly review these logs to spot any suspicious activity. + +### Model Obfuscation + +Protecting your model from being reverse-engineered or misuse can be done through model obfuscation. It involves encrypting model parameters, such as weights and biases in neural networks, to make it difficult for unauthorized individuals to understand or alter the model. You can also obfuscate the model's architecture by renaming layers and parameters or adding dummy layers, making it harder for attackers to reverse-engineer it. You can also serve the model in a secure environment, like a secure enclave or using a trusted execution environment (TEE), can provide an extra layer of protection during inference. + +## Share Ideas With Your Peers + +Being part of a community of computer vision enthusiasts can help you solve problems and learn faster. Here are some ways to connect, get help, and share ideas. + +### Community Resources + +- **GitHub Issues:** Explore the [YOLOv8 GitHub repository](https://github.com/ultralytics/ultralytics/issues) and use the Issues tab to ask questions, report bugs, and suggest new features. The community and maintainers are very active and ready to help. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to chat with other users and developers, get support, and share your experiences. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Visit the [official YOLOv8 documentation](./index.md) for detailed guides and helpful tips on various computer vision projects. + +Using these resources will help you solve challenges and stay up-to-date with the latest trends and practices in the computer vision community. + +## Conclusion and Next Steps + +We walked through some best practices to follow when deploying computer vision models. By securing data, controlling access, and obfuscating model details, you can protect sensitive information while keeping your models running smoothly. We also discussed how to address common issues like reduced accuracy and slow inferences using strategies such as warm-up runs, optimizing engines, asynchronous processing, profiling pipelines, and choosing the right precision. + +After deploying your model, the next step would be monitoring, maintaining, and documenting your application. Regular monitoring helps catch and fix issues quickly, maintenance keeps your models up-to-date and functional, and good documentation tracks all changes and updates. These steps will help you achieve the [goals of your computer vision project](./defining-project-goals.md). + +## FAQ + +### What are the best practices for deploying a machine learning model using Ultralytics YOLOv8? + +Deploying a machine learning model, particularly with Ultralytics YOLOv8, involves several best practices to ensure efficiency and reliability. First, choose the deployment environment that suits your needs—cloud, edge, or local. Optimize your model through techniques like [pruning, quantization, and knowledge distillation](#model-optimization-techniques) for efficient deployment in resource-constrained environments. Lastly, ensure data consistency and preprocessing steps align with the training phase to maintain performance. You can also refer to [model deployment options](./model-deployment-options.md) for more detailed guidelines. + +### How can I troubleshoot common deployment issues with Ultralytics YOLOv8 models? + +Troubleshooting deployment issues can be broken down into a few key steps. If your model's accuracy drops after deployment, check for data consistency, validate preprocessing steps, and ensure the hardware/software environment matches what you used during training. For slow inference times, perform warm-up runs, optimize your inference engine, use asynchronous processing, and profile your inference pipeline. Refer to [troubleshooting deployment issues](#troubleshooting-deployment-issues) for a detailed guide on these best practices. + +### How does Ultralytics YOLOv8 optimization enhance model performance on edge devices? + +Optimizing Ultralytics YOLOv8 models for edge devices involves using techniques like pruning to reduce the model size, quantization to convert weights to lower precision, and knowledge distillation to train smaller models that mimic larger ones. These techniques ensure the model runs efficiently on devices with limited computational power. Tools like [TensorFlow Lite](../integrations/tflite.md) and [NVIDIA Jetson](./nvidia-jetson.md) are particularly useful for these optimizations. Learn more about these techniques in our section on [model optimization](#model-optimization-techniques). + +### What are the security considerations for deploying machine learning models with Ultralytics YOLOv8? + +Security is paramount when deploying machine learning models. Ensure secure data transmission using encryption protocols like TLS. Implement robust access controls, including strong authentication and role-based access control (RBAC). Model obfuscation techniques, such as encrypting model parameters and serving models in a secure environment like a trusted execution environment (TEE), offer additional protection. For detailed practices, refer to [security considerations](#security-considerations-in-model-deployment). + +### How do I choose the right deployment environment for my Ultralytics YOLOv8 model? + +Selecting the optimal deployment environment for your Ultralytics YOLOv8 model depends on your application's specific needs. Cloud deployment offers scalability and ease of access, making it ideal for applications with high data volumes. Edge deployment is best for low-latency applications requiring real-time responses, using tools like [TensorFlow Lite](../integrations/tflite.md). Local deployment suits scenarios needing stringent data privacy and control. For a comprehensive overview of each environment, check out our section on [choosing a deployment environment](#choosing-a-deployment-environment). diff --git a/ultralytics/docs/en/guides/model-evaluation-insights.md b/ultralytics/docs/en/guides/model-evaluation-insights.md new file mode 100644 index 0000000000000000000000000000000000000000..b6d04f33abdb95479affd0f377d7a08def8aa022 --- /dev/null +++ b/ultralytics/docs/en/guides/model-evaluation-insights.md @@ -0,0 +1,188 @@ +--- +comments: true +description: Explore the most effective ways to assess and refine YOLOv8 models for better performance. Learn about evaluation metrics, fine-tuning processes, and how to customize your model for specific needs. +keywords: Model Evaluation, Machine Learning Model Evaluation, Fine Tuning Machine Learning, Fine Tune Model, Evaluating Models, Model Fine Tuning, How to Fine Tune a Model +--- + +# Insights on Model Evaluation and Fine-Tuning + +## Introduction + +Once you've [trained](./model-training-tips.md) your computer vision model, evaluating and refining it to perform optimally is essential. Just training your model isn't enough. You need to make sure that your model is accurate, efficient, and fulfills the [objective](./defining-project-goals.md) of your computer vision project. By evaluating and fine-tuning your model, you can identify weaknesses, improve its accuracy, and boost overall performance. + +In this guide, we'll share insights on model evaluation and fine-tuning that'll make this [step of a computer vision project](./steps-of-a-cv-project.md) more approachable. We'll discuss how to understand evaluation metrics and implement fine-tuning techniques, giving you the knowledge to elevate your model's capabilities. + +## Evaluating Model Performance Using Metrics + +Evaluating how well a model performs helps us understand how effectively it works. Various metrics are used to measure performance. These [performance metrics](./yolo-performance-metrics.md) provide clear, numerical insights that can guide improvements toward making sure the model meets its intended goals. Let's take a closer look at a few key metrics. + +### Confidence Score + +The confidence score represents the model's certainty that a detected object belongs to a particular class. It ranges from 0 to 1, with higher scores indicating greater confidence. The confidence score helps filter predictions; only detections with confidence scores above a specified threshold are considered valid. + +_Quick Tip:_ When running inferences, if you aren't seeing any predictions and you've checked everything else, try lowering the confidence score. Sometimes, the threshold is too high, causing the model to ignore valid predictions. Lowering the score allows the model to consider more possibilities. This might not meet your project goals, but it's a good way to see what the model can do and decide how to fine-tune it. + +### Intersection over Union + +Intersection over Union (IoU) is a metric in object detection that measures how well the predicted bounding box overlaps with the ground truth bounding box. IoU values range from 0 to 1, where one stands for a perfect match. IoU is essential because it measures how closely the predicted boundaries match the actual object boundaries. + ++ +
+ +### Mean Average Precision + +Mean Average Precision (mAP) is a way to measure how well an object detection model performs. It looks at the precision of detecting each object class, averages these scores, and gives an overall number that shows how accurately the model can identify and classify objects. + +Let's focus on two specific mAP metrics: + +- *mAP@.5:* Measures the average precision at a single IoU (Intersection over Union) threshold of 0.5. This metric checks if the model can correctly find objects with a looser accuracy requirement. It focuses on whether the object is roughly in the right place, not needing perfect placement. It helps see if the model is generally good at spotting objects. +- *mAP@.5:.95:* Averages the mAP values calculated at multiple IoU thresholds, from 0.5 to 0.95 in 0.05 increments. This metric is more detailed and strict. It gives a fuller picture of how accurately the model can find objects at different levels of strictness and is especially useful for applications that need precise object detection. + +Other mAP metrics include mAP@0.75, which uses a stricter IoU threshold of 0.75, and mAP@small, medium, and large, which evaluate precision across objects of different sizes. + ++ +
+ +## Evaluating YOLOv8 Model Performance + +With respect to YOLOv8, you can use the [validation mode](../modes/val.md) to evaluate the model. Also, be sure to take a look at our guide that goes in-depth into [YOLOv8 performance metrics](./yolo-performance-metrics.md) and how they can be interpreted. + +### Common Community Questions + +When evaluating your YOLOv8 model, you might run into a few hiccups. Based on common community questions, here are some tips to help you get the most out of your YOLOv8 model: + +#### Handling Variable Image Sizes + +Evaluating your YOLOv8 model with images of different sizes can help you understand its performance on diverse datasets. Using the `rect=true` validation parameter, YOLOv8 adjusts the network's stride for each batch based on the image sizes, allowing the model to handle rectangular images without forcing them to a single size. + +The `imgsz` validation parameter sets the maximum dimension for image resizing, which is 640 by default. You can adjust this based on your dataset's maximum dimensions and the GPU memory available. Even with `imgsz` set, `rect=true` lets the model manage varying image sizes effectively by dynamically adjusting the stride. + +#### Accessing YOLOv8 Metrics + +If you want to get a deeper understanding of your YOLOv8 model's performance, you can easily access specific evaluation metrics with a few lines of Python code. The code snippet below will let you load your model, run an evaluation, and print out various metrics that show how well your model is doing. + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the model + model = YOLO("yolov8n.pt") + + # Run the evaluation + results = model.val(data="coco8.yaml") + + # Print specific metrics + print("Class indices with average precision:", results.ap_class_index) + print("Average precision for all classes:", results.box.all_ap) + print("Average precision:", results.box.ap) + print("Average precision at IoU=0.50:", results.box.ap50) + print("Class indices for average precision:", results.box.ap_class_index) + print("Class-specific results:", results.box.class_result) + print("F1 score:", results.box.f1) + print("F1 score curve:", results.box.f1_curve) + print("Overall fitness score:", results.box.fitness) + print("Mean average precision:", results.box.map) + print("Mean average precision at IoU=0.50:", results.box.map50) + print("Mean average precision at IoU=0.75:", results.box.map75) + print("Mean average precision for different IoU thresholds:", results.box.maps) + print("Mean results for different metrics:", results.box.mean_results) + print("Mean precision:", results.box.mp) + print("Mean recall:", results.box.mr) + print("Precision:", results.box.p) + print("Precision curve:", results.box.p_curve) + print("Precision values:", results.box.prec_values) + print("Specific precision metrics:", results.box.px) + print("Recall:", results.box.r) + print("Recall curve:", results.box.r_curve) + ``` + +The results object also includes speed metrics like preprocess time, inference time, loss, and postprocess time. By analyzing these metrics, you can fine-tune and optimize your YOLOv8 model for better performance, making it more effective for your specific use case. + +## How Does Fine-Tuning Work? + +Fine-tuning involves taking a pre-trained model and adjusting its parameters to improve performance on a specific task or dataset. The process, also known as model retraining, allows the model to better understand and predict outcomes for the specific data it will encounter in real-world applications. You can retrain your model based on your model evaluation to achieve optimal results. + +## Tips for Fine-Tuning Your Model + +Fine-tuning a model means paying close attention to several vital parameters and techniques to achieve optimal performance. Here are some essential tips to guide you through the process. + +### Starting With a Higher Learning Rate + +Usually, during the initial training epochs, the learning rate starts low and gradually increases to stabilize the training process. However, since your model has already learned some features from the previous dataset, starting with a higher learning rate right away can be more beneficial. + +When evaluating your YOLOv8 model, you can set the `warmup_epochs` validation parameter to `warmup_epochs=0` to prevent the learning rate from starting too high. By following this process, the training will continue from the provided weights, adjusting to the nuances of your new data. + +### Image Tiling for Small Objects + +Image tiling can improve detection accuracy for small objects. By dividing larger images into smaller segments, such as splitting 1280x1280 images into multiple 640x640 segments, you maintain the original resolution, and the model can learn from high-resolution fragments. When using YOLOv8, make sure to adjust your labels for these new segments correctly. + +## Engage with the Community + +Sharing your ideas and questions with other computer vision enthusiasts can inspire creative solutions to roadblocks in your projects. Here are some excellent ways to learn, troubleshoot, and connect. + +### Finding Help and Support + +- **GitHub Issues:** Explore the YOLOv8 GitHub repository and use the [Issues tab](https://github.com/ultralytics/ultralytics/issues) to ask questions, report bugs, and suggest features. The community and maintainers are available to assist with any issues you encounter. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to connect with other users and developers, get support, share knowledge, and brainstorm ideas. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Check out the [official YOLOv8 documentation](./index.md) for comprehensive guides and valuable insights on various computer vision tasks and projects. + +## Final Thoughts + +Evaluating and fine-tuning your computer vision model are important steps for successful model deployment. These steps help make sure that your model is accurate, efficient, and suited to your overall application. The key to training the best model possible is continuous experimentation and learning. Don't hesitate to tweak parameters, try new techniques, and explore different datasets. Keep experimenting and pushing the boundaries of what's possible! + +## FAQ + +### What are the key metrics for evaluating YOLOv8 model performance? + +To evaluate YOLOv8 model performance, important metrics include Confidence Score, Intersection over Union (IoU), and Mean Average Precision (mAP). Confidence Score measures the model's certainty for each detected object class. IoU evaluates how well the predicted bounding box overlaps with the ground truth. Mean Average Precision (mAP) aggregates precision scores across classes, with mAP@.5 and mAP@.5:.95 being two common types for varying IoU thresholds. Learn more about these metrics in our [YOLOv8 performance metrics guide](./yolo-performance-metrics.md). + +### How can I fine-tune a pre-trained YOLOv8 model for my specific dataset? + +Fine-tuning a pre-trained YOLOv8 model involves adjusting its parameters to improve performance on a specific task or dataset. Start by evaluating your model using metrics, then set a higher initial learning rate by adjusting the `warmup_epochs` parameter to 0 for immediate stability. Use parameters like `rect=true` for handling varied image sizes effectively. For more detailed guidance, refer to our section on [fine-tuning YOLOv8 models](#how-does-fine-tuning-work). + +### How can I handle variable image sizes when evaluating my YOLOv8 model? + +To handle variable image sizes during evaluation, use the `rect=true` parameter in YOLOv8, which adjusts the network's stride for each batch based on image sizes. The `imgsz` parameter sets the maximum dimension for image resizing, defaulting to 640. Adjust `imgsz` to suit your dataset and GPU memory. For more details, visit our [section on handling variable image sizes](#handling-variable-image-sizes). + +### What practical steps can I take to improve mean average precision for my YOLOv8 model? + +Improving mean average precision (mAP) for a YOLOv8 model involves several steps: + +1. **Tuning Hyperparameters**: Experiment with different learning rates, batch sizes, and image augmentations. +2. **Data Augmentation**: Use techniques like Mosaic and MixUp to create diverse training samples. +3. **Image Tiling**: Split larger images into smaller tiles to improve detection accuracy for small objects. + Refer to our detailed guide on [model fine-tuning](#tips-for-fine-tuning-your-model) for specific strategies. + +### How do I access YOLOv8 model evaluation metrics in Python? + +You can access YOLOv8 model evaluation metrics using Python with the following steps: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the model + model = YOLO("yolov8n.pt") + + # Run the evaluation + results = model.val(data="coco8.yaml") + + # Print specific metrics + print("Class indices with average precision:", results.ap_class_index) + print("Average precision for all classes:", results.box.all_ap) + print("Mean average precision at IoU=0.50:", results.box.map50) + print("Mean recall:", results.box.mr) + ``` + +Analyzing these metrics helps fine-tune and optimize your YOLOv8 model. For a deeper dive, check out our guide on [YOLOv8 metrics](../modes/val.md). diff --git a/ultralytics/docs/en/guides/model-monitoring-and-maintenance.md b/ultralytics/docs/en/guides/model-monitoring-and-maintenance.md new file mode 100644 index 0000000000000000000000000000000000000000..57fda1d3636d42c4d679954d843a4ecd6b9d97b3 --- /dev/null +++ b/ultralytics/docs/en/guides/model-monitoring-and-maintenance.md @@ -0,0 +1,172 @@ +--- +comments: true +description: Understand the key practices for monitoring, maintaining, and documenting computer vision models to guarantee accuracy, spot anomalies, and mitigate data drift. +keywords: Computer Vision Models, AI Model Monitoring, Data Drift Detection, Anomaly Detection in AI, Model Monitoring +--- + +# Maintaining Your Computer Vision Models After Deployment + +## Introduction + +If you are here, we can assume you've completed many [steps in your computer vision project](./steps-of-a-cv-project.md): from [gathering requirements](./defining-project-goals.md), [annotating data](./data-collection-and-annotation.md), and [training the model](./model-training-tips.md) to finally [deploying](./model-deployment-practices.md) it. Your application is now running in production, but your project doesn't end here. The most important part of a computer vision project is making sure your model continues to fulfill your [project's objectives](./defining-project-goals.md) over time, and that's where monitoring, maintaining, and documenting your computer vision model enters the picture. + +In this guide, we'll take a closer look at how you can maintain your computer vision models after deployment. We'll explore how model monitoring can help you catch problems early on, how to keep your model accurate and up-to-date, and why documentation is important for troubleshooting. + +## Model Monitoring is Key + +Keeping a close eye on your deployed computer vision models is essential. Without proper monitoring, models can lose accuracy. A common issue is data distribution shift or data drift, where the data the model encounters changes from what it was trained on. When the model has to make predictions on data it doesn't recognize, it can lead to misinterpretations and poor performance. Outliers, or unusual data points, can also throw off the model's accuracy. + +Regular model monitoring helps developers track the [model's performance](./model-evaluation-insights.md), spot anomalies, and quickly address problems like data drift. It also helps manage resources by indicating when updates are needed, avoiding expensive overhauls, and keeping the model relevant. + +### Best Practices for Model Monitoring + +Here are some best practices to keep in mind while monitoring your computer vision model in production: + +- **Track Performance Regularly**: Continuously monitor the model's performance to detect changes over time. +- **Double Check the Data Quality**: Check for missing values or anomalies in the data. +- **Use Diverse Data Sources**: Monitor data from various sources to get a comprehensive view of the model's performance. +- **Combine Monitoring Techniques**: Use a mix of drift detection algorithms and rule-based approaches to identify a wide range of issues. +- **Monitor Inputs and Outputs**: Keep an eye on both the data the model processes and the results it produces to make sure everything is functioning correctly. +- **Set Up Alerts**: Implement alerts for unusual behavior, such as performance drops, to be able to make quick corrective actions. + +### Tools for AI Model Monitoring + +You can use automated monitoring tools to make it easier to monitor models after deployment. Many tools offer real-time insights and alerting capabilities. Here are some examples of open-source model monitoring tools that can work together: + +- **[Prometheus](https://prometheus.io/)**: Prometheus is an open-source monitoring tool that collects and stores metrics for detailed performance tracking. It integrates easily with Kubernetes and Docker, collecting data at set intervals and storing it in a time-series database. Prometheus can also scrape HTTP endpoints to gather real-time metrics. Collected data can be queried using the PromQL language. +- **[Grafana](https://grafana.com/)**: Grafana is an open-source data visualization and monitoring tool that allows you to query, visualize, alert on, and understand your metrics no matter where they are stored. It works well with Prometheus and offers advanced data visualization features. You can create custom dashboards to show important metrics for your computer vision models, like inference latency, error rates, and resource usage. Grafana turns collected data into easy-to-read dashboards with line graphs, heat maps, and histograms. It also supports alerts, which can be sent through channels like Slack to quickly notify teams of any issues. +- **[Evidently AI](https://www.evidentlyai.com/)**: Evidently AI is an open-source tool designed for monitoring and debugging machine learning models in production. It generates interactive reports from pandas DataFrames, helping analyze machine learning models. Evidently AI can detect data drift, model performance degradation, and other issues that may arise with your deployed models. + +The three tools introduced above, Evidently AI, Prometheus, and Grafana, can work together seamlessly as a fully open-source ML monitoring solution that is ready for production. Evidently AI is used to collect and calculate metrics, Prometheus stores these metrics, and Grafana displays them and sets up alerts. While there are many other tools available, this setup is an exciting open-source option that provides robust capabilities for monitoring and maintaining your models. + ++ +
+ +### Anomaly Detection and Alert Systems + +An anomaly is any data point or pattern that deviates quite a bit from what is expected. With respect to computer vision models, anomalies can be images that are very different from the ones the model was trained on. These unexpected images can be signs of issues like changes in data distribution, outliers, or behaviors that might reduce model performance. Setting up alert systems to detect these anomalies is an important part of model monitoring. + +By setting standard performance levels and limits for key metrics, you can catch problems early. When performance goes outside these limits, alerts are triggered, prompting quick fixes. Regularly updating and retraining models with new data keeps them relevant and accurate as the data changes. + +#### Things to Keep in Mind When Configuring Thresholds and Alerts + +When you are setting up your alert systems, keep these best practices in mind: + +- **Standardized Alerts**: Use consistent tools and formats for all alerts, such as email or messaging apps like Slack. Standardization makes it easier for you to quickly understand and respond to alerts. +- **Include Expected Behavior**: Alert messages should clearly state what went wrong, what was expected, and the timeframe evaluated. It helps you gauge the urgency and context of the alert. +- **Configurable Alerts**: Make alerts easily configurable to adapt to changing conditions. Allow yourself to edit thresholds, snooze, disable, or acknowledge alerts. + +### Data Drift Detection + +Data drift detection is a concept that helps identify when the statistical properties of the input data change over time, which can degrade model performance. Before you decide to retrain or adjust your models, this technique helps spot that there is an issue. Data drift deals with changes in the overall data landscape over time, while anomaly detection focuses on identifying rare or unexpected data points that may require immediate attention. + ++ +
+ +Here are several methods to detect data drift: + +**Continuous Monitoring**: Regularly monitor the model's input data and outputs for signs of drift. Track key metrics and compare them against historical data to identify significant changes. + +**Statistical Techniques**: Use methods like the Kolmogorov-Smirnov test or Population Stability Index (PSI) to detect changes in data distributions. These tests compare the distribution of new data with the training data to identify significant differences. + +**Feature Drift**: Monitor individual features for drift. Sometimes, the overall data distribution may remain stable, but individual features may drift. Identifying which features are drifting helps in fine-tuning the retraining process. + +## Model Maintenance + +Model maintenance is crucial to keep computer vision models accurate and relevant over time. Model maintenance involves regularly updating and retraining models, addressing data drift, and ensuring the model stays relevant as data and environments change. You might be wondering how model maintenance differs from model monitoring. Monitoring is about watching the model's performance in real time to catch issues early. Maintenance, on the other hand, is about fixing these issues. + +### Regular Updates and Re-training + +Once a model is deployed, while monitoring, you may notice changes in data patterns or performance, indicating model drift. Regular updates and re-training become essential parts of model maintenance to ensure the model can handle new patterns and scenarios. There are a few techniques you can use based on how your data is changing. + ++ +
+ +For example, if the data is changing gradually over time, incremental learning is a good approach. Incremental learning involves updating the model with new data without completely retraining it from scratch, saving computational resources and time. However, if the data has changed drastically, a periodic full re-training might be a better option to ensure the model does not overfit on the new data while losing track of older patterns. + +Regardless of the method, validation and testing are a must after updates. It is important to validate the model on a separate [test dataset](./model-testing.md) to check for performance improvements or degradation. + +### Deciding When to Retrain Your Model + +The frequency of retraining your computer vision model depends on data changes and model performance. Retrain your model whenever you observe a significant performance drop or detect data drift. Regular evaluations can help determine the right retraining schedule by testing the model against new data. Monitoring performance metrics and data patterns lets you decide if your model needs more frequent updates to maintain accuracy. + ++ +
+ +## Documentation + +Documenting a computer vision project makes it easier to understand, reproduce, and collaborate on. Good documentation covers model architecture, hyperparameters, datasets, evaluation metrics, and more. It provides transparency, helping team members and stakeholders understand what has been done and why. Documentation also aids in troubleshooting, maintenance, and future enhancements by providing a clear reference of past decisions and methods. + +### Key Elements to Document + +These are some of the key elements that should be included in project documentation: + +- **[Project Overview](./steps-of-a-cv-project.md)**: Provide a high-level summary of the project, including the problem statement, solution approach, expected outcomes, and project scope. Explain the role of computer vision in addressing the problem and outline the stages and deliverables. +- **Model Architecture**: Detail the structure and design of the model, including its components, layers, and connections. Explain the chosen hyperparameters and the rationale behind these choices. +- **[Data Preparation](./data-collection-and-annotation.md)**: Describe the data sources, types, formats, sizes, and preprocessing steps. Discuss data quality, reliability, and any transformations applied before training the model. +- **[Training Process](./model-training-tips.md)**: Document the training procedure, including the datasets used, training parameters, and loss functions. Explain how the model was trained and any challenges encountered during training. +- **[Evaluation Metrics](./model-evaluation-insights.md)**: Specify the metrics used to evaluate the model's performance, such as accuracy, precision, recall, and F1-score. Include performance results and an analysis of these metrics. +- **[Deployment Steps](./model-deployment-options.md)**: Outline the steps taken to deploy the model, including the tools and platforms used, deployment configurations, and any specific challenges or considerations. +- **Monitoring and Maintenance Procedure**: Provide a detailed plan for monitoring the model's performance post-deployment. Include methods for detecting and addressing data and model drift, and describe the process for regular updates and retraining. + +### Tools for Documentation + +There are many options when it comes to documenting AI projects, with open-source tools being particularly popular. Two of these are Jupyter Notebooks and MkDocs. Jupyter Notebooks allow you to create interactive documents with embedded code, visualizations, and text, making them ideal for sharing experiments and analyses. MkDocs is a static site generator that is easy to set up and deploy and is perfect for creating and hosting project documentation online. + +## Connect with the Community + +Joining a community of computer vision enthusiasts can help you solve problems and learn more quickly. Here are some ways to connect, get support, and share ideas. + +### Community Resources + +- **GitHub Issues:** Check out the [YOLOv8 GitHub repository](https://github.com/ultralytics/ultralytics/issues) and use the Issues tab to ask questions, report bugs, and suggest new features. The community and maintainers are highly active and supportive. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to chat with other users and developers, get support, and share your experiences. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Visit the [official YOLOv8 documentation](./index.md) for detailed guides and helpful tips on various computer vision projects. + +Using these resources will help you solve challenges and stay up-to-date with the latest trends and practices in the computer vision community. + +## Key Takeaways + +We covered key tips for monitoring, maintaining, and documenting your computer vision models. Regular updates and re-training help the model adapt to new data patterns. Detecting and fixing data drift helps your model stay accurate. Continuous monitoring catches issues early, and good documentation makes collaboration and future updates easier. Following these steps will help your computer vision project stay successful and effective over time. + +## FAQ + +### How do I monitor the performance of my deployed computer vision model? + +Monitoring the performance of your deployed computer vision model is crucial to ensure its accuracy and reliability over time. You can use tools like [Prometheus](https://prometheus.io/), [Grafana](https://grafana.com/), and [Evidently AI](https://www.evidentlyai.com/) to track key metrics, detect anomalies, and identify data drift. Regularly monitor inputs and outputs, set up alerts for unusual behavior, and use diverse data sources to get a comprehensive view of your model's performance. For more details, check out our section on [Model Monitoring](#model-monitoring-is-key). + +### What are the best practices for maintaining computer vision models after deployment? + +Maintaining computer vision models involves regular updates, retraining, and monitoring to ensure continued accuracy and relevance. Best practices include: + +- **Continuous Monitoring**: Track performance metrics and data quality regularly. +- **Data Drift Detection**: Use statistical techniques to identify changes in data distributions. +- **Regular Updates and Retraining**: Implement incremental learning or periodic full retraining based on data changes. +- **Documentation**: Maintain detailed documentation of model architecture, training processes, and evaluation metrics. For more insights, visit our [Model Maintenance](#model-maintenance) section. + +### Why is data drift detection important for AI models? + +Data drift detection is essential because it helps identify when the statistical properties of the input data change over time, which can degrade model performance. Techniques like continuous monitoring, statistical tests (e.g., Kolmogorov-Smirnov test), and feature drift analysis can help spot issues early. Addressing data drift ensures that your model remains accurate and relevant in changing environments. Learn more about data drift detection in our [Data Drift Detection](#data-drift-detection) section. + +### What tools can I use for anomaly detection in computer vision models? + +For anomaly detection in computer vision models, tools like [Prometheus](https://prometheus.io/), [Grafana](https://grafana.com/), and [Evidently AI](https://www.evidentlyai.com/) are highly effective. These tools can help you set up alert systems to detect unusual data points or patterns that deviate from expected behavior. Configurable alerts and standardized messages can help you respond quickly to potential issues. Explore more in our [Anomaly Detection and Alert Systems](#anomaly-detection-and-alert-systems) section. + +### How can I document my computer vision project effectively? + +Effective documentation of a computer vision project should include: + +- **Project Overview**: High-level summary, problem statement, and solution approach. +- **Model Architecture**: Details of the model structure, components, and hyperparameters. +- **Data Preparation**: Information on data sources, preprocessing steps, and transformations. +- **Training Process**: Description of the training procedure, datasets used, and challenges encountered. +- **Evaluation Metrics**: Metrics used for performance evaluation and analysis. +- **Deployment Steps**: Steps taken for model deployment and any specific challenges. +- **Monitoring and Maintenance Procedure**: Plan for ongoing monitoring and maintenance. For more comprehensive guidelines, refer to our [Documentation](#documentation) section. diff --git a/ultralytics/docs/en/guides/model-testing.md b/ultralytics/docs/en/guides/model-testing.md new file mode 100644 index 0000000000000000000000000000000000000000..432c976b067e7d08489e91883045925b9ebf5eda --- /dev/null +++ b/ultralytics/docs/en/guides/model-testing.md @@ -0,0 +1,200 @@ +--- +comments: true +description: Explore effective methods for testing computer vision models to make sure they are reliable, perform well, and are ready to be deployed. +keywords: Overfitting and Underfitting in Machine Learning, Model Testing, Data Leakage Machine Learning, Testing a Model, Testing Machine Learning Models, How to Test AI Models +--- + +# A Guide on Model Testing + +## Introduction + +After [training](./model-training-tips.md) and [evaluating](./model-evaluation-insights.md) your model, it's time to test it. Model testing involves assessing how well it performs in real-world scenarios. Testing considers factors like accuracy, reliability, fairness, and how easy it is to understand the model's decisions. The goal is to make sure the model performs as intended, delivers the expected results, and fits into the [overall objective of your application](./defining-project-goals.md) or project. + +Model testing is quite similar to model evaluation, but they are two distinct [steps in a computer vision project](./steps-of-a-cv-project.md). Model evaluation involves metrics and plots to assess the model's accuracy. On the other hand, model testing checks if the model's learned behavior is the same as expectations. In this guide, we'll explore strategies for testing your computer vision models. + +## Model Testing Vs. Model Evaluation + +First, let's understand the difference between model evaluation and testing with an example. + +Suppose you have trained a computer vision model to recognize cats and dogs, and you want to deploy this model at a pet store to monitor the animals. During the model evaluation phase, you use a labeled dataset to calculate metrics like accuracy, precision, recall, and F1 score. For instance, the model might have an accuracy of 98% in distinguishing between cats and dogs in a given dataset. + +After evaluation, you test the model using images from a pet store to see how well it identifies cats and dogs in more varied and realistic conditions. You check if it can correctly label cats and dogs when they are moving, in different lighting conditions, or partially obscured by objects like toys or furniture. Model testing checks that the model behaves as expected outside the controlled evaluation environment. + +## Preparing for Model Testing + +Computer vision models learn from datasets by detecting patterns, making predictions, and evaluating their performance. These [datasets](./preprocessing_annotated_data.md) are usually divided into training and testing sets to simulate real-world conditions. Training data teaches the model while testing data verifies its accuracy. + +Here are two points to keep in mind before testing your model: + +- **Realistic Representation:** The previously unseen testing data should be similar to the data that the model will have to handle when deployed. This helps get a realistic understanding of the model's capabilities. +- **Sufficient Size:** The size of the testing dataset needs to be large enough to provide reliable insights into how well the model performs. + +## Testing Your Computer Vision Model + +Here are the key steps to take to test your computer vision model and understand its performance. + +- **Run Predictions:** Use the model to make predictions on the test dataset. +- **Compare Predictions:** Check how well the model's predictions match the actual labels (ground truth). +- **Calculate Performance Metrics:** [Compute metrics](./yolo-performance-metrics.md) like accuracy, precision, recall, and F1 score to understand the model's strengths and weaknesses. Testing focuses on how these metrics reflect real-world performance. +- **Visualize Results:** Create visual aids like confusion matrices and ROC curves. These help you spot specific areas where the model might not be performing well in practical applications. + +Next, the testing results can be analyzed: + +- **Misclassified Images:** Identify and review images that the model misclassified to understand where it is going wrong. +- **Error Analysis:** Perform a thorough error analysis to understand the types of errors (e.g., false positives vs. false negatives) and their potential causes. +- **Bias and Fairness:** Check for any biases in the model's predictions. Ensure that the model performs equally well across different subsets of the data, especially if it includes sensitive attributes like race, gender, or age. + +## Testing Your YOLOv8 Model + +To test your YOLOv8 model, you can use the validation mode. It's a straightforward way to understand the model's strengths and areas that need improvement. Also, you'll need to format your test dataset correctly for YOLOv8. For more details on how to use the validation mode, check out the [Model Validation](../modes/val.md) docs page. + +## Using YOLOv8 to Predict on Multiple Test Images + +If you want to test your trained YOLOv8 model on multiple images stored in a folder, you can easily do so in one go. Instead of using the validation mode, which is typically used to evaluate model performance on a validation set and provide detailed metrics, you might just want to see predictions on all images in your test set. For this, you can use the [prediction mode](../modes/predict.md). + +### Difference Between Validation and Prediction Modes + +- **[Validation Mode](../modes/val.md):** Used to evaluate the model's performance by comparing predictions against known labels (ground truth). It provides detailed metrics such as accuracy, precision, recall, and F1 score. +- **[Prediction Mode](../modes/predict.md):** Used to run the model on new, unseen data to generate predictions. It does not provide detailed performance metrics but allows you to see how the model performs on real-world images. + +## Running YOLOv8 Predictions Without Custom Training + +If you are interested in testing the basic YOLOv8 model to understand whether it can be used for your application without custom training, you can use the prediction mode. While the model is pre-trained on datasets like COCO, running predictions on your own dataset can give you a quick sense of how well it might perform in your specific context. + +## Overfitting and Underfitting in Machine Learning + +When testing a machine learning model, especially in computer vision, it's important to watch out for overfitting and underfitting. These issues can significantly affect how well your model works with new data. + +### Overfitting + +Overfitting happens when your model learns the training data too well, including the noise and details that don't generalize to new data. In computer vision, this means your model might do great with training images but struggle with new ones. + +#### Signs of Overfitting + +- **High Training Accuracy, Low Validation Accuracy:** If your model performs very well on training data but poorly on validation or test data, it's likely overfitting. +- **Visual Inspection:** Sometimes, you can see overfitting if your model is too sensitive to minor changes or irrelevant details in images. + +### Underfitting + +Underfitting occurs when your model can't capture the underlying patterns in the data. In computer vision, an underfitted model might not even recognize objects correctly in the training images. + +#### Signs of Underfitting + +- **Low Training Accuracy:** If your model can't achieve high accuracy on the training set, it might be underfitting. +- **Visual Misclassification:** Consistent failure to recognize obvious features or objects suggests underfitting. + +### Balancing Overfitting and Underfitting + +The key is to find a balance between overfitting and underfitting. Ideally, a model should perform well on both training and validation datasets. Regularly monitoring your model's performance through metrics and visual inspections, along with applying the right strategies, can help you achieve the best results. + ++ +
+ +## Data Leakage in Computer Vision and How to Avoid It + +While testing your model, something important to keep in mind is data leakage. Data leakage happens when information from outside the training dataset accidentally gets used to train the model. The model may seem very accurate during training, but it won't perform well on new, unseen data when data leakage occurs. + +### Why Data Leakage Happens + +Data leakage can be tricky to spot and often comes from hidden biases in the training data. Here are some common ways it can happen in computer vision: + +- **Camera Bias:** Different angles, lighting, shadows, and camera movements can introduce unwanted patterns. +- **Overlay Bias:** Logos, timestamps, or other overlays in images can mislead the model. +- **Font and Object Bias:** Specific fonts or objects that frequently appear in certain classes can skew the model's learning. +- **Spatial Bias:** Imbalances in foreground-background, bounding box distributions, and object locations can affect training. +- **Label and Domain Bias:** Incorrect labels or shifts in data types can lead to leakage. + +### Detecting Data Leakage + +To find data leakage, you can: + +- **Check Performance:** If the model's results are surprisingly good, it might be leaking. +- **Look at Feature Importance:** If one feature is much more important than others, it could indicate leakage. +- **Visual Inspection:** Double-check that the model's decisions make sense intuitively. +- **Verify Data Separation:** Make sure data was divided correctly before any processing. + +### Avoiding Data Leakage + +To prevent data leakage, use a diverse dataset with images or videos from different cameras and environments. Carefully review your data and check that there are no hidden biases, such as all positive samples being taken at a specific time of day. Avoiding data leakage will help make your computer vision models more reliable and effective in real-world situations. + +## What Comes After Model Testing + +After testing your model, the next steps depend on the results. If your model performs well, you can deploy it into a real-world environment. If the results aren't satisfactory, you'll need to make improvements. This might involve analyzing errors, [gathering more data](./data-collection-and-annotation.md), improving data quality, [adjusting hyperparameters](./hyperparameter-tuning.md), and retraining the model. + +## Join the AI Conversation + +Becoming part of a community of computer vision enthusiasts can aid in solving problems and learning more efficiently. Here are some ways to connect, seek help, and share your thoughts. + +### Community Resources + +- **GitHub Issues:** Explore the [YOLOv8 GitHub repository](https://github.com/ultralytics/ultralytics/issues) and use the Issues tab to ask questions, report bugs, and suggest new features. The community and maintainers are very active and ready to help. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to chat with other users and developers, get support, and share your experiences. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Check out the [official YOLOv8 documentation](./index.md) for detailed guides and helpful tips on various computer vision projects. + +These resources will help you navigate challenges and remain updated on the latest trends and practices within the computer vision community. + +## In Summary + +Building trustworthy computer vision models relies on rigorous model testing. By testing the model with previously unseen data, we can analyze it and spot weaknesses like overfitting and data leakage. Addressing these issues before deployment helps the model perform well in real-world applications. It's important to remember that model testing is just as crucial as model evaluation in guaranteeing the model's long-term success and effectiveness. + +## FAQ + +### What are the key differences between model evaluation and model testing in computer vision? + +Model evaluation and model testing are distinct steps in a computer vision project. Model evaluation involves using a labeled dataset to compute metrics such as accuracy, precision, recall, and F1 score, providing insights into the model's performance with a controlled dataset. Model testing, on the other hand, assesses the model's performance in real-world scenarios by applying it to new, unseen data, ensuring the model's learned behavior aligns with expectations outside the evaluation environment. For a detailed guide, refer to the [steps in a computer vision project](./steps-of-a-cv-project.md). + +### How can I test my Ultralytics YOLOv8 model on multiple images? + +To test your Ultralytics YOLOv8 model on multiple images, you can use the [prediction mode](../modes/predict.md). This mode allows you to run the model on new, unseen data to generate predictions without providing detailed metrics. This is ideal for real-world performance testing on larger image sets stored in a folder. For evaluating performance metrics, use the [validation mode](../modes/val.md) instead. + +### What should I do if my computer vision model shows signs of overfitting or underfitting? + +To address **overfitting**: + +- Regularization techniques like dropout. +- Increase the size of the training dataset. +- Simplify the model architecture. + +To address **underfitting**: + +- Use a more complex model. +- Provide more relevant features. +- Increase training iterations or epochs. + +Review misclassified images, perform thorough error analysis, and regularly track performance metrics to maintain a balance. For more information on these concepts, explore our section on [Overfitting and Underfitting](#overfitting-and-underfitting-in-machine-learning). + +### How can I detect and avoid data leakage in computer vision? + +To detect data leakage: + +- Verify that the testing performance is not unusually high. +- Check feature importance for unexpected insights. +- Intuitively review model decisions. +- Ensure correct data division before processing. + +To avoid data leakage: + +- Use diverse datasets with various environments. +- Carefully review data for hidden biases. +- Ensure no overlapping information between training and testing sets. + +For detailed strategies on preventing data leakage, refer to our section on [Data Leakage in Computer Vision](#data-leakage-in-computer-vision-and-how-to-avoid-it). + +### What steps should I take after testing my computer vision model? + +Post-testing, if the model performance meets the project goals, proceed with deployment. If the results are unsatisfactory, consider: + +- Error analysis. +- Gathering more diverse and high-quality data. +- Hyperparameter tuning. +- Retraining the model. + +Gain insights from the [Model Testing Vs. Model Evaluation](#model-testing-vs-model-evaluation) section to refine and enhance model effectiveness in real-world applications. + +### How do I run YOLOv8 predictions without custom training? + +You can run predictions using the pre-trained YOLOv8 model on your dataset to see if it suits your application needs. Utilize the [prediction mode](../modes/predict.md) to get a quick sense of performance results without diving into custom training. diff --git a/ultralytics/docs/en/guides/model-training-tips.md b/ultralytics/docs/en/guides/model-training-tips.md new file mode 100644 index 0000000000000000000000000000000000000000..017f82b7cef1fcd7cfa113fdbda49e8714b7ab7d --- /dev/null +++ b/ultralytics/docs/en/guides/model-training-tips.md @@ -0,0 +1,182 @@ +--- +comments: true +description: Find best practices, optimization strategies, and troubleshooting advice for training computer vision models. Improve your model training efficiency and accuracy. +keywords: Model Training Machine Learning, AI Model Training, Number of Epochs, How to Train a Model in Machine Learning, Machine Learning Best Practices, What is Model Training +--- + +# Machine Learning Best Practices and Tips for Model Training + +## Introduction + +One of the most important steps when working on a [computer vision project](./steps-of-a-cv-project.md) is model training. Before reaching this step, you need to [define your goals](./defining-project-goals.md) and [collect and annotate your data](./data-collection-and-annotation.md). After [preprocessing the data](./preprocessing_annotated_data.md) to make sure it is clean and consistent, you can move on to training your model. + +So, what is [model training](../modes/train.md)? Model training is the process of teaching your model to recognize visual patterns and make predictions based on your data. It directly impacts the performance and accuracy of your application. In this guide, we'll cover best practices, optimization techniques, and troubleshooting tips to help you train your computer vision models effectively. + +## How to Train a Machine Learning Model + +A computer vision model is trained by adjusting its internal parameters to minimize errors. Initially, the model is fed a large set of labeled images. It makes predictions about what is in these images, and the predictions are compared to the actual labels or contents to calculate errors. These errors show how far off the model's predictions are from the true values. + +During training, the model iteratively makes predictions, calculates errors, and updates its parameters through a process called backpropagation. In this process, the model adjusts its internal parameters (weights and biases) to reduce the errors. By repeating this cycle many times, the model gradually improves its accuracy. Over time, it learns to recognize complex patterns such as shapes, colors, and textures. + ++ +
+ +This learning process makes it possible for the computer vision model to perform various [tasks](../tasks/index.md), including [object detection](../tasks/detect.md), [instance segmentation](../tasks/segment.md), and [image classification](../tasks/classify.md). The ultimate goal is to create a model that can generalize its learning to new, unseen images so that it can accurately understand visual data in real-world applications. + +Now that we know what is happening behind the scenes when we train a model, let's look at points to consider when training a model. + +## Training on Large Datasets + +There are a few different aspects to think about when you are planning on using a large dataset to train a model. For example, you can adjust the batch size, control the GPU utilization, choose to use multiscale training, etc. Let's walk through each of these options in detail. + +### Batch Size and GPU Utilization + +When training models on large datasets, efficiently utilizing your GPU is key. Batch size is an important factor. It is the number of data samples that a machine learning model processes in a single training iteration. +Using the maximum batch size supported by your GPU, you can fully take advantage of its capabilities and reduce the time model training takes. However, you want to avoid running out of GPU memory. If you encounter memory errors, reduce the batch size incrementally until the model trains smoothly. + +With respect to YOLOv8, you can set the `batch_size` parameter in the [training configuration](../modes/train.md) to match your GPU capacity. Also, setting `batch=-1` in your training script will automatically determine the batch size that can be efficiently processed based on your device's capabilities. By fine-tuning the batch size, you can make the most of your GPU resources and improve the overall training process. + +### Subset Training + +Subset training is a smart strategy that involves training your model on a smaller set of data that represents the larger dataset. It can save time and resources, especially during initial model development and testing. If you are running short on time or experimenting with different model configurations, subset training is a good option. + +When it comes to YOLOv8, you can easily implement subset training by using the `fraction` parameter. This parameter lets you specify what fraction of your dataset to use for training. For example, setting `fraction=0.1` will train your model on 10% of the data. You can use this technique for quick iterations and tuning your model before committing to training a model using a full dataset. Subset training helps you make rapid progress and identify potential issues early on. + +### Multi-scale Training + +Multiscale training is a technique that improves your model's ability to generalize by training it on images of varying sizes. Your model can learn to detect objects at different scales and distances and become more robust. + +For example, when you train YOLOv8, you can enable multiscale training by setting the `scale` parameter. This parameter adjusts the size of training images by a specified factor, simulating objects at different distances. For example, setting `scale=0.5` will reduce the image size by half, while `scale=2.0` will double it. Configuring this parameter allows your model to experience a variety of image scales and improve its detection capabilities across different object sizes and scenarios. + +### Caching + +Caching is an important technique to improve the efficiency of training machine learning models. By storing preprocessed images in memory, caching reduces the time the GPU spends waiting for data to be loaded from the disk. The model can continuously receive data without delays caused by disk I/O operations. + +Caching can be controlled when training YOLOv8 using the `cache` parameter: + +- _`cache=True`_: Stores dataset images in RAM, providing the fastest access speed but at the cost of increased memory usage. +- _`cache='disk'`_: Stores the images on disk, slower than RAM but faster than loading fresh data each time. +- _`cache=False`_: Disables caching, relying entirely on disk I/O, which is the slowest option. + +### Mixed Precision Training + +Mixed precision training uses both 16-bit (FP16) and 32-bit (FP32) floating-point types. The strengths of both FP16 and FP32 are leveraged by using FP16 for faster computation and FP32 to maintain precision where needed. Most of the neural network's operations are done in FP16 to benefit from faster computation and lower memory usage. However, a master copy of the model's weights is kept in FP32 to ensure accuracy during the weight update steps. You can handle larger models or larger batch sizes within the same hardware constraints. + ++ +
+ +To implement mixed precision training, you'll need to modify your training scripts and ensure your hardware (like GPUs) supports it. Many modern deep learning frameworks, such as Tensorflow, offer built-in support for mixed precision. + +Mixed precision training is straightforward when working with YOLOv8. You can use the `amp` flag in your training configuration. Setting `amp=True` enables Automatic Mixed Precision (AMP) training. Mixed precision training is a simple yet effective way to optimize your model training process. + +### Pre-trained Weights + +Using pretrained weights is a smart way to speed up your model's training process. Pretrained weights come from models already trained on large datasets, giving your model a head start. Transfer learning adapts pretrained models to new, related tasks. Fine-tuning a pre-trained model involves starting with these weights and then continuing training on your specific dataset. This method of training results in faster training times and often better performance because the model starts with a solid understanding of basic features. + +The `pretrained` parameter makes transfer learning easy with YOLOv8. Setting `pretrained=True` will use default pre-trained weights, or you can specify a path to a custom pre-trained model. Using pre-trained weights and transfer learning effectively boosts your model's capabilities and reduces training costs. + +### Other Techniques to Consider When Handling a Large Dataset + +There are a couple of other techniques to consider when handling a large dataset: + +- **Learning Rate Schedulers**: Implementing learning rate schedulers dynamically adjusts the learning rate during training. A well-tuned learning rate can prevent the model from overshooting minima and improve stability. When training YOLOv8, the `lrf` parameter helps manage learning rate scheduling by setting the final learning rate as a fraction of the initial rate. +- **Distributed Training**: For handling large datasets, distributed training can be a game-changer. You can reduce the training time by spreading the training workload across multiple GPUs or machines. + +## The Number of Epochs To Train For + +When training a model, an epoch refers to one complete pass through the entire training dataset. During an epoch, the model processes each example in the training set once and updates its parameters based on the learning algorithm. Multiple epochs are usually needed to allow the model to learn and refine its parameters over time. + +A common question that comes up is how to determine the number of epochs to train the model for. A good starting point is 300 epochs. If the model overfits early, you can reduce the number of epochs. If overfitting does not occur after 300 epochs, you can extend the training to 600, 1200, or more epochs. + +However, the ideal number of epochs can vary based on your dataset's size and project goals. Larger datasets might require more epochs for the model to learn effectively, while smaller datasets might need fewer epochs to avoid overfitting. With respect to YOLOv8, you can set the `epochs` parameter in your training script. + +## Early Stopping + +Early stopping is a valuable technique for optimizing model training. By monitoring validation performance, you can halt training once the model stops improving. You can save computational resources and prevent overfitting. + +The process involves setting a patience parameter that determines how many epochs to wait for an improvement in validation metrics before stopping training. If the model's performance does not improve within these epochs, training is stopped to avoid wasting time and resources. + ++ +
+ +For YOLOv8, you can enable early stopping by setting the patience parameter in your training configuration. For example, `patience=5` means training will stop if there's no improvement in validation metrics for 5 consecutive epochs. Using this method ensures the training process remains efficient and achieves optimal performance without excessive computation. + +## Choosing Between Cloud and Local Training + +There are two options for training your model: cloud training and local training. + +Cloud training offers scalability and powerful hardware and is ideal for handling large datasets and complex models. Platforms like Google Cloud, AWS, and Azure provide on-demand access to high-performance GPUs and TPUs, speeding up training times and enabling experiments with larger models. However, cloud training can be expensive, especially for long periods, and data transfer can add to costs and latency. + +Local training provides greater control and customization, letting you tailor your environment to specific needs and avoid ongoing cloud costs. It can be more economical for long-term projects, and since your data stays on-premises, it's more secure. However, local hardware may have resource limitations and require maintenance, which can lead to longer training times for large models. + +## Selecting an Optimizer + +An optimizer is an algorithm that adjusts the weights of your neural network to minimize the loss function, which measures how well the model is performing. In simpler terms, the optimizer helps the model learn by tweaking its parameters to reduce errors. Choosing the right optimizer directly affects how quickly and accurately the model learns. + +You can also fine-tune optimizer parameters to improve model performance. Adjusting the learning rate sets the size of the steps when updating parameters. For stability, you might start with a moderate learning rate and gradually decrease it over time to improve long-term learning. Additionally, setting the momentum determines how much influence past updates have on current updates. A common value for momentum is around 0.9. It generally provides a good balance. + +### Common Optimizers + +Different optimizers have various strengths and weaknesses. Let's take a glimpse at a few common optimizers. + +- **SGD (Stochastic Gradient Descent)**: + + - Updates model parameters using the gradient of the loss function with respect to the parameters. + - Simple and efficient but can be slow to converge and might get stuck in local minima. + +- **Adam (Adaptive Moment Estimation)**: + + - Combines the benefits of both SGD with momentum and RMSProp. + - Adjusts the learning rate for each parameter based on estimates of the first and second moments of the gradients. + - Well-suited for noisy data and sparse gradients. + - Efficient and generally requires less tuning, making it a recommended optimizer for YOLOv8. + +- **RMSProp (Root Mean Square Propagation)**: + - Adjusts the learning rate for each parameter by dividing the gradient by a running average of the magnitudes of recent gradients. + - Helps in handling the vanishing gradient problem and is effective for recurrent neural networks. + +For YOLOv8, the `optimizer` parameter lets you choose from various optimizers, including SGD, Adam, AdamW, NAdam, RAdam, and RMSProp, or you can set it to `auto` for automatic selection based on model configuration. + +## Connecting with the Community + +Being part of a community of computer vision enthusiasts can help you solve problems and learn faster. Here are some ways to connect, get help, and share ideas. + +### Community Resources + +- **GitHub Issues:** Visit the [YOLOv8 GitHub repository](https://github.com/ultralytics/ultralytics/issues) and use the Issues tab to ask questions, report bugs, and suggest new features. The community and maintainers are very active and ready to help. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to chat with other users and developers, get support, and share your experiences. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Check out the [official YOLOv8 documentation](./index.md) for detailed guides and helpful tips on various computer vision projects. + +Using these resources will help you solve challenges and stay up-to-date with the latest trends and practices in the computer vision community. + +## Key Takeaways + +Training computer vision models involves following good practices, optimizing your strategies, and solving problems as they arise. Techniques like adjusting batch sizes, mixed precision training, and starting with pre-trained weights can make your models work better and train faster. Methods like subset training and early stopping help you save time and resources. Staying connected with the community and keeping up with new trends will help you keep improving your model training skills. + +## FAQ + +### How can I improve GPU utilization when training a large dataset with Ultralytics YOLO? + +To improve GPU utilization, set the `batch_size` parameter in your training configuration to the maximum size supported by your GPU. This ensures that you make full use of the GPU's capabilities, reducing training time. If you encounter memory errors, incrementally reduce the batch size until training runs smoothly. For YOLOv8, setting `batch=-1` in your training script will automatically determine the optimal batch size for efficient processing. For further information, refer to the [training configuration](../modes/train.md). + +### What is mixed precision training, and how do I enable it in YOLOv8? + +Mixed precision training utilizes both 16-bit (FP16) and 32-bit (FP32) floating-point types to balance computational speed and precision. This approach speeds up training and reduces memory usage without sacrificing model accuracy. To enable mixed precision training in YOLOv8, set the `amp` parameter to `True` in your training configuration. This activates Automatic Mixed Precision (AMP) training. For more details on this optimization technique, see the [training configuration](../modes/train.md). + +### How does multiscale training enhance YOLOv8 model performance? + +Multiscale training enhances model performance by training on images of varying sizes, allowing the model to better generalize across different scales and distances. In YOLOv8, you can enable multiscale training by setting the `scale` parameter in the training configuration. For example, `scale=0.5` reduces the image size by half, while `scale=2.0` doubles it. This technique simulates objects at different distances, making the model more robust across various scenarios. For settings and more details, check out the [training configuration](../modes/train.md). + +### How can I use pre-trained weights to speed up training in YOLOv8? + +Using pre-trained weights can significantly reduce training times and improve model performance by starting from a model that already understands basic features. In YOLOv8, you can set the `pretrained` parameter to `True` or specify a path to custom pre-trained weights in your training configuration. This approach, known as transfer learning, leverages knowledge from large datasets to adapt to your specific task. Learn more about pre-trained weights and their advantages [here](../modes/train.md). + +### What is the recommended number of epochs for training a model, and how do I set this in YOLOv8? + +The number of epochs refers to the complete passes through the training dataset during model training. A typical starting point is 300 epochs. If your model overfits early, you can reduce the number. Alternatively, if overfitting isn't observed, you might extend training to 600, 1200, or more epochs. To set this in YOLOv8, use the `epochs` parameter in your training script. For additional advice on determining the ideal number of epochs, refer to this section on [number of epochs](#the-number-of-epochs-to-train-for). diff --git a/ultralytics/docs/en/guides/nvidia-jetson.md b/ultralytics/docs/en/guides/nvidia-jetson.md new file mode 100644 index 0000000000000000000000000000000000000000..caf8d75fdec5c8a300ff0cd8cd371159c34f237c --- /dev/null +++ b/ultralytics/docs/en/guides/nvidia-jetson.md @@ -0,0 +1,466 @@ +--- +comments: true +description: Learn to deploy Ultralytics YOLOv8 on NVIDIA Jetson devices with our detailed guide. Explore performance benchmarks and maximize AI capabilities. +keywords: Ultralytics, YOLOv8, NVIDIA Jetson, JetPack, AI deployment, performance benchmarks, embedded systems, deep learning, TensorRT, computer vision +--- + +# Quick Start Guide: NVIDIA Jetson with Ultralytics YOLOv8 + +This comprehensive guide provides a detailed walkthrough for deploying Ultralytics YOLOv8 on [NVIDIA Jetson](https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/) devices. Additionally, it showcases performance benchmarks to demonstrate the capabilities of YOLOv8 on these small and powerful devices. + +
+
+
+
+ Watch: How to Setup NVIDIA Jetson with Ultralytics YOLOv8
+
+
+
+
+ Watch: Object Blurring using Ultralytics YOLOv8
+
+
+ + Watch: Object Counting using Ultralytics YOLOv8 + |
+
+
+ + Watch: Class-wise Object Counting using Ultralytics YOLOv8 + |
+
+
+
+
+ Watch: Object Cropping using Ultralytics YOLOv8
+
+
+
+
+ Watch: How to Implement Parking Management Using Ultralytics YOLOv8 🚀
+
+ +
+ +With respect to YOLOv8, you can [augment your custom dataset](../modes/train.md) by modifying the dataset configuration file, a .yaml file. In this file, you can add an augmentation section with parameters that specify how you want to augment your data. + +The [Ultralytics YOLOv8 repository](https://github.com/ultralytics/ultralytics/tree/main) supports a wide range of data augmentations. You can apply various transformations such as: + +- Random Crops +- Flipping: Images can be flipped horizontally or vertically. +- Rotation: Images can be rotated by specific angles. +- Distortion + +Also, you can adjust the intensity of these augmentation techniques through specific parameters to generate more data variety. + +## A Case Study of Preprocessing + +Consider a project aimed at developing a model to detect and classify different types of vehicles in traffic images using YOLOv8. We've collected traffic images and annotated them with bounding boxes and labels. + +Here's what each step of preprocessing would look like for this project: + +- Resizing Images: Since YOLOv8 handles flexible input sizes and performs resizing automatically, manual resizing is not required. The model will adjust the image size according to the specified 'imgsz' parameter during training. +- Normalizing Pixel Values: YOLOv8 automatically normalizes pixel values to a range of 0 to 1 during preprocessing, so it's not required. +- Splitting the Dataset: Divide the dataset into training (70%), validation (20%), and test (10%) sets using tools like scikit-learn. +- Data Augmentation: Modify the dataset configuration file (.yaml) to include data augmentation techniques such as random crops, horizontal flips, and brightness adjustments. + +These steps make sure the dataset is prepared without any potential issues and is ready for Exploratory Data Analysis (EDA). + +## Exploratory Data Analysis Techniques + +After preprocessing and augmenting your dataset, the next step is to gain insights through Exploratory Data Analysis. EDA uses statistical techniques and visualization tools to understand the patterns and distributions in your data. You can identify issues like class imbalances or outliers and make informed decisions about further data preprocessing or model training adjustments. + +### Statistical EDA Techniques + +Statistical techniques often begin with calculating basic metrics such as mean, median, standard deviation, and range. These metrics provide a quick overview of your image dataset's properties, such as pixel intensity distributions. Understanding these basic statistics helps you grasp the overall quality and characteristics of your data, allowing you to spot any irregularities early on. + +### Visual EDA Techniques + +Visualizations are key in EDA for image datasets. For example, class imbalance analysis is another vital aspect of EDA. It helps determine if certain classes are underrepresented in your dataset, Visualizing the distribution of different image classes or categories using bar charts can quickly reveal any imbalances. Similarly, outliers can be identified using visualization tools like box plots, which highlight anomalies in pixel intensity or feature distributions. Outlier detection prevents unusual data points from skewing your results. + +Common tools for visualizations include: + +- Histograms and Box Plots: Useful for understanding the distribution of pixel values and identifying outliers. +- Scatter Plots: Helpful for exploring relationships between image features or annotations. +- Heatmaps: Effective for visualizing the distribution of pixel intensities or the spatial distribution of annotated features within images. + +### Using Ultralytics Explorer for EDA + +For a more advanced approach to EDA, you can use the Ultralytics Explorer tool. It offers robust capabilities for exploring computer vision datasets. By supporting semantic search, SQL queries, and vector similarity search, the tool makes it easy to analyze and understand your data. With Ultralytics Explorer, you can create embeddings for your dataset to find similar images, run SQL queries for detailed analysis, and perform semantic searches, all through a user-friendly graphical interface. + ++ +
+ +## Reach Out and Connect + +Having discussions about your project with other computer vision enthusiasts can give you new ideas from different perspectives. Here are some great ways to learn, troubleshoot, and network: + +### Channels to Connect with the Community + +- **GitHub Issues:** Visit the YOLOv8 GitHub repository and use the [Issues tab](https://github.com/ultralytics/ultralytics/issues) to raise questions, report bugs, and suggest features. The community and maintainers are there to help with any issues you face. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to connect with other users and developers, get support, share knowledge, and brainstorm ideas. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Refer to the [official YOLOv8 documentation](./index.md) for thorough guides and valuable insights on numerous computer vision tasks and projects. + +## Your Dataset Is Ready! + +Properly resized, normalized, and augmented data improves model performance by reducing noise and improving generalization. By following the preprocessing techniques and best practices outlined in this guide, you can create a solid dataset. With your preprocessed dataset ready, you can confidently proceed to the next steps in your project. + +## FAQ + +### What is the importance of data preprocessing in computer vision projects? + +Data preprocessing is essential in computer vision projects because it ensures that the data is clean, consistent, and in a format that is optimal for model training. By addressing issues such as noise, inconsistency, and imbalance in raw data, preprocessing steps like resizing, normalization, augmentation, and dataset splitting help reduce computational load and improve model performance. For more details, visit the [steps of a computer vision project](../guides/steps-of-a-cv-project.md). + +### How can I use Ultralytics YOLO for data augmentation? + +For data augmentation with Ultralytics YOLOv8, you need to modify the dataset configuration file (.yaml). In this file, you can specify various augmentation techniques such as random crops, horizontal flips, and brightness adjustments. This can be effectively done using the training configurations [explained here](../modes/train.md). Data augmentation helps create a more robust dataset, reduce overfitting, and improve model generalization. + +### What are the best data normalization techniques for computer vision data? + +Normalization scales pixel values to a standard range for faster convergence and improved performance during training. Common techniques include: + +- **Min-Max Scaling**: Scales pixel values to a range of 0 to 1. +- **Z-Score Normalization**: Scales pixel values based on their mean and standard deviation. + +For YOLOv8, normalization is handled automatically, including conversion to RGB and pixel value scaling. Learn more about it in the [model training section](../modes/train.md). + +### How should I split my annotated dataset for training? + +To split your dataset, a common practice is to divide it into 70% for training, 20% for validation, and 10% for testing. It is important to maintain the data distribution of classes across these splits and avoid data leakage by performing augmentation only on the training set. Use tools like scikit-learn or TensorFlow for efficient dataset splitting. See the detailed guide on [dataset preparation](../guides/data-collection-and-annotation.md). + +### Can I handle varying image sizes in YOLOv8 without manual resizing? + +Yes, Ultralytics YOLOv8 can handle varying image sizes through the 'imgsz' parameter during model training. This parameter ensures that images are resized so their largest dimension matches the specified size (e.g., 640 pixels), while maintaining the aspect ratio. For more flexible input handling and automatic adjustments, check the [model training section](../modes/train.md). diff --git a/ultralytics/docs/en/guides/queue-management.md b/ultralytics/docs/en/guides/queue-management.md new file mode 100644 index 0000000000000000000000000000000000000000..6ac4c2eda08db70651d417353d3f7056ad7c9f89 --- /dev/null +++ b/ultralytics/docs/en/guides/queue-management.md @@ -0,0 +1,250 @@ +--- +comments: true +description: Learn how to manage and optimize queues using Ultralytics YOLOv8 to reduce wait times and increase efficiency in various real-world applications. +keywords: queue management, YOLOv8, Ultralytics, reduce wait times, efficiency, customer satisfaction, retail, airports, healthcare, banks +--- + +# Queue Management using Ultralytics YOLOv8 🚀 + +## What is Queue Management? + +Queue management using [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics/) involves organizing and controlling lines of people or vehicles to reduce wait times and enhance efficiency. It's about optimizing queues to improve customer satisfaction and system performance in various settings like retail, banks, airports, and healthcare facilities. + +
+
+
+
+ Watch: How to Implement Queue Management with Ultralytics YOLOv8 | Airport and Metro Station
+
+
+
+
+ Watch: Raspberry Pi 5 updates and improvements.
+
+
+
+
+ Watch: Ultralytics YOLOv8 Object Counting in Multiple & Movable Regions
+
+
ROS Introduction (captioned) from Open Robotics on Vimeo.
+ +## What is ROS? + +The [Robot Operating System (ROS)](https://www.ros.org/) is an open-source framework widely used in robotics research and industry. ROS provides a collection of [libraries and tools](https://www.ros.org/blog/ecosystem/) to help developers create robot applications. ROS is designed to work with various [robotic platforms](https://robots.ros.org/), making it a flexible and powerful tool for roboticists. + +### Key Features of ROS + +1. **Modular Architecture**: ROS has a modular architecture, allowing developers to build complex systems by combining smaller, reusable components called [nodes](https://wiki.ros.org/ROS/Tutorials/UnderstandingNodes). Each node typically performs a specific function, and nodes communicate with each other using messages over [topics](https://wiki.ros.org/ROS/Tutorials/UnderstandingTopics) or [services](https://wiki.ros.org/ROS/Tutorials/UnderstandingServicesParams). + +2. **Communication Middleware**: ROS offers a robust communication infrastructure that supports inter-process communication and distributed computing. This is achieved through a publish-subscribe model for data streams (topics) and a request-reply model for service calls. + +3. **Hardware Abstraction**: ROS provides a layer of abstraction over the hardware, enabling developers to write device-agnostic code. This allows the same code to be used with different hardware setups, facilitating easier integration and experimentation. + +4. **Tools and Utilities**: ROS comes with a rich set of tools and utilities for visualization, debugging, and simulation. For instance, RViz is used for visualizing sensor data and robot state information, while Gazebo provides a powerful simulation environment for testing algorithms and robot designs. + +5. **Extensive Ecosystem**: The ROS ecosystem is vast and continually growing, with numerous packages available for different robotic applications, including navigation, manipulation, perception, and more. The community actively contributes to the development and maintenance of these packages. + +???+ note "Evolution of ROS Versions" + + Since its development in 2007, ROS has evolved through [multiple versions](https://wiki.ros.org/Distributions), each introducing new features and improvements to meet the growing needs of the robotics community. The development of ROS can be categorized into two main series: ROS 1 and ROS 2. This guide focuses on the Long Term Support (LTS) version of ROS 1, known as ROS Noetic Ninjemys, the code should also work with earlier versions. + + ### ROS 1 vs. ROS 2 + + While ROS 1 provided a solid foundation for robotic development, ROS 2 addresses its shortcomings by offering: + + - **Real-time Performance**: Improved support for real-time systems and deterministic behavior. + - **Security**: Enhanced security features for safe and reliable operation in various environments. + - **Scalability**: Better support for multi-robot systems and large-scale deployments. + - **Cross-platform Support**: Expanded compatibility with various operating systems beyond Linux, including Windows and macOS. + - **Flexible Communication**: Use of DDS for more flexible and efficient inter-process communication. + +### ROS Messages and Topics + +In ROS, communication between nodes is facilitated through [messages](https://wiki.ros.org/Messages) and [topics](https://wiki.ros.org/Topics). A message is a data structure that defines the information exchanged between nodes, while a topic is a named channel over which messages are sent and received. Nodes can publish messages to a topic or subscribe to messages from a topic, enabling them to communicate with each other. This publish-subscribe model allows for asynchronous communication and decoupling between nodes. Each sensor or actuator in a robotic system typically publishes data to a topic, which can then be consumed by other nodes for processing or control. For the purpose of this guide, we will focus on Image, Depth and PointCloud messages and camera topics. + +## Setting Up Ultralytics YOLO with ROS + +This guide has been tested using [this ROS environment](https://github.com/ambitious-octopus/rosbot_ros/tree/noetic), which is a fork of the [ROSbot ROS repository](https://github.com/husarion/rosbot_ros). This environment includes the Ultralytics YOLO package, a Docker container for easy setup, comprehensive ROS packages, and Gazebo worlds for rapid testing. It is designed to work with the [Husarion ROSbot 2 PRO](https://husarion.com/manuals/rosbot/). The code examples provided will work in any ROS Noetic/Melodic environment, including both simulation and real-world. + ++ +
+ +### Dependencies Installation + +Apart from the ROS environment, you will need to install the following dependencies: + +- **[ROS Numpy package](https://github.com/eric-wieser/ros_numpy)**: This is required for fast conversion between ROS Image messages and numpy arrays. + + ```bash + pip install ros_numpy + ``` + +- **Ultralytics package**: + + ```bash + pip install ultralytics + ``` + +## Use Ultralytics with ROS `sensor_msgs/Image` + +The `sensor_msgs/Image` [message type](https://docs.ros.org/en/api/sensor_msgs/html/msg/Image.html) is commonly used in ROS for representing image data. It contains fields for encoding, height, width, and pixel data, making it suitable for transmitting images captured by cameras or other sensors. Image messages are widely used in robotic applications for tasks such as visual perception, object detection, and navigation. + ++ +
+ +### Image Step-by-Step Usage + +The following code snippet demonstrates how to use the Ultralytics YOLO package with ROS. In this example, we subscribe to a camera topic, process the incoming image using YOLO, and publish the detected objects to new topics for [detection](../tasks/detect.md) and [segmentation](../tasks/segment.md). + +First, import the necessary libraries and instantiate two models: one for [segmentation](../tasks/segment.md) and one for [detection](../tasks/detect.md). Initialize a ROS node (with the name `ultralytics`) to enable communication with the ROS master. To ensure a stable connection, we include a brief pause, giving the node sufficient time to establish the connection before proceeding. + +```python +import time + +import rospy + +from ultralytics import YOLO + +detection_model = YOLO("yolov8m.pt") +segmentation_model = YOLO("yolov8m-seg.pt") +rospy.init_node("ultralytics") +time.sleep(1) +``` + +Initialize two ROS topics: one for [detection](../tasks/detect.md) and one for [segmentation](../tasks/segment.md). These topics will be used to publish the annotated images, making them accessible for further processing. The communication between nodes is facilitated using `sensor_msgs/Image` messages. + +```python +from sensor_msgs.msg import Image + +det_image_pub = rospy.Publisher("/ultralytics/detection/image", Image, queue_size=5) +seg_image_pub = rospy.Publisher("/ultralytics/segmentation/image", Image, queue_size=5) +``` + +Finally, create a subscriber that listens to messages on the `/camera/color/image_raw` topic and calls a callback function for each new message. This callback function receives messages of type `sensor_msgs/Image`, converts them into a numpy array using `ros_numpy`, processes the images with the previously instantiated YOLO models, annotates the images, and then publishes them back to the respective topics: `/ultralytics/detection/image` for detection and `/ultralytics/segmentation/image` for segmentation. + +```python +import ros_numpy + + +def callback(data): + """Callback function to process image and publish annotated images.""" + array = ros_numpy.numpify(data) + if det_image_pub.get_num_connections(): + det_result = detection_model(array) + det_annotated = det_result[0].plot(show=False) + det_image_pub.publish(ros_numpy.msgify(Image, det_annotated, encoding="rgb8")) + + if seg_image_pub.get_num_connections(): + seg_result = segmentation_model(array) + seg_annotated = seg_result[0].plot(show=False) + seg_image_pub.publish(ros_numpy.msgify(Image, seg_annotated, encoding="rgb8")) + + +rospy.Subscriber("/camera/color/image_raw", Image, callback) + +while True: + rospy.spin() +``` + +??? Example "Complete code" + + ```python + import time + + import ros_numpy + import rospy + from sensor_msgs.msg import Image + + from ultralytics import YOLO + + detection_model = YOLO("yolov8m.pt") + segmentation_model = YOLO("yolov8m-seg.pt") + rospy.init_node("ultralytics") + time.sleep(1) + + det_image_pub = rospy.Publisher("/ultralytics/detection/image", Image, queue_size=5) + seg_image_pub = rospy.Publisher("/ultralytics/segmentation/image", Image, queue_size=5) + + + def callback(data): + """Callback function to process image and publish annotated images.""" + array = ros_numpy.numpify(data) + if det_image_pub.get_num_connections(): + det_result = detection_model(array) + det_annotated = det_result[0].plot(show=False) + det_image_pub.publish(ros_numpy.msgify(Image, det_annotated, encoding="rgb8")) + + if seg_image_pub.get_num_connections(): + seg_result = segmentation_model(array) + seg_annotated = seg_result[0].plot(show=False) + seg_image_pub.publish(ros_numpy.msgify(Image, seg_annotated, encoding="rgb8")) + + + rospy.Subscriber("/camera/color/image_raw", Image, callback) + + while True: + rospy.spin() + ``` + +???+ tip "Debugging" + + Debugging ROS (Robot Operating System) nodes can be challenging due to the system's distributed nature. Several tools can assist with this process: + + 1. `rostopic echo+ +
+ +The `sensor_msgs/PointCloud2` [message type](https://docs.ros.org/en/api/sensor_msgs/html/msg/PointCloud2.html) is a data structure used in ROS to represent 3D point cloud data. This message type is integral to robotic applications, enabling tasks such as 3D mapping, object recognition, and localization. + +A point cloud is a collection of data points defined within a three-dimensional coordinate system. These data points represent the external surface of an object or a scene, captured via 3D scanning technologies. Each point in the cloud has `X`, `Y`, and `Z` coordinates, which correspond to its position in space, and may also include additional information such as color and intensity. + +!!! warning "Reference frame" + + When working with `sensor_msgs/PointCloud2`, it's essential to consider the reference frame of the sensor from which the point cloud data was acquired. The point cloud is initially captured in the sensor's reference frame. You can determine this reference frame by listening to the `/tf_static` topic. However, depending on your specific application requirements, you might need to convert the point cloud into another reference frame. This transformation can be achieved using the `tf2_ros` package, which provides tools for managing coordinate frames and transforming data between them. + +!!! tip "Obtaining Point clouds" + + Point Clouds can be obtained using various sensors: + + 1. **LIDAR (Light Detection and Ranging)**: Uses laser pulses to measure distances to objects and create high-precision 3D maps. + 2. **Depth Cameras**: Capture depth information for each pixel, allowing for 3D reconstruction of the scene. + 3. **Stereo Cameras**: Utilize two or more cameras to obtain depth information through triangulation. + 4. **Structured Light Scanners**: Project a known pattern onto a surface and measure the deformation to calculate depth. + +### Using YOLO with Point Clouds + +To integrate YOLO with `sensor_msgs/PointCloud2` type messages, we can employ a method similar to the one used for depth maps. By leveraging the color information embedded in the point cloud, we can extract a 2D image, perform segmentation on this image using YOLO, and then apply the resulting mask to the three-dimensional points to isolate the 3D object of interest. + +For handling point clouds, we recommend using Open3D (`pip install open3d`), a user-friendly Python library. Open3D provides robust tools for managing point cloud data structures, visualizing them, and executing complex operations seamlessly. This library can significantly simplify the process and enhance our ability to manipulate and analyze point clouds in conjunction with YOLO-based segmentation. + +#### Point Clouds Step-by-Step Usage + +Import the necessary libraries and instantiate the YOLO model for segmentation. + +```python +import time + +import rospy + +from ultralytics import YOLO + +rospy.init_node("ultralytics") +time.sleep(1) +segmentation_model = YOLO("yolov8m-seg.pt") +``` + +Create a function `pointcloud2_to_array`, which transforms a `sensor_msgs/PointCloud2` message into two numpy arrays. The `sensor_msgs/PointCloud2` messages contain `n` points based on the `width` and `height` of the acquired image. For instance, a `480 x 640` image will have `307,200` points. Each point includes three spatial coordinates (`xyz`) and the corresponding color in `RGB` format. These can be considered as two separate channels of information. + +The function returns the `xyz` coordinates and `RGB` values in the format of the original camera resolution (`width x height`). Most sensors have a maximum distance, known as the clip distance, beyond which values are represented as inf (`np.inf`). Before processing, it is important to filter out these null values and assign them a value of `0`. + +```python +import numpy as np +import ros_numpy + + +def pointcloud2_to_array(pointcloud2: PointCloud2) -> tuple: + """ + Convert a ROS PointCloud2 message to a numpy array. + + Args: + pointcloud2 (PointCloud2): the PointCloud2 message + + Returns: + (tuple): tuple containing (xyz, rgb) + """ + pc_array = ros_numpy.point_cloud2.pointcloud2_to_array(pointcloud2) + split = ros_numpy.point_cloud2.split_rgb_field(pc_array) + rgb = np.stack([split["b"], split["g"], split["r"]], axis=2) + xyz = ros_numpy.point_cloud2.get_xyz_points(pc_array, remove_nans=False) + xyz = np.array(xyz).reshape((pointcloud2.height, pointcloud2.width, 3)) + nan_rows = np.isnan(xyz).all(axis=2) + xyz[nan_rows] = [0, 0, 0] + rgb[nan_rows] = [0, 0, 0] + return xyz, rgb +``` + +Next, subscribe to the `/camera/depth/points` topic to receive the point cloud message and convert the `sensor_msgs/PointCloud2` message into numpy arrays containing the XYZ coordinates and RGB values (using the `pointcloud2_to_array` function). Process the RGB image using the YOLO model to extract segmented objects. For each detected object, extract the segmentation mask and apply it to both the RGB image and the XYZ coordinates to isolate the object in 3D space. + +Processing the mask is straightforward since it consists of binary values, with `1` indicating the presence of the object and `0` indicating the absence. To apply the mask, simply multiply the original channels by the mask. This operation effectively isolates the object of interest within the image. Finally, create an Open3D point cloud object and visualize the segmented object in 3D space with associated colors. + +```python +import sys + +import open3d as o3d + +ros_cloud = rospy.wait_for_message("/camera/depth/points", PointCloud2) +xyz, rgb = pointcloud2_to_array(ros_cloud) +result = segmentation_model(rgb) + +if not len(result[0].boxes.cls): + print("No objects detected") + sys.exit() + +classes = result[0].boxes.cls.cpu().numpy().astype(int) +for index, class_id in enumerate(classes): + mask = result[0].masks.data.cpu().numpy()[index, :, :].astype(int) + mask_expanded = np.stack([mask, mask, mask], axis=2) + + obj_rgb = rgb * mask_expanded + obj_xyz = xyz * mask_expanded + + pcd = o3d.geometry.PointCloud() + pcd.points = o3d.utility.Vector3dVector(obj_xyz.reshape((ros_cloud.height * ros_cloud.width, 3))) + pcd.colors = o3d.utility.Vector3dVector(obj_rgb.reshape((ros_cloud.height * ros_cloud.width, 3)) / 255) + o3d.visualization.draw_geometries([pcd]) +``` + +??? Example "Complete code" + + ```python + import sys + import time + + import numpy as np + import open3d as o3d + import ros_numpy + import rospy + + from ultralytics import YOLO + + rospy.init_node("ultralytics") + time.sleep(1) + segmentation_model = YOLO("yolov8m-seg.pt") + + + def pointcloud2_to_array(pointcloud2: PointCloud2) -> tuple: + """ + Convert a ROS PointCloud2 message to a numpy array. + + Args: + pointcloud2 (PointCloud2): the PointCloud2 message + + Returns: + (tuple): tuple containing (xyz, rgb) + """ + pc_array = ros_numpy.point_cloud2.pointcloud2_to_array(pointcloud2) + split = ros_numpy.point_cloud2.split_rgb_field(pc_array) + rgb = np.stack([split["b"], split["g"], split["r"]], axis=2) + xyz = ros_numpy.point_cloud2.get_xyz_points(pc_array, remove_nans=False) + xyz = np.array(xyz).reshape((pointcloud2.height, pointcloud2.width, 3)) + nan_rows = np.isnan(xyz).all(axis=2) + xyz[nan_rows] = [0, 0, 0] + rgb[nan_rows] = [0, 0, 0] + return xyz, rgb + + + ros_cloud = rospy.wait_for_message("/camera/depth/points", PointCloud2) + xyz, rgb = pointcloud2_to_array(ros_cloud) + result = segmentation_model(rgb) + + if not len(result[0].boxes.cls): + print("No objects detected") + sys.exit() + + classes = result[0].boxes.cls.cpu().numpy().astype(int) + for index, class_id in enumerate(classes): + mask = result[0].masks.data.cpu().numpy()[index, :, :].astype(int) + mask_expanded = np.stack([mask, mask, mask], axis=2) + + obj_rgb = rgb * mask_expanded + obj_xyz = xyz * mask_expanded + + pcd = o3d.geometry.PointCloud() + pcd.points = o3d.utility.Vector3dVector(obj_xyz.reshape((ros_cloud.height * ros_cloud.width, 3))) + pcd.colors = o3d.utility.Vector3dVector(obj_rgb.reshape((ros_cloud.height * ros_cloud.width, 3)) / 255) + o3d.visualization.draw_geometries([pcd]) + ``` + ++ +
+ +## FAQ + +### What is the Robot Operating System (ROS)? + +The [Robot Operating System (ROS)](https://www.ros.org/) is an open-source framework commonly used in robotics to help developers create robust robot applications. It provides a collection of [libraries and tools](https://www.ros.org/blog/ecosystem/) for building and interfacing with robotic systems, enabling easier development of complex applications. ROS supports communication between nodes using messages over [topics](https://wiki.ros.org/ROS/Tutorials/UnderstandingTopics) or [services](https://wiki.ros.org/ROS/Tutorials/UnderstandingServicesParams). + +### How do I integrate Ultralytics YOLO with ROS for real-time object detection? + +Integrating Ultralytics YOLO with ROS involves setting up a ROS environment and using YOLO for processing sensor data. Begin by installing the required dependencies like `ros_numpy` and Ultralytics YOLO: + +```bash +pip install ros_numpy ultralytics +``` + +Next, create a ROS node and subscribe to an [image topic](../tasks/detect.md) to process the incoming data. Here is a minimal example: + +```python +import ros_numpy +import rospy +from sensor_msgs.msg import Image + +from ultralytics import YOLO + +detection_model = YOLO("yolov8m.pt") +rospy.init_node("ultralytics") +det_image_pub = rospy.Publisher("/ultralytics/detection/image", Image, queue_size=5) + + +def callback(data): + array = ros_numpy.numpify(data) + det_result = detection_model(array) + det_annotated = det_result[0].plot(show=False) + det_image_pub.publish(ros_numpy.msgify(Image, det_annotated, encoding="rgb8")) + + +rospy.Subscriber("/camera/color/image_raw", Image, callback) +rospy.spin() +``` + +### What are ROS topics and how are they used in Ultralytics YOLO? + +ROS topics facilitate communication between nodes in a ROS network by using a publish-subscribe model. A topic is a named channel that nodes use to send and receive messages asynchronously. In the context of Ultralytics YOLO, you can make a node subscribe to an image topic, process the images using YOLO for tasks like detection or segmentation, and publish outcomes to new topics. + +For example, subscribe to a camera topic and process the incoming image for detection: + +```python +rospy.Subscriber("/camera/color/image_raw", Image, callback) +``` + +### Why use depth images with Ultralytics YOLO in ROS? + +Depth images in ROS, represented by `sensor_msgs/Image`, provide the distance of objects from the camera, crucial for tasks like obstacle avoidance, 3D mapping, and localization. By [using depth information](https://en.wikipedia.org/wiki/Depth_map) along with RGB images, robots can better understand their 3D environment. + +With YOLO, you can extract segmentation masks from RGB images and apply these masks to depth images to obtain precise 3D object information, improving the robot's ability to navigate and interact with its surroundings. + +### How can I visualize 3D point clouds with YOLO in ROS? + +To visualize 3D point clouds in ROS with YOLO: + +1. Convert `sensor_msgs/PointCloud2` messages to numpy arrays. +2. Use YOLO to segment RGB images. +3. Apply the segmentation mask to the point cloud. + +Here's an example using Open3D for visualization: + +```python +import sys + +import open3d as o3d +import ros_numpy +import rospy +from sensor_msgs.msg import PointCloud2 + +from ultralytics import YOLO + +rospy.init_node("ultralytics") +segmentation_model = YOLO("yolov8m-seg.pt") + + +def pointcloud2_to_array(pointcloud2): + pc_array = ros_numpy.point_cloud2.pointcloud2_to_array(pointcloud2) + split = ros_numpy.point_cloud2.split_rgb_field(pc_array) + rgb = np.stack([split["b"], split["g"], split["r"]], axis=2) + xyz = ros_numpy.point_cloud2.get_xyz_points(pc_array, remove_nans=False) + xyz = np.array(xyz).reshape((pointcloud2.height, pointcloud2.width, 3)) + return xyz, rgb + + +ros_cloud = rospy.wait_for_message("/camera/depth/points", PointCloud2) +xyz, rgb = pointcloud2_to_array(ros_cloud) +result = segmentation_model(rgb) + +if not len(result[0].boxes.cls): + print("No objects detected") + sys.exit() + +classes = result[0].boxes.cls.cpu().numpy().astype(int) +for index, class_id in enumerate(classes): + mask = result[0].masks.data.cpu().numpy()[index, :, :].astype(int) + mask_expanded = np.stack([mask, mask, mask], axis=2) + + obj_rgb = rgb * mask_expanded + obj_xyz = xyz * mask_expanded + + pcd = o3d.geometry.PointCloud() + pcd.points = o3d.utility.Vector3dVector(obj_xyz.reshape((-1, 3))) + pcd.colors = o3d.utility.Vector3dVector(obj_rgb.reshape((-1, 3)) / 255) + o3d.visualization.draw_geometries([pcd]) +``` + +This approach provides a 3D visualization of segmented objects, useful for tasks like navigation and manipulation. diff --git a/ultralytics/docs/en/guides/sahi-tiled-inference.md b/ultralytics/docs/en/guides/sahi-tiled-inference.md new file mode 100644 index 0000000000000000000000000000000000000000..2bbddb3dc8fd4b94c072a5775f668b56f9c82ec1 --- /dev/null +++ b/ultralytics/docs/en/guides/sahi-tiled-inference.md @@ -0,0 +1,295 @@ +--- +comments: true +description: Learn how to implement YOLOv8 with SAHI for sliced inference. Optimize memory usage and enhance detection accuracy for large-scale applications. +keywords: YOLOv8, SAHI, Sliced Inference, Object Detection, Ultralytics, High-resolution Images, Computational Efficiency, Integration Guide +--- + +# Ultralytics Docs: Using YOLOv8 with SAHI for Sliced Inference + +Welcome to the Ultralytics documentation on how to use YOLOv8 with [SAHI](https://github.com/obss/sahi) (Slicing Aided Hyper Inference). This comprehensive guide aims to furnish you with all the essential knowledge you'll need to implement SAHI alongside YOLOv8. We'll deep-dive into what SAHI is, why sliced inference is critical for large-scale applications, and how to integrate these functionalities with YOLOv8 for enhanced object detection performance. + ++ +
+ +## Introduction to SAHI + +SAHI (Slicing Aided Hyper Inference) is an innovative library designed to optimize object detection algorithms for large-scale and high-resolution imagery. Its core functionality lies in partitioning images into manageable slices, running object detection on each slice, and then stitching the results back together. SAHI is compatible with a range of object detection models, including the YOLO series, thereby offering flexibility while ensuring optimized use of computational resources. + +
+
+
+
+ Watch: Inference with SAHI (Slicing Aided Hyper Inference) using Ultralytics YOLOv8
+
YOLOv8 without SAHI | +YOLOv8 with SAHI | +
---|---|
+ | + |
+
+
+
+ Watch: Security Alarm System Project with Ultralytics YOLOv8 Object Detection
+
+
+
+
+ Watch: Speed Estimation using Ultralytics YOLOv8
+
+
+
+
+ Watch: How to Do Computer Vision Projects | A Step-by-Step Guide
+
+ +
+ +Working on your own computer vision projects is a great way to understand and learn more about computer vision. However, a computer vision project can consist of many steps, and it might seem confusing at first. By the end of this guide, you'll be familiar with the steps involved in a computer vision project. We'll walk through everything from the beginning to the end of a project, explaining why each part is important. Let's get started and make your computer vision project a success! + +## An Overview of a Computer Vision Project + +Before discussing the details of each step involved in a computer vision project, let's look at the overall process. If you started a computer vision project today, you'd take the following steps: + +- Your first priority would be to understand your project's requirements. +- Then, you'd collect and accurately label the images that will help train your model. +- Next, you'd clean your data and apply augmentation techniques to prepare it for model training. +- After model training, you'd thoroughly test and evaluate your model to make sure it performs consistently under different conditions. +- Finally, you'd deploy your model into the real world and update it based on new insights and feedback. + ++ +
+ +Now that we know what to expect, let's dive right into the steps and get your project moving forward. + +## Step 1: Defining Your Project's Goals + +The first step in any computer vision project is clearly defining the problem you're trying to solve. Knowing the end goal helps you start to build a solution. This is especially true when it comes to computer vision because your project's objective will directly affect which computer vision task you need to focus on. + +Here are some examples of project objectives and the computer vision tasks that can be used to reach these objectives: + +- **Objective:** To develop a system that can monitor and manage the flow of different vehicle types on highways, improving traffic management and safety. + + - **Computer Vision Task:** Object detection is ideal for traffic monitoring because it efficiently locates and identifies multiple vehicles. It is less computationally demanding than image segmentation, which provides unnecessary detail for this task, ensuring faster, real-time analysis. + +- **Objective:** To develop a tool that assists radiologists by providing precise, pixel-level outlines of tumors in medical imaging scans. + + - **Computer Vision Task:** Image segmentation is suitable for medical imaging because it provides accurate and detailed boundaries of tumors that are crucial for assessing size, shape, and treatment planning. + +- **Objective:** To create a digital system that categorizes various documents (e.g., invoices, receipts, legal paperwork) to improve organizational efficiency and document retrieval. + - **Computer Vision Task:** Image classification is ideal here as it handles one document at a time, without needing to consider the document's position in the image. This approach simplifies and accelerates the sorting process. + +### Step 1.5: Selecting the Right Model and Training Approach + +After understanding the project objective and suitable computer vision tasks, an essential part of defining the project goal is [selecting the right model](../models/index.md) and training approach. + +Depending on the objective, you might choose to select the model first or after seeing what data you are able to collect in Step 2. For example, suppose your project is highly dependent on the availability of specific types of data. In that case, it may be more practical to gather and analyze the data first before selecting a model. On the other hand, if you have a clear understanding of the model requirements, you can choose the model first and then collect data that fits those specifications. + +Choosing between training from scratch or using transfer learning affects how you prepare your data. Training from scratch requires a diverse dataset to build the model's understanding from the ground up. Transfer learning, on the other hand, allows you to use a pre-trained model and adapt it with a smaller, more specific dataset. Also, choosing a specific model to train will determine how you need to prepare your data, such as resizing images or adding annotations, according to the model's specific requirements. + ++ +
+ +Note: When choosing a model, consider its [deployment](./model-deployment-options.md) to ensure compatibility and performance. For example, lightweight models are ideal for edge computing due to their efficiency on resource-constrained devices. To learn more about the key points related to defining your project, read [our guide](./defining-project-goals.md) on defining your project's goals and selecting the right model. + +Before getting into the hands-on work of a computer vision project, it's important to have a clear understanding of these details. Double-check that you've considered the following before moving on to Step 2: + +- Clearly define the problem you're trying to solve. +- Determine the end goal of your project. +- Identify the specific computer vision task needed (e.g., object detection, image classification, image segmentation). +- Decide whether to train a model from scratch or use transfer learning. +- Select the appropriate model for your task and deployment needs. + +## Step 2: Data Collection and Data Annotation + +The quality of your computer vision models depend on the quality of your dataset. You can either collect images from the internet, take your own pictures, or use pre-existing datasets. Here are some great resources for downloading high-quality datasets: [Google Dataset Search Engine](https://datasetsearch.research.google.com/), [UC Irvine Machine Learning Repository](https://archive.ics.uci.edu/), and [Kaggle Datasets](https://www.kaggle.com/datasets). + +Some libraries, like Ultralytics, provide [built-in support for various datasets](../datasets/index.md), making it easier to get started with high-quality data. These libraries often include utilities for using popular datasets seamlessly, which can save you a lot of time and effort in the initial stages of your project. + +However, if you choose to collect images or take your own pictures, you'll need to annotate your data. Data annotation is the process of labeling your data to impart knowledge to your model. The type of data annotation you'll work with depends on your specific computer vision technique. Here are some examples: + +- **Image Classification:** You'll label the entire image as a single class. +- **Object Detection:** You'll draw bounding boxes around each object in the image and label each box. +- **Image Segmentation:** You'll label each pixel in the image according to the object it belongs to, creating detailed object boundaries. + ++ +
+ +[Data collection and annotation](./data-collection-and-annotation.md) can be a time-consuming manual effort. Annotation tools can help make this process easier. Here are some useful open annotation tools: [LabeI Studio](https://github.com/HumanSignal/label-studio), [CVAT](https://github.com/cvat-ai/cvat), and [Labelme](https://github.com/labelmeai/labelme). + +## Step 3: Data Augmentation and Splitting Your Dataset + +After collecting and annotating your image data, it's important to first split your dataset into training, validation, and test sets before performing data augmentation. Splitting your dataset before augmentation is crucial to test and validate your model on original, unaltered data. It helps accurately assess how well the model generalizes to new, unseen data. + +Here's how to split your data: + +- **Training Set:** It is the largest portion of your data, typically 70-80% of the total, used to train your model. +- **Validation Set:** Usually around 10-15% of your data; this set is used to tune hyperparameters and validate the model during training, helping to prevent overfitting. +- **Test Set:** The remaining 10-15% of your data is set aside as the test set. It is used to evaluate the model's performance on unseen data after training is complete. + +After splitting your data, you can perform data augmentation by applying transformations like rotating, scaling, and flipping images to artificially increase the size of your dataset. Data augmentation makes your model more robust to variations and improves its performance on unseen images. + ++ +
+ +Libraries like OpenCV, Albumentations, and TensorFlow offer flexible augmentation functions that you can use. Additionally, some libraries, such as Ultralytics, have [built-in augmentation settings](../modes/train.md) directly within its model training function, simplifying the process. + +To understand your data better, you can use tools like [Matplotlib](https://matplotlib.org/) or [Seaborn](https://seaborn.pydata.org/) to visualize the images and analyze their distribution and characteristics. Visualizing your data helps identify patterns, anomalies, and the effectiveness of your augmentation techniques. You can also use [Ultralytics Explorer](../datasets/explorer/index.md), a tool for exploring computer vision datasets with semantic search, SQL queries, and vector similarity search. + ++ +
+ +By properly [understanding, splitting, and augmenting your data](./preprocessing_annotated_data.md), you can develop a well-trained, validated, and tested model that performs well in real-world applications. + +## Step 4: Model Training + +Once your dataset is ready for training, you can focus on setting up the necessary environment, managing your datasets, and training your model. + +First, you'll need to make sure your environment is configured correctly. Typically, this includes the following: + +- Installing essential libraries and frameworks like TensorFlow, PyTorch, or [Ultralytics](../quickstart.md). +- If you are using a GPU, installing libraries like CUDA and cuDNN will help enable GPU acceleration and speed up the training process. + +Then, you can load your training and validation datasets into your environment. Normalize and preprocess the data through resizing, format conversion, or augmentation. With your model selected, configure the layers and specify hyperparameters. Compile the model by setting the loss function, optimizer, and performance metrics. + +Libraries like Ultralytics simplify the training process. You can [start training](../modes/train.md) by feeding data into the model with minimal code. These libraries handle weight adjustments, backpropagation, and validation automatically. They also offer tools to monitor progress and adjust hyperparameters easily. After training, save the model and its weights with a few commands. + +It's important to keep in mind that proper dataset management is vital for efficient training. Use version control for datasets to track changes and ensure reproducibility. Tools like [DVC (Data Version Control)](../integrations/dvc.md) can help manage large datasets. + +## Step 5: Model Evaluation and Model Finetuning + +It's important to assess your model's performance using various metrics and refine it to improve accuracy. [Evaluating](../modes/val.md) helps identify areas where the model excels and where it may need improvement. Fine-tuning ensures the model is optimized for the best possible performance. + +- **[Performance Metrics](./yolo-performance-metrics.md):** Use metrics like accuracy, precision, recall, and F1-score to evaluate your model's performance. These metrics provide insights into how well your model is making predictions. +- **[Hyperparameter Tuning](./hyperparameter-tuning.md):** Adjust hyperparameters to optimize model performance. Techniques like grid search or random search can help find the best hyperparameter values. + +- Fine-Tuning: Make small adjustments to the model architecture or training process to enhance performance. This might involve tweaking learning rates, batch sizes, or other model parameters. + +## Step 6: Model Testing + +In this step, you can make sure that your model performs well on completely unseen data, confirming its readiness for deployment. The difference between model testing and model evaluation is that it focuses on verifying the final model's performance rather than iteratively improving it. + +It's important to thoroughly test and debug any common issues that may arise. Test your model on a separate test dataset that was not used during training or validation. This dataset should represent real-world scenarios to ensure the model's performance is consistent and reliable. + +Also, address common problems such as overfitting, underfitting, and data leakage. Use techniques like cross-validation and anomaly detection to identify and fix these issues. + +## Step 7: Model Deployment + +Once your model has been thoroughly tested, it's time to deploy it. Deployment involves making your model available for use in a production environment. Here are the steps to deploy a computer vision model: + +- Setting Up the Environment: Configure the necessary infrastructure for your chosen deployment option, whether it's cloud-based (AWS, Google Cloud, Azure) or edge-based (local devices, IoT). + +- **[Exporting the Model](../modes/export.md):** Export your model to the appropriate format (e.g., ONNX, TensorRT, CoreML for YOLOv8) to ensure compatibility with your deployment platform. +- **Deploying the Model:** Deploy the model by setting up APIs or endpoints and integrating it with your application. +- **Ensuring Scalability**: Implement load balancers, auto-scaling groups, and monitoring tools to manage resources and handle increasing data and user requests. + +## Step 8: Monitoring, Maintenance, and Documentation + +Once your model is deployed, it's important to continuously monitor its performance, maintain it to handle any issues, and document the entire process for future reference and improvements. + +Monitoring tools can help you track key performance indicators (KPIs) and detect anomalies or drops in accuracy. By monitoring the model, you can be aware of model drift, where the model's performance declines over time due to changes in the input data. Periodically retrain the model with updated data to maintain accuracy and relevance. + ++ +
+ +In addition to monitoring and maintenance, documentation is also key. Thoroughly document the entire process, including model architecture, training procedures, hyperparameters, data preprocessing steps, and any changes made during deployment and maintenance. Good documentation ensures reproducibility and makes future updates or troubleshooting easier. By effectively monitoring, maintaining, and documenting your model, you can ensure it remains accurate, reliable, and easy to manage over its lifecycle. + +## Engaging with the Community + +Connecting with a community of computer vision enthusiasts can help you tackle any issues you face while working on your computer vision project with confidence. Here are some ways to learn, troubleshoot, and network effectively. + +### Community Resources + +- **GitHub Issues:** Check out the [YOLOv8 GitHub repository](https://github.com/ultralytics/ultralytics/issues) and use the Issues tab to ask questions, report bugs, and suggest new features. The active community and maintainers are there to help with specific issues. +- **Ultralytics Discord Server:** Join the [Ultralytics Discord server](https://ultralytics.com/discord/) to interact with other users and developers, get support, and share insights. + +### Official Documentation + +- **Ultralytics YOLOv8 Documentation:** Explore the [official YOLOv8 documentation](./index.md) for detailed guides with helpful tips on different computer vision tasks and projects. + +Using these resources will help you overcome challenges and stay updated with the latest trends and best practices in the computer vision community. + +## Kickstart Your Computer Vision Project Today! + +Taking on a computer vision project can be exciting and rewarding. By following the steps in this guide, you can build a solid foundation for success. Each step is crucial for developing a solution that meets your objectives and works well in real-world scenarios. As you gain experience, you'll discover advanced techniques and tools to improve your projects. Stay curious, keep learning, and explore new methods and innovations! + +## FAQ + +### How do I choose the right computer vision task for my project? + +Choosing the right computer vision task depends on your project's end goal. For instance, if you want to monitor traffic, **object detection** is suitable as it can locate and identify multiple vehicle types in real-time. For medical imaging, **image segmentation** is ideal for providing detailed boundaries of tumors, aiding in diagnosis and treatment planning. Learn more about specific tasks like [object detection](../tasks/detect.md), [image classification](../tasks/classify.md), and [instance segmentation](../tasks/segment.md). + +### Why is data annotation crucial in computer vision projects? + +Data annotation is vital for teaching your model to recognize patterns. The type of annotation varies with the task: + +- **Image Classification**: Entire image labeled as a single class. +- **Object Detection**: Bounding boxes drawn around objects. +- **Image Segmentation**: Each pixel labeled according to the object it belongs to. + +Tools like [Label Studio](https://github.com/HumanSignal/label-studio), [CVAT](https://github.com/cvat-ai/cvat), and [Labelme](https://github.com/labelmeai/labelme) can assist in this process. For more details, refer to our [data collection and annotation guide](./data-collection-and-annotation.md). + +### What steps should I follow to augment and split my dataset effectively? + +Splitting your dataset before augmentation helps validate model performance on original, unaltered data. Follow these steps: + +- **Training Set**: 70-80% of your data. +- **Validation Set**: 10-15% for hyperparameter tuning. +- **Test Set**: Remaining 10-15% for final evaluation. + +After splitting, apply data augmentation techniques like rotation, scaling, and flipping to increase dataset diversity. Libraries such as Albumentations and OpenCV can help. Ultralytics also offers [built-in augmentation settings](../modes/train.md) for convenience. + +### How can I export my trained computer vision model for deployment? + +Exporting your model ensures compatibility with different deployment platforms. Ultralytics provides multiple formats, including ONNX, TensorRT, and CoreML. To export your YOLOv8 model, follow this guide: + +- Use the `export` function with the desired format parameter. +- Ensure the exported model fits the specifications of your deployment environment (e.g., edge devices, cloud). + +For more information, check out the [model export guide](../modes/export.md). + +### What are the best practices for monitoring and maintaining a deployed computer vision model? + +Continuous monitoring and maintenance are essential for a model's long-term success. Implement tools for tracking Key Performance Indicators (KPIs) and detecting anomalies. Regularly retrain the model with updated data to counteract model drift. Document the entire process, including model architecture, hyperparameters, and changes, to ensure reproducibility and ease of future updates. Learn more in our [monitoring and maintenance guide](#step-8-monitoring-maintenance-and-documentation). diff --git a/ultralytics/docs/en/guides/streamlit-live-inference.md b/ultralytics/docs/en/guides/streamlit-live-inference.md new file mode 100644 index 0000000000000000000000000000000000000000..c9c7102da00f12bcdffab2469fbf74881df9bd7e --- /dev/null +++ b/ultralytics/docs/en/guides/streamlit-live-inference.md @@ -0,0 +1,154 @@ +--- +comments: true +description: Learn how to set up a real-time object detection application using Streamlit and Ultralytics YOLOv8. Follow this step-by-step guide to implement webcam-based object detection. +keywords: Streamlit, YOLOv8, Real-time Object Detection, Streamlit Application, YOLOv8 Streamlit Tutorial, Webcam Object Detection +--- + +# Live Inference with Streamlit Application using Ultralytics YOLOv8 + +## Introduction + +Streamlit makes it simple to build and deploy interactive web applications. Combining this with Ultralytics YOLOv8 allows for real-time object detection and analysis directly in your browser. YOLOv8 high accuracy and speed ensure seamless performance for live video streams, making it ideal for applications in security, retail, and beyond. + +| Aquaculture | Animals husbandry | +| :---------------------------------------------------------------------------------------------------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------: | +| ![Fish Detection using Ultralytics YOLOv8](https://github.com/RizwanMunawar/RizwanMunawar/assets/62513924/ea6d7ece-cded-4db7-b810-1f8433df2c96) | ![Animals Detection using Ultralytics YOLOv8](https://github.com/RizwanMunawar/RizwanMunawar/assets/62513924/2e1f4781-60ab-4e72-b3e4-726c10cd223c) | +| Fish Detection using Ultralytics YOLOv8 | Animals Detection using Ultralytics YOLOv8 | + +## Advantages of Live Inference + +- **Seamless Real-Time Object Detection**: Streamlit combined with YOLOv8 enables real-time object detection directly from your webcam feed. This allows for immediate analysis and insights, making it ideal for applications requiring instant feedback. +- **User-Friendly Deployment**: Streamlit's interactive interface makes it easy to deploy and use the application without extensive technical knowledge. Users can start live inference with a simple click, enhancing accessibility and usability. +- **Efficient Resource Utilization**: YOLOv8 optimized algorithm ensure high-speed processing with minimal computational resources. This efficiency allows for smooth and reliable webcam inference even on standard hardware, making advanced computer vision accessible to a wider audience. + +## Streamlit Application Code + +!!! tip "Ultralytics Installation" + + Before you start building the application, ensure you have the Ultralytics Python Package installed. You can install it using the command **pip install ultralytics** + +!!! Example "Streamlit Application" + + === "Python" + + ```python + from ultralytics import solutions + + solutions.inference() + + ### Make sure to run the file using command `streamlit run
+
+
+
+ Watch: Getting Started with NVIDIA Triton Inference Server.
+
+ +
+ +Image from the [libsixel](https://saitoha.github.io/libsixel/) website. + +## Motivation + +When connecting to a remote machine, normally visualizing image results is not possible or requires moving data to a local device with a GUI. The VSCode integrated terminal allows for directly rendering images. This is a short demonstration on how to use this in conjunction with `ultralytics` with [prediction results](../modes/predict.md). + +!!! warning + + Only compatible with Linux and MacOS. Check the [VSCode repository](https://github.com/microsoft/vscode), check [Issue status](https://github.com/microsoft/vscode/issues/198622), or [documentation](https://code.visualstudio.com/docs) for updates about Windows support to view images in terminal with `sixel`. + +The VSCode compatible protocols for viewing images using the integrated terminal are [`sixel`](https://en.wikipedia.org/wiki/Sixel) and [`iTerm`](https://iterm2.com/documentation-images.html). This guide will demonstrate use of the `sixel` protocol. + +## Process + +1. First, you must enable settings `terminal.integrated.enableImages` and `terminal.integrated.gpuAcceleration` in VSCode. + + ```yaml + "terminal.integrated.gpuAcceleration": "auto" # "auto" is default, can also use "on" + "terminal.integrated.enableImages": false + ``` + ++ +
+ +2. Install the `python-sixel` library in your virtual environment. This is a [fork](https://github.com/lubosz/python-sixel?tab=readme-ov-file) of the `PySixel` library, which is no longer maintained. + + ```bash + pip install sixel + ``` + +3. Load a model and execute inference, then plot the results and store in a variable. See more about inference arguments and working with results on the [predict mode](../modes/predict.md) page. + + ```{ .py .annotate } + from ultralytics import YOLO + + # Load a model + model = YOLO("yolov8n.pt") + + # Run inference on an image + results = model.predict(source="ultralytics/assets/bus.jpg") + + # Plot inference results + plot = results[0].plot() # (1)! + ``` + + 1. See [plot method parameters](../modes/predict.md#plot-method-parameters) to see possible arguments to use. + +4. Now, use OpenCV to convert the `numpy.ndarray` to `bytes` data. Then use `io.BytesIO` to make a "file-like" object. + + ```{ .py .annotate } + import io + + import cv2 + + # Results image as bytes + im_bytes = cv2.imencode( + ".png", # (1)! + plot, + )[1].tobytes() # (2)! + + # Image bytes as a file-like object + mem_file = io.BytesIO(im_bytes) + ``` + + 1. It's possible to use other image extensions as well. + 2. Only the object at index `1` that is returned is needed. + +5. Create a `SixelWriter` instance, and then use the `.draw()` method to draw the image in the terminal. + + ```python + from sixel import SixelWriter + + # Create sixel writer object + w = SixelWriter() + + # Draw the sixel image in the terminal + w.draw(mem_file) + ``` + +## Example Inference Results + ++ +
+ +!!! danger + + Using this example with videos or animated GIF frames has **not** been tested. Attempt at your own risk. + +## Full Code Example + +```{ .py .annotate } +import io + +import cv2 +from sixel import SixelWriter + +from ultralytics import YOLO + +# Load a model +model = YOLO("yolov8n.pt") + +# Run inference on an image +results = model.predict(source="ultralytics/assets/bus.jpg") + +# Plot inference results +plot = results[0].plot() # (3)! + +# Results image as bytes +im_bytes = cv2.imencode( + ".png", # (1)! + plot, +)[1].tobytes() # (2)! + +mem_file = io.BytesIO(im_bytes) +w = SixelWriter() +w.draw(mem_file) +``` + +1. It's possible to use other image extensions as well. +2. Only the object at index `1` that is returned is needed. +3. See [plot method parameters](../modes/predict.md#plot-method-parameters) to see possible arguments to use. + +--- + +!!! tip + + You may need to use `clear` to "erase" the view of the image in the terminal. + +## FAQ + +### How can I view YOLO inference results in a VSCode terminal on macOS or Linux? + +To view YOLO inference results in a VSCode terminal on macOS or Linux, follow these steps: + +1. Enable the necessary VSCode settings: + + ```yaml + "terminal.integrated.enableImages": true + "terminal.integrated.gpuAcceleration": "auto" + ``` + +2. Install the sixel library: + + ```bash + pip install sixel + ``` + +3. Load your YOLO model and run inference: + + ```python + from ultralytics import YOLO + + model = YOLO("yolov8n.pt") + results = model.predict(source="path_to_image") + plot = results[0].plot() + ``` + +4. Convert the inference result image to bytes and display it in the terminal: + + ```python + import io + + import cv2 + from sixel import SixelWriter + + im_bytes = cv2.imencode(".png", plot)[1].tobytes() + mem_file = io.BytesIO(im_bytes) + SixelWriter().draw(mem_file) + ``` + +For further details, visit the [predict mode](../modes/predict.md) page. + +### Why does the sixel protocol only work on Linux and macOS? + +The sixel protocol is currently only supported on Linux and macOS because these platforms have native terminal capabilities compatible with sixel graphics. Windows support for terminal graphics using sixel is still under development. For updates on Windows compatibility, check the [VSCode Issue status](https://github.com/microsoft/vscode/issues/198622) and [documentation](https://code.visualstudio.com/docs). + +### What if I encounter issues with displaying images in the VSCode terminal? + +If you encounter issues displaying images in the VSCode terminal using sixel: + +1. Ensure the necessary settings in VSCode are enabled: + + ```yaml + "terminal.integrated.enableImages": true + "terminal.integrated.gpuAcceleration": "auto" + ``` + +2. Verify the sixel library installation: + + ```bash + pip install sixel + ``` + +3. Check your image data conversion and plotting code for errors. For example: + + ```python + import io + + import cv2 + from sixel import SixelWriter + + im_bytes = cv2.imencode(".png", plot)[1].tobytes() + mem_file = io.BytesIO(im_bytes) + SixelWriter().draw(mem_file) + ``` + +If problems persist, consult the [VSCode repository](https://github.com/microsoft/vscode), and visit the [plot method parameters](../modes/predict.md#plot-method-parameters) section for additional guidance. + +### Can YOLO display video inference results in the terminal using sixel? + +Displaying video inference results or animated GIF frames using sixel in the terminal is currently untested and may not be supported. We recommend starting with static images and verifying compatibility. Attempt video results at your own risk, keeping in mind performance constraints. For more information on plotting inference results, visit the [predict mode](../modes/predict.md) page. + +### How can I troubleshoot issues with the `python-sixel` library? + +To troubleshoot issues with the `python-sixel` library: + +1. Ensure the library is correctly installed in your virtual environment: + + ```bash + pip install sixel + ``` + +2. Verify that you have the necessary Python and system dependencies. + +3. Refer to the [python-sixel GitHub repository](https://github.com/lubosz/python-sixel) for additional documentation and community support. + +4. Double-check your code for potential errors, specifically the usage of `SixelWriter` and image data conversion steps. + +For further assistance on working with YOLO models and sixel integration, see the [export](../modes/export.md) and [predict mode](../modes/predict.md) documentation pages. diff --git a/ultralytics/docs/en/guides/vision-eye.md b/ultralytics/docs/en/guides/vision-eye.md new file mode 100644 index 0000000000000000000000000000000000000000..8dbd9a9437de21c4c4b8548bdc073fa092dc0f46 --- /dev/null +++ b/ultralytics/docs/en/guides/vision-eye.md @@ -0,0 +1,308 @@ +--- +comments: true +description: Discover VisionEye's object mapping and tracking powered by Ultralytics YOLOv8. Simulate human eye precision, track objects, and calculate distances effortlessly. +keywords: VisionEye, YOLOv8, Ultralytics, object mapping, object tracking, distance calculation, computer vision, AI, machine learning, Python, tutorial +--- + +# VisionEye View Object Mapping using Ultralytics YOLOv8 🚀 + +## What is VisionEye Object Mapping? + +[Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics/) VisionEye offers the capability for computers to identify and pinpoint objects, simulating the observational precision of the human eye. This functionality enables computers to discern and focus on specific objects, much like the way the human eye observes details from a particular viewpoint. + +## Samples + +| VisionEye View | VisionEye View With Object Tracking | VisionEye View With Distance Calculation | +| :----------------------------------------------------------------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------: | +| ![VisionEye View Object Mapping using Ultralytics YOLOv8](https://github.com/RizwanMunawar/ultralytics/assets/62513924/7d593acc-2e37-41b0-ad0e-92b4ffae6647) | ![VisionEye View Object Mapping with Object Tracking using Ultralytics YOLOv8](https://github.com/RizwanMunawar/ultralytics/assets/62513924/fcd85952-390f-451e-8fb0-b82e943af89c) | ![VisionEye View with Distance Calculation using Ultralytics YOLOv8](https://github.com/RizwanMunawar/RizwanMunawar/assets/62513924/18c4dafe-a22e-4fa9-a7d4-2bb293562a95) | +| VisionEye View Object Mapping using Ultralytics YOLOv8 | VisionEye View Object Mapping with Object Tracking using Ultralytics YOLOv8 | VisionEye View with Distance Calculation using Ultralytics YOLOv8 | + +!!! Example "VisionEye Object Mapping using YOLOv8" + + === "VisionEye Object Mapping" + + ```python + import cv2 + + from ultralytics import YOLO + from ultralytics.utils.plotting import Annotator, colors + + model = YOLO("yolov8n.pt") + names = model.model.names + cap = cv2.VideoCapture("path/to/video/file.mp4") + w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) + + out = cv2.VideoWriter("visioneye-pinpoint.avi", cv2.VideoWriter_fourcc(*"MJPG"), fps, (w, h)) + + center_point = (-10, h) + + while True: + ret, im0 = cap.read() + if not ret: + print("Video frame is empty or video processing has been successfully completed.") + break + + results = model.predict(im0) + boxes = results[0].boxes.xyxy.cpu() + clss = results[0].boxes.cls.cpu().tolist() + + annotator = Annotator(im0, line_width=2) + + for box, cls in zip(boxes, clss): + annotator.box_label(box, label=names[int(cls)], color=colors(int(cls))) + annotator.visioneye(box, center_point) + + out.write(im0) + cv2.imshow("visioneye-pinpoint", im0) + + if cv2.waitKey(1) & 0xFF == ord("q"): + break + + out.release() + cap.release() + cv2.destroyAllWindows() + ``` + + === "VisionEye Object Mapping with Object Tracking" + + ```python + import cv2 + + from ultralytics import YOLO + from ultralytics.utils.plotting import Annotator, colors + + model = YOLO("yolov8n.pt") + cap = cv2.VideoCapture("path/to/video/file.mp4") + w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) + + out = cv2.VideoWriter("visioneye-pinpoint.avi", cv2.VideoWriter_fourcc(*"MJPG"), fps, (w, h)) + + center_point = (-10, h) + + while True: + ret, im0 = cap.read() + if not ret: + print("Video frame is empty or video processing has been successfully completed.") + break + + annotator = Annotator(im0, line_width=2) + + results = model.track(im0, persist=True) + boxes = results[0].boxes.xyxy.cpu() + + if results[0].boxes.id is not None: + track_ids = results[0].boxes.id.int().cpu().tolist() + + for box, track_id in zip(boxes, track_ids): + annotator.box_label(box, label=str(track_id), color=colors(int(track_id))) + annotator.visioneye(box, center_point) + + out.write(im0) + cv2.imshow("visioneye-pinpoint", im0) + + if cv2.waitKey(1) & 0xFF == ord("q"): + break + + out.release() + cap.release() + cv2.destroyAllWindows() + ``` + + === "VisionEye with Distance Calculation" + + ```python + import math + + import cv2 + + from ultralytics import YOLO + from ultralytics.utils.plotting import Annotator + + model = YOLO("yolov8s.pt") + cap = cv2.VideoCapture("Path/to/video/file.mp4") + + w, h, fps = (int(cap.get(x)) for x in (cv2.CAP_PROP_FRAME_WIDTH, cv2.CAP_PROP_FRAME_HEIGHT, cv2.CAP_PROP_FPS)) + + out = cv2.VideoWriter("visioneye-distance-calculation.avi", cv2.VideoWriter_fourcc(*"MJPG"), fps, (w, h)) + + center_point = (0, h) + pixel_per_meter = 10 + + txt_color, txt_background, bbox_clr = ((0, 0, 0), (255, 255, 255), (255, 0, 255)) + + while True: + ret, im0 = cap.read() + if not ret: + print("Video frame is empty or video processing has been successfully completed.") + break + + annotator = Annotator(im0, line_width=2) + + results = model.track(im0, persist=True) + boxes = results[0].boxes.xyxy.cpu() + + if results[0].boxes.id is not None: + track_ids = results[0].boxes.id.int().cpu().tolist() + + for box, track_id in zip(boxes, track_ids): + annotator.box_label(box, label=str(track_id), color=bbox_clr) + annotator.visioneye(box, center_point) + + x1, y1 = int((box[0] + box[2]) // 2), int((box[1] + box[3]) // 2) # Bounding box centroid + + distance = (math.sqrt((x1 - center_point[0]) ** 2 + (y1 - center_point[1]) ** 2)) / pixel_per_meter + + text_size, _ = cv2.getTextSize(f"Distance: {distance:.2f} m", cv2.FONT_HERSHEY_SIMPLEX, 1.2, 3) + cv2.rectangle(im0, (x1, y1 - text_size[1] - 10), (x1 + text_size[0] + 10, y1), txt_background, -1) + cv2.putText(im0, f"Distance: {distance:.2f} m", (x1, y1 - 5), cv2.FONT_HERSHEY_SIMPLEX, 1.2, txt_color, 3) + + out.write(im0) + cv2.imshow("visioneye-distance-calculation", im0) + + if cv2.waitKey(1) & 0xFF == ord("q"): + break + + out.release() + cap.release() + cv2.destroyAllWindows() + ``` + +### `visioneye` Arguments + +| Name | Type | Default | Description | +| ----------- | ------- | ---------------- | ------------------------------ | +| `color` | `tuple` | `(235, 219, 11)` | Line and object centroid color | +| `pin_color` | `tuple` | `(255, 0, 255)` | VisionEye pinpoint color | + +## Note + +For any inquiries, feel free to post your questions in the [Ultralytics Issue Section](https://github.com/ultralytics/ultralytics/issues/new/choose) or the discussion section mentioned below. + +## FAQ + +### How do I start using VisionEye Object Mapping with Ultralytics YOLOv8? + +To start using VisionEye Object Mapping with Ultralytics YOLOv8, first, you'll need to install the Ultralytics YOLO package via pip. Then, you can use the sample code provided in the documentation to set up object detection with VisionEye. Here's a simple example to get you started: + +```python +import cv2 + +from ultralytics import YOLO + +model = YOLO("yolov8n.pt") +cap = cv2.VideoCapture("path/to/video/file.mp4") + +while True: + ret, frame = cap.read() + if not ret: + break + + results = model.predict(frame) + for result in results: + # Perform custom logic with result + pass + + cv2.imshow("visioneye", frame) + if cv2.waitKey(1) & 0xFF == ord("q"): + break + +cap.release() +cv2.destroyAllWindows() +``` + +### What are the key features of VisionEye's object tracking capability using Ultralytics YOLOv8? + +VisionEye's object tracking with Ultralytics YOLOv8 allows users to follow the movement of objects within a video frame. Key features include: + +1. **Real-Time Object Tracking**: Keeps up with objects as they move. +2. **Object Identification**: Utilizes YOLOv8's powerful detection algorithms. +3. **Distance Calculation**: Calculates distances between objects and specified points. +4. **Annotation and Visualization**: Provides visual markers for tracked objects. + +Here's a brief code snippet demonstrating tracking with VisionEye: + +```python +import cv2 + +from ultralytics import YOLO + +model = YOLO("yolov8n.pt") +cap = cv2.VideoCapture("path/to/video/file.mp4") + +while True: + ret, frame = cap.read() + if not ret: + break + + results = model.track(frame, persist=True) + for result in results: + # Annotate and visualize tracking + pass + + cv2.imshow("visioneye-tracking", frame) + if cv2.waitKey(1) & 0xFF == ord("q"): + break + +cap.release() +cv2.destroyAllWindows() +``` + +For a comprehensive guide, visit the [VisionEye Object Mapping with Object Tracking](#samples). + +### How can I calculate distances with VisionEye's YOLOv8 model? + +Distance calculation with VisionEye and Ultralytics YOLOv8 involves determining the distance of detected objects from a specified point in the frame. It enhances spatial analysis capabilities, useful in applications such as autonomous driving and surveillance. + +Here's a simplified example: + +```python +import math + +import cv2 + +from ultralytics import YOLO + +model = YOLO("yolov8s.pt") +cap = cv2.VideoCapture("path/to/video/file.mp4") +center_point = (0, 480) # Example center point +pixel_per_meter = 10 + +while True: + ret, frame = cap.read() + if not ret: + break + + results = model.track(frame, persist=True) + for result in results: + # Calculate distance logic + distances = [ + (math.sqrt((box[0] - center_point[0]) ** 2 + (box[1] - center_point[1]) ** 2)) / pixel_per_meter + for box in results + ] + + cv2.imshow("visioneye-distance", frame) + if cv2.waitKey(1) & 0xFF == ord("q"): + break + +cap.release() +cv2.destroyAllWindows() +``` + +For detailed instructions, refer to the [VisionEye with Distance Calculation](#samples). + +### Why should I use Ultralytics YOLOv8 for object mapping and tracking? + +Ultralytics YOLOv8 is renowned for its speed, accuracy, and ease of integration, making it a top choice for object mapping and tracking. Key advantages include: + +1. **State-of-the-art Performance**: Delivers high accuracy in real-time object detection. +2. **Flexibility**: Supports various tasks such as detection, tracking, and distance calculation. +3. **Community and Support**: Extensive documentation and active GitHub community for troubleshooting and enhancements. +4. **Ease of Use**: Intuitive API simplifies complex tasks, allowing for rapid deployment and iteration. + +For more information on applications and benefits, check out the [Ultralytics YOLOv8 documentation](https://docs.ultralytics.com/models/yolov8/). + +### How can I integrate VisionEye with other machine learning tools like Comet or ClearML? + +Ultralytics YOLOv8 can integrate seamlessly with various machine learning tools like Comet and ClearML, enhancing experiment tracking, collaboration, and reproducibility. Follow the detailed guides on [how to use YOLOv5 with Comet](https://www.ultralytics.com/blog/how-to-use-yolov5-with-comet) and [integrate YOLOv8 with ClearML](https://docs.ultralytics.com/integrations/clearml/) to get started. + +For further exploration and integration examples, check our [Ultralytics Integrations Guide](https://docs.ultralytics.com/integrations/). diff --git a/ultralytics/docs/en/guides/workouts-monitoring.md b/ultralytics/docs/en/guides/workouts-monitoring.md new file mode 100644 index 0000000000000000000000000000000000000000..c680862f16708a087c6dced61dfc901ae624d4f0 --- /dev/null +++ b/ultralytics/docs/en/guides/workouts-monitoring.md @@ -0,0 +1,261 @@ +--- +comments: true +description: Optimize your fitness routine with real-time workouts monitoring using Ultralytics YOLOv8. Track and improve your exercise form and performance. +keywords: workouts monitoring, Ultralytics YOLOv8, pose estimation, fitness tracking, exercise assessment, real-time feedback, exercise form, performance metrics +--- + +# Workouts Monitoring using Ultralytics YOLOv8 + +Monitoring workouts through pose estimation with [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics/) enhances exercise assessment by accurately tracking key body landmarks and joints in real-time. This technology provides instant feedback on exercise form, tracks workout routines, and measures performance metrics, optimizing training sessions for users and trainers alike. + +
+
+
+
+ Watch: Workouts Monitoring using Ultralytics YOLOv8 | Pushups, Pullups, Ab Workouts
+
+ +
+ +## Introduction + +This guide serves as a comprehensive aid for troubleshooting common issues encountered while working with YOLOv8 on your Ultralytics projects. Navigating through these issues can be a breeze with the right guidance, ensuring your projects remain on track without unnecessary delays. + +
+
+
+
+ Watch: Ultralytics YOLOv8 Common Issues | Installation Errors, Model Training Issues
+
+
+
+
+ Watch: Ultralytics YOLOv8 Performance Metrics | MAP, F1 Score, Precision, IoU & Accuracy
+
+ +
+ +While this sounds like a limitation, threads can still provide concurrency, especially for I/O-bound operations or when using operations that release the GIL, like those performed by YOLO's underlying C libraries. + +## The Danger of Shared Model Instances + +Instantiating a YOLO model outside your threads and sharing this instance across multiple threads can lead to race conditions, where the internal state of the model is inconsistently modified due to concurrent accesses. This is particularly problematic when the model or its components hold state that is not designed to be thread-safe. + +### Non-Thread-Safe Example: Single Model Instance + +When using threads in Python, it's important to recognize patterns that can lead to concurrency issues. Here is what you should avoid: sharing a single YOLO model instance across multiple threads. + +```python +# Unsafe: Sharing a single model instance across threads +from threading import Thread + +from ultralytics import YOLO + +# Instantiate the model outside the thread +shared_model = YOLO("yolov8n.pt") + + +def predict(image_path): + """Predicts objects in an image using a preloaded YOLO model, take path string to image as argument.""" + results = shared_model.predict(image_path) + # Process results + + +# Starting threads that share the same model instance +Thread(target=predict, args=("image1.jpg",)).start() +Thread(target=predict, args=("image2.jpg",)).start() +``` + +In the example above, the `shared_model` is used by multiple threads, which can lead to unpredictable results because `predict` could be executed simultaneously by multiple threads. + +### Non-Thread-Safe Example: Multiple Model Instances + +Similarly, here is an unsafe pattern with multiple YOLO model instances: + +```python +# Unsafe: Sharing multiple model instances across threads can still lead to issues +from threading import Thread + +from ultralytics import YOLO + +# Instantiate multiple models outside the thread +shared_model_1 = YOLO("yolov8n_1.pt") +shared_model_2 = YOLO("yolov8n_2.pt") + + +def predict(model, image_path): + """Runs prediction on an image using a specified YOLO model, returning the results.""" + results = model.predict(image_path) + # Process results + + +# Starting threads with individual model instances +Thread(target=predict, args=(shared_model_1, "image1.jpg")).start() +Thread(target=predict, args=(shared_model_2, "image2.jpg")).start() +``` + +Even though there are two separate model instances, the risk of concurrency issues still exists. If the internal implementation of `YOLO` is not thread-safe, using separate instances might not prevent race conditions, especially if these instances share any underlying resources or states that are not thread-local. + +## Thread-Safe Inference + +To perform thread-safe inference, you should instantiate a separate YOLO model within each thread. This ensures that each thread has its own isolated model instance, eliminating the risk of race conditions. + +### Thread-Safe Example + +Here's how to instantiate a YOLO model inside each thread for safe parallel inference: + +```python +# Safe: Instantiating a single model inside each thread +from threading import Thread + +from ultralytics import YOLO + + +def thread_safe_predict(image_path): + """Predict on an image using a new YOLO model instance in a thread-safe manner; takes image path as input.""" + local_model = YOLO("yolov8n.pt") + results = local_model.predict(image_path) + # Process results + + +# Starting threads that each have their own model instance +Thread(target=thread_safe_predict, args=("image1.jpg",)).start() +Thread(target=thread_safe_predict, args=("image2.jpg",)).start() +``` + +In this example, each thread creates its own `YOLO` instance. This prevents any thread from interfering with the model state of another, thus ensuring that each thread performs inference safely and without unexpected interactions with the other threads. + +## Conclusion + +When using YOLO models with Python's `threading`, always instantiate your models within the thread that will use them to ensure thread safety. This practice avoids race conditions and makes sure that your inference tasks run reliably. + +For more advanced scenarios and to further optimize your multi-threaded inference performance, consider using process-based parallelism with `multiprocessing` or leveraging a task queue with dedicated worker processes. + +## FAQ + +### How can I avoid race conditions when using YOLO models in a multi-threaded Python environment? + +To prevent race conditions when using Ultralytics YOLO models in a multi-threaded Python environment, instantiate a separate YOLO model within each thread. This ensures that each thread has its own isolated model instance, avoiding concurrent modification of the model state. + +Example: + +```python +from threading import Thread + +from ultralytics import YOLO + + +def thread_safe_predict(image_path): + """Predict on an image in a thread-safe manner.""" + local_model = YOLO("yolov8n.pt") + results = local_model.predict(image_path) + # Process results + + +Thread(target=thread_safe_predict, args=("image1.jpg",)).start() +Thread(target=thread_safe_predict, args=("image2.jpg",)).start() +``` + +For more information on ensuring thread safety, visit the [Thread-Safe Inference with YOLO Models](#thread-safe-inference). + +### What are the best practices for running multi-threaded YOLO model inference in Python? + +To run multi-threaded YOLO model inference safely in Python, follow these best practices: + +1. Instantiate YOLO models within each thread rather than sharing a single model instance across threads. +2. Use Python's `multiprocessing` module for parallel processing to avoid issues related to Global Interpreter Lock (GIL). +3. Release the GIL by using operations performed by YOLO's underlying C libraries. + +Example for thread-safe model instantiation: + +```python +from threading import Thread + +from ultralytics import YOLO + + +def thread_safe_predict(image_path): + """Runs inference in a thread-safe manner with a new YOLO model instance.""" + model = YOLO("yolov8n.pt") + results = model.predict(image_path) + # Process results + + +# Initiate multiple threads +Thread(target=thread_safe_predict, args=("image1.jpg",)).start() +Thread(target=thread_safe_predict, args=("image2.jpg",)).start() +``` + +For additional context, refer to the section on [Thread-Safe Inference](#thread-safe-inference). + +### Why should each thread have its own YOLO model instance? + +Each thread should have its own YOLO model instance to prevent race conditions. When a single model instance is shared among multiple threads, concurrent accesses can lead to unpredictable behavior and modifications of the model's internal state. By using separate instances, you ensure thread isolation, making your multi-threaded tasks reliable and safe. + +For detailed guidance, check the [Non-Thread-Safe Example: Single Model Instance](#non-thread-safe-example-single-model-instance) and [Thread-Safe Example](#thread-safe-example) sections. + +### How does Python's Global Interpreter Lock (GIL) affect YOLO model inference? + +Python's Global Interpreter Lock (GIL) allows only one thread to execute Python bytecode at a time, which can limit the performance of CPU-bound multi-threading tasks. However, for I/O-bound operations or processes that use libraries releasing the GIL, like YOLO's C libraries, you can still achieve concurrency. For enhanced performance, consider using process-based parallelism with Python's `multiprocessing` module. + +For more about threading in Python, see the [Understanding Python Threading](#understanding-python-threading) section. + +### Is it safer to use process-based parallelism instead of threading for YOLO model inference? + +Yes, using Python's `multiprocessing` module is safer and often more efficient for running YOLO model inference in parallel. Process-based parallelism creates separate memory spaces, avoiding the Global Interpreter Lock (GIL) and reducing the risk of concurrency issues. Each process will operate independently with its own YOLO model instance. + +For further details on process-based parallelism with YOLO models, refer to the page on [Thread-Safe Inference](#thread-safe-inference). diff --git a/ultralytics/docs/en/help/CI.md b/ultralytics/docs/en/help/CI.md new file mode 100644 index 0000000000000000000000000000000000000000..cfd588f49ce1009daff1f53ddb2f97ad252f7739 --- /dev/null +++ b/ultralytics/docs/en/help/CI.md @@ -0,0 +1,87 @@ +--- +comments: true +description: Learn about Ultralytics CI actions, Docker deployment, broken link checks, CodeQL analysis, and PyPI publishing to ensure high-quality code. +keywords: Ultralytics, Continuous Integration, CI, Docker deployment, CodeQL, PyPI publishing, code quality, automated testing +--- + +# Continuous Integration (CI) + +Continuous Integration (CI) is an essential aspect of software development which involves integrating changes and testing them automatically. CI allows us to maintain high-quality code by catching issues early and often in the development process. At Ultralytics, we use various CI tests to ensure the quality and integrity of our codebase. + +## CI Actions + +Here's a brief description of our CI actions: + +- **[CI](https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml):** This is our primary CI test that involves running unit tests, linting checks, and sometimes more comprehensive tests depending on the repository. +- **[Docker Deployment](https://github.com/ultralytics/ultralytics/actions/workflows/docker.yaml):** This test checks the deployment of the project using Docker to ensure the Dockerfile and related scripts are working correctly. +- **[Broken Links](https://github.com/ultralytics/ultralytics/actions/workflows/links.yml):** This test scans the codebase for any broken or dead links in our markdown or HTML files. +- **[CodeQL](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml):** CodeQL is a tool from GitHub that performs semantic analysis on our code, helping to find potential security vulnerabilities and maintain high-quality code. +- **[PyPI Publishing](https://github.com/ultralytics/ultralytics/actions/workflows/publish.yml):** This test checks if the project can be packaged and published to PyPi without any errors. + +### CI Results + +Below is the table showing the status of these CI tests for our main repositories: + +| Repository | CI | Docker Deployment | Broken Links | CodeQL | PyPI and Docs Publishing | +| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [yolov3](https://github.com/ultralytics/yolov3) | [![YOLOv3 CI](https://github.com/ultralytics/yolov3/actions/workflows/ci-testing.yml/badge.svg)](https://github.com/ultralytics/yolov3/actions/workflows/ci-testing.yml) | [![Publish Docker Images](https://github.com/ultralytics/yolov3/actions/workflows/docker.yml/badge.svg)](https://github.com/ultralytics/yolov3/actions/workflows/docker.yml) | [![Check Broken links](https://github.com/ultralytics/yolov3/actions/workflows/links.yml/badge.svg)](https://github.com/ultralytics/yolov3/actions/workflows/links.yml) | [![CodeQL](https://github.com/ultralytics/yolov3/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/ultralytics/yolov3/actions/workflows/codeql-analysis.yml) | | +| [yolov5](https://github.com/ultralytics/yolov5) | [![YOLOv5 CI](https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml/badge.svg)](https://github.com/ultralytics/yolov5/actions/workflows/ci-testing.yml) | [![Publish Docker Images](https://github.com/ultralytics/yolov5/actions/workflows/docker.yml/badge.svg)](https://github.com/ultralytics/yolov5/actions/workflows/docker.yml) | [![Check Broken links](https://github.com/ultralytics/yolov5/actions/workflows/links.yml/badge.svg)](https://github.com/ultralytics/yolov5/actions/workflows/links.yml) | [![CodeQL](https://github.com/ultralytics/yolov5/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/ultralytics/yolov5/actions/workflows/codeql-analysis.yml) | | +| [ultralytics](https://github.com/ultralytics/ultralytics) | [![ultralytics CI](https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml) | [![Publish Docker Images](https://github.com/ultralytics/ultralytics/actions/workflows/docker.yaml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/docker.yaml) | [![Check Broken links](https://github.com/ultralytics/ultralytics/actions/workflows/links.yml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/links.yml) | [![CodeQL](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml) | [![Publish to PyPI and Deploy Docs](https://github.com/ultralytics/ultralytics/actions/workflows/publish.yml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/publish.yml) | +| [hub](https://github.com/ultralytics/hub) | [![HUB CI](https://github.com/ultralytics/hub/actions/workflows/ci.yaml/badge.svg)](https://github.com/ultralytics/hub/actions/workflows/ci.yaml) | | [![Check Broken links](https://github.com/ultralytics/hub/actions/workflows/links.yml/badge.svg)](https://github.com/ultralytics/hub/actions/workflows/links.yml) | | | +| [docs](https://github.com/ultralytics/docs) | | | [![Check Broken links](https://github.com/ultralytics/docs/actions/workflows/links.yml/badge.svg)](https://github.com/ultralytics/docs/actions/workflows/links.yml)[![Check Domains](https://github.com/ultralytics/docs/actions/workflows/check_domains.yml/badge.svg)](https://github.com/ultralytics/docs/actions/workflows/check_domains.yml) | | [![pages-build-deployment](https://github.com/ultralytics/docs/actions/workflows/pages/pages-build-deployment/badge.svg)](https://github.com/ultralytics/docs/actions/workflows/pages/pages-build-deployment) | + +Each badge shows the status of the last run of the corresponding CI test on the `main` branch of the respective repository. If a test fails, the badge will display a "failing" status, and if it passes, it will display a "passing" status. + +If you notice a test failing, it would be a great help if you could report it through a GitHub issue in the respective repository. + +Remember, a successful CI test does not mean that everything is perfect. It is always recommended to manually review the code before deployment or merging changes. + +## Code Coverage + +Code coverage is a metric that represents the percentage of your codebase that is executed when your tests run. It provides insight into how well your tests exercise your code and can be crucial in identifying untested parts of your application. A high code coverage percentage is often associated with a lower likelihood of bugs. However, it's essential to understand that code coverage does not guarantee the absence of defects. It merely indicates which parts of the code have been executed by the tests. + +### Integration with [codecov.io](https://codecov.io/) + +At Ultralytics, we have integrated our repositories with [codecov.io](https://codecov.io/), a popular online platform for measuring and visualizing code coverage. Codecov provides detailed insights, coverage comparisons between commits, and visual overlays directly on your code, indicating which lines were covered. + +By integrating with Codecov, we aim to maintain and improve the quality of our code by focusing on areas that might be prone to errors or need further testing. + +### Coverage Results + +To quickly get a glimpse of the code coverage status of the `ultralytics` python package, we have included a badge and sunburst visual of the `ultralytics` coverage results. These images show the percentage of code covered by our tests, offering an at-a-glance metric of our testing efforts. For full details please see https://codecov.io/github/ultralytics/ultralytics. + +| Repository | Code Coverage | +| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | +| [ultralytics](https://github.com/ultralytics/ultralytics) | [![codecov](https://codecov.io/gh/ultralytics/ultralytics/branch/main/graph/badge.svg?token=HHW7IIVFVY)](https://codecov.io/gh/ultralytics/ultralytics) | + +In the sunburst graphic below, the innermost circle is the entire project, moving away from the center are folders then, finally, a single file. The size and color of each slice is representing the number of statements and the coverage, respectively. + + + + + +## FAQ + +### What is Continuous Integration (CI) in Ultralytics? + +Continuous Integration (CI) in Ultralytics involves automatically integrating and testing code changes to ensure high-quality standards. Our CI setup includes running [unit tests, linting checks, and comprehensive tests](https://github.com/ultralytics/ultralytics/actions/workflows/ci.yaml). Additionally, we perform [Docker deployment](https://github.com/ultralytics/ultralytics/actions/workflows/docker.yaml), [broken link checks](https://github.com/ultralytics/ultralytics/actions/workflows/links.yml), [CodeQL analysis](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml) for security vulnerabilities, and [PyPI publishing](https://github.com/ultralytics/ultralytics/actions/workflows/publish.yml) to package and distribute our software. + +### How does Ultralytics check for broken links in documentation and code? + +Ultralytics uses a specific CI action to [check for broken links](https://github.com/ultralytics/ultralytics/actions/workflows/links.yml) within our markdown and HTML files. This helps maintain the integrity of our documentation by scanning and identifying dead or broken links, ensuring that users always have access to accurate and live resources. + +### Why is CodeQL analysis important for Ultralytics' codebase? + +[CodeQL analysis](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml) is crucial for Ultralytics as it performs semantic code analysis to find potential security vulnerabilities and maintain high-quality standards. With CodeQL, we can proactively identify and mitigate risks in our code, helping us deliver robust and secure software solutions. + +### How does Ultralytics utilize Docker for deployment? + +Ultralytics employs Docker to validate the deployment of our projects through a dedicated CI action. This process ensures that our [Dockerfile and associated scripts](https://github.com/ultralytics/ultralytics/actions/workflows/docker.yaml) are functioning correctly, allowing for consistent and reproducible deployment environments which are critical for scalable and reliable AI solutions. + +### What is the role of automated PyPI publishing in Ultralytics? + +Automated [PyPI publishing](https://github.com/ultralytics/ultralytics/actions/workflows/publish.yml) ensures that our projects can be packaged and published without errors. This step is essential for distributing Ultralytics' Python packages, allowing users to easily install and use our tools via the Python Package Index (PyPI). + +### How does Ultralytics measure code coverage and why is it important? + +Ultralytics measures code coverage by integrating with [Codecov](https://codecov.io/github/ultralytics/ultralytics), providing insights into how much of the codebase is executed during tests. High code coverage can indicate well-tested code, helping to uncover untested areas that might be prone to bugs. Detailed code coverage metrics can be explored via badges displayed on our main repositories or directly on [Codecov](https://codecov.io/gh/ultralytics/ultralytics). diff --git a/ultralytics/docs/en/help/CLA.md b/ultralytics/docs/en/help/CLA.md new file mode 100644 index 0000000000000000000000000000000000000000..2b317aef5af1400d5c4fe8a238bb7764e7f54795 --- /dev/null +++ b/ultralytics/docs/en/help/CLA.md @@ -0,0 +1,50 @@ +--- +description: Review the terms for contributing to Ultralytics projects. Learn about copyright, patent licenses, and moral rights for your contributions. +keywords: Ultralytics, Contributor License Agreement, open source, contributions, copyright license, patent license, moral rights +--- + +# Ultralytics Individual Contributor License Agreement + +Thank you for your interest in contributing to open source software projects (“Projects”) made available by Ultralytics Inc. (“Ultralytics”). This Individual Contributor License Agreement (“Agreement”) sets out the terms governing any source code, object code, bug fixes, configuration changes, tools, specifications, documentation, data, materials, feedback, information or other works of authorship that you submit or have submitted, in any form and in any manner, to Ultralytics in respect of any Projects (collectively “Contributions”). If you have any questions respecting this Agreement, please contact hello@ultralytics.com. + +You agree that the following terms apply to all of your past, present and future Contributions. Except for the licenses granted in this Agreement, you retain all of your right, title and interest in and to your Contributions. + +**Copyright License.** You hereby grant, and agree to grant, to Ultralytics a non-exclusive, perpetual, irrevocable, worldwide, fully-paid, royalty-free, transferable copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, and distribute your Contributions and such derivative works, with the right to sublicense the foregoing rights through multiple tiers of sublicensees. + +**Patent License.** You hereby grant, and agree to grant, to Ultralytics a non-exclusive, perpetual, irrevocable, worldwide, fully-paid, royalty-free, transferable patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer your Contributions, where such license applies only to those patent claims licensable by you that are necessarily infringed by your Contributions alone or by combination of your Contributions with the Project to which such Contributions were submitted, with the right to sublicense the foregoing rights through multiple tiers of sublicensees. + +**Moral Rights.** To the fullest extent permitted under applicable law, you hereby waive, and agree not to assert, all of your “moral rights” in or relating to your Contributions for the benefit of Ultralytics, its assigns, and their respective direct and indirect sublicensees. + +**Third Party Content/Rights.** If your Contribution includes or is based on any source code, object code, bug fixes, configuration changes, tools, specifications, documentation, data, materials, feedback, information or other works of authorship that were not authored by you (“Third Party Content”) or if you are aware of any third party intellectual property or proprietary rights associated with your Contribution (“Third Party Rights”), then you agree to include with the submission of your Contribution full details respecting such Third Party Content and Third Party Rights, including, without limitation, identification of which aspects of your Contribution contain Third Party Content or are associated with Third Party Rights, the owner/author of the Third Party Content and Third Party Rights, where you obtained the Third Party Content, and any applicable third party license terms or restrictions respecting the Third Party Content and Third Party Rights. For greater certainty, the foregoing obligations respecting the identification of Third Party Content and Third Party Rights do not apply to any portion of a Project that is incorporated into your Contribution to that same Project. + +**Representations.** You represent that, other than the Third Party Content and Third Party Rights identified by you in accordance with this Agreement, you are the sole author of your Contributions and are legally entitled to grant the foregoing licenses and waivers in respect of your Contributions. If your Contributions were created in the course of your employment with your past or present employer(s), you represent that such employer(s) has authorized you to make your Contributions on behalf of such employer(s) or such employer(s) has waived all of their right, title or interest in or to your Contributions. + +**Disclaimer.** To the fullest extent permitted under applicable law, your Contributions are provided on an "asis" basis, without any warranties or conditions, express or implied, including, without limitation, any implied warranties or conditions of non-infringement, merchantability or fitness for a particular purpose. You are not required to provide support for your Contributions, except to the extent you desire to provide support. + +**No Obligation.** You acknowledge that Ultralytics is under no obligation to use or incorporate your Contributions into any of the Projects. The decision to use or incorporate your Contributions into any of the Projects will be made at the sole discretion of Ultralytics or its authorized delegates. + +**Disputes.** This Agreement shall be governed by and construed in accordance with the laws of the State of New York, United States of America, without giving effect to its principles or rules regarding conflicts of laws, other than such principles directing application of New York law. The parties hereby submit to venue in, and jurisdiction of the courts located in New York, New York for purposes relating to this Agreement. In the event that any of the provisions of this Agreement shall be held by a court or other tribunal of competent jurisdiction to be unenforceable, the remaining portions hereof shall remain in full force and effect. + +**Assignment.** You agree that Ultralytics may assign this Agreement, and all of its rights, obligations and licenses hereunder. + +## FAQ + +### What is the purpose of the Ultralytics Individual Contributor License Agreement? + +The Ultralytics Individual Contributor License Agreement (ICLA) governs the terms under which you contribute to Ultralytics' open-source projects. It sets out the rights and obligations related to your contributions, including granting copyright and patent licenses, waiving moral rights, and disclosing any third-party content. + +### Why do I need to agree to the Copyright License in the ICLA? + +Agreeing to the Copyright License allows Ultralytics to use and distribute your contributions, including making derivative works. This ensures that your contributions can be integrated into Ultralytics projects and shared with the community, fostering collaboration and software development. + +### How does the Patent License benefit both contributors and Ultralytics? + +The Patent License grants Ultralytics the rights to use, make, and sell contributions covered by your patents, which is crucial for product development and commercialization. In return, it allows your patented innovations to be more widely used and recognized, promoting innovation within the community. + +### What should I do if my contribution contains third-party content? + +If your contribution includes third-party content or you are aware of any third-party intellectual property rights, you must provide full details of such content and rights when submitting your contribution. This includes identifying the third-party content, its author, and the applicable license terms. For more information on third-party content, refer to the Third Party Content/Rights section of the Agreement. + +### What happens if Ultralytics does not use my contributions? + +Ultralytics is not obligated to use or incorporate your contributions into any projects. The decision to use or integrate contributions is at Ultralytics' sole discretion. This means that while your contributions are valuable, they may not always align with the project's current needs or directions. For further details, see the No Obligation section. diff --git a/ultralytics/docs/en/help/FAQ.md b/ultralytics/docs/en/help/FAQ.md new file mode 100644 index 0000000000000000000000000000000000000000..5ad7300a39a4528e9733160775680cad60d2e6b0 --- /dev/null +++ b/ultralytics/docs/en/help/FAQ.md @@ -0,0 +1,229 @@ +--- +comments: true +description: Explore common questions and solutions related to Ultralytics YOLO, from hardware requirements to model fine-tuning and real-time detection. +keywords: Ultralytics, YOLO, FAQ, object detection, hardware requirements, fine-tuning, ONNX, TensorFlow, real-time detection, model accuracy +--- + +# Ultralytics YOLO Frequently Asked Questions (FAQ) + +This FAQ section addresses common questions and issues users might encounter while working with [Ultralytics](https://ultralytics.com) YOLO repositories. + +## FAQ + +### What is Ultralytics and what does it offer? + +Ultralytics is a computer vision AI company specializing in state-of-the-art object detection and image segmentation models, with a focus on the YOLO (You Only Look Once) family. Their offerings include: + +- Open-source implementations of [YOLOv5](https://docs.ultralytics.com/models/yolov5/) and [YOLOv8](https://docs.ultralytics.com/models/yolov8/) +- A wide range of [pre-trained models](https://docs.ultralytics.com/models/) for various computer vision tasks +- A comprehensive [Python package](https://docs.ultralytics.com/usage/python/) for seamless integration of YOLO models into projects +- Versatile [tools](https://docs.ultralytics.com/modes/) for training, testing, and deploying models +- [Extensive documentation](https://docs.ultralytics.com/) and a supportive community + +### How do I install the Ultralytics package? + +Installing the Ultralytics package is straightforward using pip: + +``` +pip install ultralytics +``` + +For the latest development version, install directly from the GitHub repository: + +``` +pip install git+https://github.com/ultralytics/ultralytics.git +``` + +Detailed installation instructions can be found in the [quickstart guide](https://docs.ultralytics.com/quickstart/). + +### What are the system requirements for running Ultralytics models? + +Minimum requirements: + +- Python 3.7+ +- PyTorch 1.7+ +- CUDA-compatible GPU (for GPU acceleration) + +Recommended setup: + +- Python 3.8+ +- PyTorch 1.10+ +- NVIDIA GPU with CUDA 11.2+ +- 8GB+ RAM +- 50GB+ free disk space (for dataset storage and model training) + +For troubleshooting common issues, visit the [YOLO Common Issues](https://docs.ultralytics.com/guides/yolo-common-issues/) page. + +### How can I train a custom YOLOv8 model on my own dataset? + +To train a custom YOLOv8 model: + +1. Prepare your dataset in YOLO format (images and corresponding label txt files). +2. Create a YAML file describing your dataset structure and classes. +3. Use the following Python code to start training: + +```python +from ultralytics import YOLO + +# Load a model +model = YOLO("yolov8n.yaml") # build a new model from scratch +model = YOLO("yolov8n.pt") # load a pretrained model (recommended for training) + +# Train the model +results = model.train(data="path/to/your/data.yaml", epochs=100, imgsz=640) +``` + +For a more in-depth guide, including data preparation and advanced training options, refer to the comprehensive [training guide](https://docs.ultralytics.com/modes/train/). + +### What pretrained models are available in Ultralytics? + +Ultralytics offers a diverse range of pretrained YOLOv8 models for various tasks: + +- Object Detection: YOLOv8n, YOLOv8s, YOLOv8m, YOLOv8l, YOLOv8x +- Instance Segmentation: YOLOv8n-seg, YOLOv8s-seg, YOLOv8m-seg, YOLOv8l-seg, YOLOv8x-seg +- Classification: YOLOv8n-cls, YOLOv8s-cls, YOLOv8m-cls, YOLOv8l-cls, YOLOv8x-cls + +These models vary in size and complexity, offering different trade-offs between speed and accuracy. Explore the full range of [pretrained models](https://docs.ultralytics.com/models/yolov8/) to find the best fit for your project. + +### How do I perform inference using a trained Ultralytics model? + +To perform inference with a trained model: + +```python +from ultralytics import YOLO + +# Load a model +model = YOLO("path/to/your/model.pt") + +# Perform inference +results = model("path/to/image.jpg") + +# Process results +for r in results: + print(r.boxes) # print bbox predictions + print(r.masks) # print mask predictions + print(r.probs) # print class probabilities +``` + +For advanced inference options, including batch processing and video inference, check out the detailed [prediction guide](https://docs.ultralytics.com/modes/predict/). + +### Can Ultralytics models be deployed on edge devices or in production environments? + +Absolutely! Ultralytics models are designed for versatile deployment across various platforms: + +- Edge devices: Optimize inference on devices like NVIDIA Jetson or Intel Neural Compute Stick using TensorRT, ONNX, or OpenVINO. +- Mobile: Deploy on Android or iOS devices by converting models to TFLite or Core ML. +- Cloud: Leverage frameworks like TensorFlow Serving or PyTorch Serve for scalable cloud deployments. +- Web: Implement in-browser inference using ONNX.js or TensorFlow.js. + +Ultralytics provides export functions to convert models to various formats for deployment. Explore the wide range of [deployment options](https://docs.ultralytics.com/guides/model-deployment-options/) to find the best solution for your use case. + +### What's the difference between YOLOv5 and YOLOv8? + +Key distinctions include: + +- Architecture: YOLOv8 features an improved backbone and head design for enhanced performance. +- Performance: YOLOv8 generally offers superior accuracy and speed compared to YOLOv5. +- Tasks: YOLOv8 natively supports object detection, instance segmentation, and classification in a unified framework. +- Codebase: YOLOv8 is implemented with a more modular and extensible architecture, facilitating easier customization and extension. +- Training: YOLOv8 incorporates advanced training techniques like multi-dataset training and hyperparameter evolution for improved results. + +For an in-depth comparison of features and performance metrics, visit the [YOLOv5 vs YOLOv8](https://www.ultralytics.com/yolo) comparison page. + +### How can I contribute to the Ultralytics open-source project? + +Contributing to Ultralytics is a great way to improve the project and expand your skills. Here's how you can get involved: + +1. Fork the Ultralytics repository on GitHub. +2. Create a new branch for your feature or bug fix. +3. Make your changes and ensure all tests pass. +4. Submit a pull request with a clear description of your changes. +5. Participate in the code review process. + +You can also contribute by reporting bugs, suggesting features, or improving documentation. For detailed guidelines and best practices, refer to the [contributing guide](https://docs.ultralytics.com/help/contributing/). + +### How do I install the Ultralytics package in Python? + +Installing the Ultralytics package in Python is simple. Use pip by running the following command in your terminal or command prompt: + +```bash +pip install ultralytics +``` + +For the cutting-edge development version, install directly from the GitHub repository: + +```bash +pip install git+https://github.com/ultralytics/ultralytics.git +``` + +For environment-specific installation instructions and troubleshooting tips, consult the comprehensive [quickstart guide](https://docs.ultralytics.com/quickstart/). + +### What are the main features of Ultralytics YOLO? + +Ultralytics YOLO boasts a rich set of features for advanced object detection and image segmentation: + +- Real-Time Detection: Efficiently detect and classify objects in real-time scenarios. +- Pre-Trained Models: Access a variety of [pretrained models](https://docs.ultralytics.com/models/yolov8/) that balance speed and accuracy for different use cases. +- Custom Training: Easily fine-tune models on custom datasets with the flexible [training pipeline](https://docs.ultralytics.com/modes/train/). +- Wide [Deployment Options](https://docs.ultralytics.com/guides/model-deployment-options/): Export models to various formats like TensorRT, ONNX, and CoreML for deployment across different platforms. +- Extensive Documentation: Benefit from comprehensive [documentation](https://docs.ultralytics.com/) and a supportive community to guide you through your computer vision journey. + +Explore the [YOLO models page](https://docs.ultralytics.com/models/yolov8/) for an in-depth look at the capabilities and architectures of different YOLO versions. + +### How can I improve the performance of my YOLO model? + +Enhancing your YOLO model's performance can be achieved through several techniques: + +1. Hyperparameter Tuning: Experiment with different hyperparameters using the [Hyperparameter Tuning Guide](https://docs.ultralytics.com/guides/hyperparameter-tuning/) to optimize model performance. +2. Data Augmentation: Implement techniques like flip, scale, rotate, and color adjustments to enhance your training dataset and improve model generalization. +3. Transfer Learning: Leverage pre-trained models and fine-tune them on your specific dataset using the [Train YOLOv8](https://docs.ultralytics.com/modes/train/) guide. +4. Export to Efficient Formats: Convert your model to optimized formats like TensorRT or ONNX for faster inference using the [Export guide](../modes/export.md). +5. Benchmarking: Utilize the [Benchmark Mode](https://docs.ultralytics.com/modes/benchmark/) to measure and improve inference speed and accuracy systematically. + +### Can I deploy Ultralytics YOLO models on mobile and edge devices? + +Yes, Ultralytics YOLO models are designed for versatile deployment, including mobile and edge devices: + +- Mobile: Convert models to TFLite or CoreML for seamless integration into Android or iOS apps. Refer to the [TFLite Integration Guide](https://docs.ultralytics.com/integrations/tflite/) and [CoreML Integration Guide](https://docs.ultralytics.com/integrations/coreml/) for platform-specific instructions. +- Edge Devices: Optimize inference on devices like NVIDIA Jetson or other edge hardware using TensorRT or ONNX. The [Edge TPU Integration Guide](https://docs.ultralytics.com/integrations/edge-tpu/) provides detailed steps for edge deployment. + +For a comprehensive overview of deployment strategies across various platforms, consult the [deployment options guide](https://docs.ultralytics.com/guides/model-deployment-options/). + +### How can I perform inference using a trained Ultralytics YOLO model? + +Performing inference with a trained Ultralytics YOLO model is straightforward: + +1. Load the Model: + +```python +from ultralytics import YOLO + +model = YOLO("path/to/your/model.pt") +``` + +2. Run Inference: + +```python +results = model("path/to/image.jpg") + +for r in results: + print(r.boxes) # print bounding box predictions + print(r.masks) # print mask predictions + print(r.probs) # print class probabilities +``` + +For advanced inference techniques, including batch processing, video inference, and custom preprocessing, refer to the detailed [prediction guide](https://docs.ultralytics.com/modes/predict/). + +### Where can I find examples and tutorials for using Ultralytics? + +Ultralytics provides a wealth of resources to help you get started and master their tools: + +- 📚 [Official documentation](https://docs.ultralytics.com/): Comprehensive guides, API references, and best practices. +- 💻 [GitHub repository](https://github.com/ultralytics/ultralytics): Source code, example scripts, and community contributions. +- ✍️ [Ultralytics blog](https://www.ultralytics.com/blog): In-depth articles, use cases, and technical insights. +- 💬 [Community forums](https://community.ultralytics.com/): Connect with other users, ask questions, and share your experiences. +- 🎥 [YouTube channel](https://youtube.com/ultralytics?sub_confirmation=1): Video tutorials, demos, and webinars on various Ultralytics topics. + +These resources provide code examples, real-world use cases, and step-by-step guides for various tasks using Ultralytics models. + +If you need further assistance, don't hesitate to consult the Ultralytics documentation or reach out to the community through [GitHub Issues](https://github.com/ultralytics/ultralytics/issues) or the official [discussion forum](https://github.com/orgs/ultralytics/discussions). diff --git a/ultralytics/docs/en/help/code_of_conduct.md b/ultralytics/docs/en/help/code_of_conduct.md new file mode 100644 index 0000000000000000000000000000000000000000..d1ab2c81462f757f71726311c5f4e11dcbadf9d1 --- /dev/null +++ b/ultralytics/docs/en/help/code_of_conduct.md @@ -0,0 +1,109 @@ +--- +comments: true +description: Join our welcoming community! Learn about the Ultralytics Code of Conduct to ensure a harassment-free experience for all participants. +keywords: Ultralytics, Contributor Covenant, Code of Conduct, community guidelines, harassment-free, inclusive community, diversity, enforcement policy +--- + +# Ultralytics Contributor Covenant Code of Conduct + +## Our Pledge + +We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socioeconomic status, nationality, personal appearance, race, religion, or sexual identity and orientation. + +We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community. + +## Our Standards + +Examples of behavior that contributes to a positive environment for our community include: + +- Demonstrating empathy and kindness toward other people +- Being respectful of differing opinions, viewpoints, and experiences +- Giving and gracefully accepting constructive feedback +- Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience +- Focusing on what is best not just for us as individuals, but for the overall community + +Examples of unacceptable behavior include: + +- The use of sexualized language or imagery, and sexual attention or advances of any kind +- Trolling, insulting or derogatory comments, and personal or political attacks +- Public or private harassment +- Publishing others' private information, such as a physical or email address, without their explicit permission +- Other conduct which could reasonably be considered inappropriate in a professional setting + +## Enforcement Responsibilities + +Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful. + +Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate. + +## Scope + +This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the community leaders responsible for enforcement at hello@ultralytics.com. All complaints will be reviewed and investigated promptly and fairly. + +All community leaders are obligated to respect the privacy and security of the reporter of any incident. + +## Enforcement Guidelines + +Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct: + +### 1. Correction + +**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community. + +**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested. + +### 2. Warning + +**Community Impact**: A violation through a single incident or series of actions. + +**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban. + +### 3. Temporary Ban + +**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior. + +**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban. + +### 4. Permanent Ban + +**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals. + +**Consequence**: A permanent ban from any sort of public interaction within the community. + +## Attribution + +This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.0, available at https://www.contributor-covenant.org/version/2/0/code_of_conduct.html. + +Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder](https://github.com/mozilla/diversity). + +For answers to common questions about this code of conduct, see the FAQ at https://www.contributor-covenant.org/faq. Translations are available at https://www.contributor-covenant.org/translations. + +[homepage]: https://www.contributor-covenant.org + +## FAQ + +### What is the Ultralytics Contributor Covenant Code of Conduct? + +The Ultralytics Contributor Covenant Code of Conduct aims to create a harassment-free experience for everyone participating in the Ultralytics community. It applies to all community interactions, including online and offline activities. The code details expected behaviors, unacceptable behaviors, and the enforcement responsibilities of community leaders. For more detailed information, see the [Enforcement Responsibilities](#enforcement-responsibilities) section. + +### How does the enforcement process work for the Ultralytics Code of Conduct? + +Enforcement of the Ultralytics Code of Conduct is managed by community leaders who can take appropriate action in response to any behavior deemed inappropriate. This could range from a private warning to a permanent ban, depending on the severity of the violation. Instances of misconduct can be reported to hello@ultralytics.com for investigation. Learn more about the enforcement steps in the [Enforcement Guidelines](#enforcement-guidelines) section. + +### Why is diversity and inclusion important in the Ultralytics community? + +Ultralytics values diversity and inclusion as fundamental aspects for fostering innovation and creativity within its community. A diverse and inclusive environment allows different perspectives and experiences to contribute to an open, welcoming, and healthy community. This commitment is reflected in our [Pledge](#our-pledge) to ensure a harassment-free experience for everyone regardless of their background. + +### How can I contribute to Ultralytics while adhering to the Code of Conduct? + +Contributing to Ultralytics means engaging positively and respectfully with other community members. You can contribute by demonstrating empathy, offering and accepting constructive feedback, and taking responsibility for any mistakes. Always aim to contribute in a way that benefits the entire community. For more details on acceptable behaviors, refer to the [Our Standards](#our-standards) section. + +### Where can I find additional information about the Ultralytics Code of Conduct? + +For more comprehensive details about the Ultralytics Code of Conduct, including reporting guidelines and enforcement policies, you can visit the [Contributor Covenant homepage](https://www.contributor-covenant.org/version/2/0/code_of_conduct.html) or check the [FAQ section of Contributor Covenant](https://www.contributor-covenant.org/faq). Learn more about Ultralytics' goals and initiatives on [our brand page](https://www.ultralytics.com/brand) and [about page](https://www.ultralytics.com/about). + +Should you have more questions or need further assistance, check our [Help Center](../help/FAQ.md) and [Contributing Guide](../help/contributing.md) for more information. diff --git a/ultralytics/docs/en/help/contributing.md b/ultralytics/docs/en/help/contributing.md new file mode 100644 index 0000000000000000000000000000000000000000..d585fb600dd5538d7f5eb767551a81284e3bbc0c --- /dev/null +++ b/ultralytics/docs/en/help/contributing.md @@ -0,0 +1,168 @@ +--- +comments: true +description: Learn how to contribute to Ultralytics YOLO open-source repositories. Follow guidelines for pull requests, code of conduct, and bug reporting. +keywords: Ultralytics, YOLO, open-source, contribution, pull request, code of conduct, bug reporting, GitHub, CLA, Google-style docstrings +--- + +# Contributing to Ultralytics Open-Source Projects + +Welcome! We're thrilled that you're considering contributing to our [Ultralytics](https://ultralytics.com) [open-source](https://github.com/ultralytics) projects. Your involvement not only helps enhance the quality of our repositories but also benefits the entire community. This guide provides clear guidelines and best practices to help you get started. + + + + +## Table of Contents + +1. [Code of Conduct](#code-of-conduct) +2. [Contributing via Pull Requests](#contributing-via-pull-requests) + - [CLA Signing](#cla-signing) + - [Google-Style Docstrings](#google-style-docstrings) + - [GitHub Actions CI Tests](#github-actions-ci-tests) +3. [Reporting Bugs](#reporting-bugs) +4. [License](#license) +5. [Conclusion](#conclusion) +6. [FAQ](#faq) + +## Code of Conduct + +To ensure a welcoming and inclusive environment for everyone, all contributors must adhere to our [Code of Conduct](https://docs.ultralytics.com/help/code_of_conduct). Respect, kindness, and professionalism are at the heart of our community. + +## Contributing via Pull Requests + +We greatly appreciate contributions in the form of pull requests. To make the review process as smooth as possible, please follow these steps: + +1. **[Fork the repository](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo):** Start by forking the Ultralytics YOLO repository to your GitHub account. + +2. **[Create a branch](https://docs.github.com/en/desktop/making-changes-in-a-branch/managing-branches-in-github-desktop):** Create a new branch in your forked repository with a clear, descriptive name that reflects your changes. + +3. **Make your changes:** Ensure your code adheres to the project's style guidelines and does not introduce any new errors or warnings. + +4. **[Test your changes](https://github.com/ultralytics/ultralytics/tree/main/tests):** Before submitting, test your changes locally to confirm they work as expected and don't cause any new issues. + +5. **[Commit your changes](https://docs.github.com/en/desktop/making-changes-in-a-branch/committing-and-reviewing-changes-to-your-project-in-github-desktop):** Commit your changes with a concise and descriptive commit message. If your changes address a specific issue, include the issue number in your commit message. + +6. **[Create a pull request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request):** Submit a pull request from your forked repository to the main Ultralytics YOLO repository. Provide a clear and detailed explanation of your changes and how they improve the project. + +### CLA Signing + +Before we can merge your pull request, you must sign our [Contributor License Agreement (CLA)](https://docs.ultralytics.com/help/CLA). This legal agreement ensures that your contributions are properly licensed, allowing the project to continue being distributed under the AGPL-3.0 license. + +After submitting your pull request, the CLA bot will guide you through the signing process. To sign the CLA, simply add a comment in your PR stating: + +``` +I have read the CLA Document and I sign the CLA +``` + +### Google-Style Docstrings + +When adding new functions or classes, please include [Google-style docstrings](https://google.github.io/styleguide/pyguide.html). These docstrings provide clear, standardized documentation that helps other developers understand and maintain your code. + +!!! Example "Example Docstrings" + + === "Google-style" + + This example illustrates a Google-style docstring. Ensure that both input and output `types` are always enclosed in parentheses, e.g., `(bool)`. + + ```python + def example_function(arg1, arg2=4): + """ + Example function demonstrating Google-style docstrings. + + Args: + arg1 (int): The first argument. + arg2 (int): The second argument, with a default value of 4. + + Returns: + (bool): True if successful, False otherwise. + + Examples: + >>> result = example_function(1, 2) # returns False + """ + if arg1 == arg2: + return True + return False + ``` + + === "Google-style with type hints" + + This example includes both a Google-style docstring and type hints for arguments and returns, though using either independently is also acceptable. + + ```python + def example_function(arg1: int, arg2: int = 4) -> bool: + """ + Example function demonstrating Google-style docstrings. + + Args: + arg1: The first argument. + arg2: The second argument, with a default value of 4. + + Returns: + True if successful, False otherwise. + + Examples: + >>> result = example_function(1, 2) # returns False + """ + if arg1 == arg2: + return True + return False + ``` + + === "Single-line" + + For smaller or simpler functions, a single-line docstring may be sufficient. The docstring must use three double-quotes, be a complete sentence, start with a capital letter, and end with a period. + + ```python + def example_small_function(arg1: int, arg2: int = 4) -> bool: + """Example function with a single-line docstring.""" + return arg1 == arg2 + ``` + +### GitHub Actions CI Tests + +All pull requests must pass the GitHub Actions [Continuous Integration](https://docs.ultralytics.com/help/CI) (CI) tests before they can be merged. These tests include linting, unit tests, and other checks to ensure that your changes meet the project's quality standards. Review the CI output and address any issues that arise. + +## Reporting Bugs + +We highly value bug reports as they help us maintain the quality of our projects. When reporting a bug, please provide a [Minimum Reproducible Example](https://docs.ultralytics.com/help/minimum_reproducible_example)—a simple, clear code example that consistently reproduces the issue. This allows us to quickly identify and resolve the problem. + +## License + +Ultralytics uses the [GNU Affero General Public License v3.0 (AGPL-3.0)](https://github.com/ultralytics/ultralytics/blob/main/LICENSE) for its repositories. This license promotes openness, transparency, and collaborative improvement in software development. It ensures that all users have the freedom to use, modify, and share the software, fostering a strong community of collaboration and innovation. + +We encourage all contributors to familiarize themselves with the terms of the AGPL-3.0 license to contribute effectively and ethically to the Ultralytics open-source community. + +## Conclusion + +Thank you for your interest in contributing to [Ultralytics](https://ultralytics.com) [open-source](https://github.com/ultralytics) YOLO projects. Your participation is essential in shaping the future of our software and building a vibrant community of innovation and collaboration. Whether you're enhancing code, reporting bugs, or suggesting new features, your contributions are invaluable. + +We're excited to see your ideas come to life and appreciate your commitment to advancing object detection technology. Together, let's continue to grow and innovate in this exciting open-source journey. Happy coding! 🚀🌟 + +## FAQ + +### Why should I contribute to Ultralytics YOLO open-source repositories? + +Contributing to Ultralytics YOLO open-source repositories improves the software, making it more robust and feature-rich for the entire community. Contributions can include code enhancements, bug fixes, documentation improvements, and new feature implementations. Additionally, contributing allows you to collaborate with other skilled developers and experts in the field, enhancing your own skills and reputation. For details on how to get started, refer to the [Contributing via Pull Requests](#contributing-via-pull-requests) section. + +### How do I sign the Contributor License Agreement (CLA) for Ultralytics YOLO? + +To sign the Contributor License Agreement (CLA), follow the instructions provided by the CLA bot after submitting your pull request. This process ensures that your contributions are properly licensed under the AGPL-3.0 license, maintaining the legal integrity of the open-source project. Add a comment in your pull request stating: + +``` +I have read the CLA Document and I sign the CLA. +``` + +For more information, see the [CLA Signing](#cla-signing) section. + +### What are Google-style docstrings, and why are they required for Ultralytics YOLO contributions? + +Google-style docstrings provide clear, concise documentation for functions and classes, improving code readability and maintainability. These docstrings outline the function's purpose, arguments, and return values with specific formatting rules. When contributing to Ultralytics YOLO, following Google-style docstrings ensures that your additions are well-documented and easily understood. For examples and guidelines, visit the [Google-Style Docstrings](#google-style-docstrings) section. + +### How can I ensure my changes pass the GitHub Actions CI tests? + +Before your pull request can be merged, it must pass all GitHub Actions Continuous Integration (CI) tests. These tests include linting, unit tests, and other checks to ensure the code meets + +the project's quality standards. Review the CI output and fix any issues. For detailed information on the CI process and troubleshooting tips, see the [GitHub Actions CI Tests](#github-actions-ci-tests) section. + +### How do I report a bug in Ultralytics YOLO repositories? + +To report a bug, provide a clear and concise [Minimum Reproducible Example](https://docs.ultralytics.com/help/minimum_reproducible_example) along with your bug report. This helps developers quickly identify and fix the issue. Ensure your example is minimal yet sufficient to replicate the problem. For more detailed steps on reporting bugs, refer to the [Reporting Bugs](#reporting-bugs) section. diff --git a/ultralytics/docs/en/help/environmental-health-safety.md b/ultralytics/docs/en/help/environmental-health-safety.md new file mode 100644 index 0000000000000000000000000000000000000000..bdf6cfbd467270f78a920d9c53895fc55dc79b33 --- /dev/null +++ b/ultralytics/docs/en/help/environmental-health-safety.md @@ -0,0 +1,63 @@ +--- +comments: false +description: Explore Ultralytics' commitment to Environmental, Health, and Safety (EHS) policies. Learn about our measures to ensure safety, compliance, and sustainability. +keywords: Ultralytics, EHS policy, safety, sustainability, environmental impact, health and safety, risk management, compliance, continuous improvement +--- + +# Ultralytics Environmental, Health and Safety (EHS) Policy + +At Ultralytics, we recognize that the long-term success of our company relies not only on the products and services we offer, but also the manner in which we conduct our business. We are committed to ensuring the safety and well-being of our employees, stakeholders, and the environment, and we will continuously strive to mitigate our impact on the environment while promoting health and safety. + +## Policy Principles + +1. **Compliance**: We will comply with all applicable laws, regulations, and standards related to EHS, and we will strive to exceed these standards where possible. + +2. **Prevention**: We will work to prevent accidents, injuries, and environmental harm by implementing risk management measures and ensuring all our operations and procedures are safe. + +3. **Continuous Improvement**: We will continuously improve our EHS performance by setting measurable objectives, monitoring our performance, auditing our operations, and revising our policies and procedures as needed. + +4. **Communication**: We will communicate openly about our EHS performance and will engage with stakeholders to understand and address their concerns and expectations. + +5. **Education and Training**: We will educate and train our employees and contractors in appropriate EHS procedures and practices. + +## Implementation Measures + +1. **Responsibility and Accountability**: Every employee and contractor working at or with Ultralytics is responsible for adhering to this policy. Managers and supervisors are accountable for ensuring this policy is implemented within their areas of control. + +2. **Risk Management**: We will identify, assess, and manage EHS risks associated with our operations and activities to prevent accidents, injuries, and environmental harm. + +3. **Resource Allocation**: We will allocate the necessary resources to ensure the effective implementation of our EHS policy, including the necessary equipment, personnel, and training. + +4. **Emergency Preparedness and Response**: We will develop, maintain, and test emergency preparedness and response plans to ensure we can respond effectively to EHS incidents. + +5. **Monitoring and Review**: We will monitor and review our EHS performance regularly to identify opportunities for improvement and ensure we are meeting our objectives. + +This policy reflects our commitment to minimizing our environmental footprint, ensuring the safety and well-being of our employees, and continuously improving our performance. + +Please remember that the implementation of an effective EHS policy requires the involvement and commitment of everyone working at or with Ultralytics. We encourage you to take personal responsibility for your safety and the safety of others, and to take care of the environment in which we live and work. + +## FAQ + +### What is Ultralytics' Environmental, Health, and Safety (EHS) policy? + +Ultralytics' Environmental, Health, and Safety (EHS) policy is a comprehensive framework designed to ensure the safety and well-being of employees, stakeholders, and the environment. It emphasizes compliance with relevant laws, accident prevention through risk management, continuous improvement through measurable objectives, open communication, and education and training for employees. By following these principles, Ultralytics aims to minimize its environmental footprint and promote sustainable practices. [Learn more about Ultralytics' commitment to EHS](https://www.ultralytics.com/about). + +### How does Ultralytics ensure compliance with EHS regulations? + +Ultralytics ensures compliance with EHS regulations by adhering to all applicable laws, regulations, and standards. The company not only strives to meet these requirements but often exceeds them by implementing stringent internal policies. Regular audits, monitoring, and reviews are conducted to ensure ongoing compliance. Managers and supervisors are also accountable for ensuring these standards are maintained within their areas of control. For more details, refer to the [Policy Principles section](#policy-principles) on the documentation page. + +### Why is continuous improvement a key principle in Ultralytics' EHS policy? + +Continuous improvement is essential in Ultralytics' EHS policy because it ensures the company consistently enhances its performance in environmental, health, and safety areas. By setting measurable objectives, monitoring performance, and revising policies and procedures as needed, Ultralytics can adapt to new challenges and optimize its processes. This approach not only mitigates risks but also demonstrates Ultralytics' commitment to sustainability and excellence. For practical examples of continuous improvement, check the [Implementation Measures section](#implementation-measures). + +### What are the roles and responsibilities of employees in implementing the EHS policy at Ultralytics? + +Every employee and contractor at Ultralytics is responsible for adhering to the EHS policy. This includes following safety protocols, participating in necessary training, and taking personal responsibility for their safety and the safety of others. Managers and supervisors have an added responsibility of ensuring the EHS policy is effectively implemented within their areas of control, which involves risk assessments and resource allocation. For more information about responsibility and accountability, see the [Implementation Measures section](#implementation-measures). + +### How does Ultralytics handle emergency preparedness and response in its EHS policy? + +Ultralytics handles emergency preparedness and response by developing, maintaining, and regularly testing emergency plans to address potential EHS incidents effectively. These plans ensure that the company can respond swiftly and efficiently to minimize harm to employees, the environment, and property. Regular training and drills are conducted to keep the response teams prepared for various emergency scenarios. For additional context, refer to the [emergency preparedness and response measure](#implementation-measures). + +### How does Ultralytics engage with stakeholders regarding its EHS performance? + +Ultralytics communicates openly with stakeholders about its EHS performance by sharing relevant information and addressing any concerns or expectations. This engagement includes regular reporting on EHS activities, performance metrics, and improvement initiatives. Stakeholders are also encouraged to provide feedback, which helps Ultralytics to refine its policies and practices continually. Learn more about this commitment in the [Communication principle](#policy-principles) section. diff --git a/ultralytics/docs/en/help/index.md b/ultralytics/docs/en/help/index.md new file mode 100644 index 0000000000000000000000000000000000000000..67242bedcf911f564657e3f8a872d3041cff4fd4 --- /dev/null +++ b/ultralytics/docs/en/help/index.md @@ -0,0 +1,49 @@ +--- +comments: true +description: Explore the Ultralytics Help Center with guides, FAQs, CI processes, and policies to support your YOLO model experience and contributions. +keywords: Ultralytics, YOLO, help center, documentation, guides, FAQ, contributing, CI, MRE, CLA, code of conduct, security policy, privacy policy +--- + +Welcome to the Ultralytics Help page! We are dedicated to providing you with detailed resources to enhance your experience with the Ultralytics YOLO models and repositories. This page serves as your portal to guides and documentation designed to assist you with various tasks and answer questions you may encounter while engaging with our repositories. + +- [Frequently Asked Questions (FAQ)](FAQ.md): Find answers to common questions and issues encountered by the community of Ultralytics YOLO users and contributors. +- [Contributing Guide](contributing.md): Discover the protocols for making contributions, including how to submit pull requests, report bugs, and more. +- [Continuous Integration (CI) Guide](CI.md): Gain insights into the CI processes we employ, complete with status reports for each Ultralytics repository. +- [Contributor License Agreement (CLA)](CLA.md): Review the CLA to understand the rights and responsibilities associated with contributing to Ultralytics projects. +- [Minimum Reproducible Example (MRE) Guide](minimum_reproducible_example.md): Learn the process for creating an MRE, which is crucial for the timely and effective resolution of bug reports. +- [Code of Conduct](code_of_conduct.md): Our community guidelines support a respectful and open atmosphere for all collaborators. +- [Environmental, Health and Safety (EHS) Policy](environmental-health-safety.md): Delve into our commitment to sustainability and the well-being of all our stakeholders. +- [Security Policy](security.md): Familiarize yourself with our security protocols and the procedure for reporting vulnerabilities. +- [Privacy Policy](privacy.md): Read our privacy policy to understand how we protect your data and respect your privacy in all our services and operations. + +We encourage you to review these resources for a seamless and productive experience. Our aim is to foster a helpful and friendly environment for everyone in the Ultralytics community. Should you require additional support, please feel free to reach out via GitHub Issues or our official discussion forums. Happy coding! + +## FAQ + +### What is Ultralytics YOLO and how does it benefit my machine learning projects? + +Ultralytics YOLO (You Only Look Once) is a state-of-the-art, real-time object detection model. Its latest version, YOLOv8, enhances speed, accuracy, and versatility, making it ideal for a wide range of applications, from real-time video analytics to advanced machine learning research. YOLO's efficiency in detecting objects in images and videos has made it the go-to solution for businesses and researchers looking to integrate robust computer vision capabilities into their projects. + +For more details on YOLOv8, visit the [YOLOv8 documentation](../tasks/detect.md). + +### How do I contribute to Ultralytics YOLO repositories? + +Contributing to Ultralytics YOLO repositories is straightforward. Start by reviewing the [Contributing Guide](../help/contributing.md) to understand the protocols for submitting pull requests, reporting bugs, and more. You'll also need to sign the [Contributor License Agreement (CLA)](../help/CLA.md) to ensure your contributions are legally recognized. For effective bug reporting, refer to the [Minimum Reproducible Example (MRE) Guide](../help/minimum_reproducible_example.md). + +### Why should I use Ultralytics HUB for my machine learning projects? + +Ultralytics HUB offers a seamless, no-code solution for managing your machine learning projects. It enables you to generate, train, and deploy AI models like YOLOv8 effortlessly. Unique features include cloud training, real-time tracking, and intuitive dataset management. Ultralytics HUB simplifies the entire workflow, from data processing to model deployment, making it an indispensable tool for both beginners and advanced users. + +To get started, visit [Ultralytics HUB Quickstart](../hub/quickstart.md). + +### What is Continuous Integration (CI) in Ultralytics, and how does it ensure high-quality code? + +Continuous Integration (CI) in Ultralytics involves automated processes that ensure the integrity and quality of the codebase. Our CI setup includes Docker deployment, broken link checks, CodeQL analysis, and PyPI publishing. These processes help maintain stable and secure repositories by automatically running tests and checks on new code submissions. + +Learn more in the [Continuous Integration (CI) Guide](../help/CI.md). + +### How is data privacy handled by Ultralytics? + +Ultralytics takes data privacy seriously. Our [Privacy Policy](../help/privacy.md) outlines how we collect and use anonymized data to improve the YOLO package while prioritizing user privacy and control. We adhere to strict data protection regulations to ensure your information is secure at all times. + +For more information, review our [Privacy Policy](../help/privacy.md). diff --git a/ultralytics/docs/en/help/minimum_reproducible_example.md b/ultralytics/docs/en/help/minimum_reproducible_example.md new file mode 100644 index 0000000000000000000000000000000000000000..693560be45894bf5bbe54543709a69c9d369f589 --- /dev/null +++ b/ultralytics/docs/en/help/minimum_reproducible_example.md @@ -0,0 +1,139 @@ +--- +comments: true +description: Learn how to create effective Minimum Reproducible Examples (MRE) for bug reports in Ultralytics YOLO repositories. Follow our guide for efficient issue resolution. +keywords: Ultralytics, YOLO, Minimum Reproducible Example, MRE, bug report, issue resolution, machine learning, deep learning +--- + +# Creating a Minimum Reproducible Example for Bug Reports in Ultralytics YOLO Repositories + +When submitting a bug report for [Ultralytics](https://ultralytics.com) [YOLO](https://github.com/ultralytics) repositories, it's essential to provide a [Minimum Reproducible Example (MRE)](https://stackoverflow.com/help/minimal-reproducible-example). An MRE is a small, self-contained piece of code that demonstrates the problem you're experiencing. Providing an MRE helps maintainers and contributors understand the issue and work on a fix more efficiently. This guide explains how to create an MRE when submitting bug reports to Ultralytics YOLO repositories. + +## 1. Isolate the Problem + +The first step in creating an MRE is to isolate the problem. Remove any unnecessary code or dependencies that are not directly related to the issue. Focus on the specific part of the code that is causing the problem and eliminate any irrelevant sections. + +## 2. Use Public Models and Datasets + +When creating an MRE, use publicly available models and datasets to reproduce the issue. For example, use the `yolov8n.pt` model and the `coco8.yaml` dataset. This ensures that the maintainers and contributors can easily run your example and investigate the problem without needing access to proprietary data or custom models. + +## 3. Include All Necessary Dependencies + +Ensure all necessary dependencies are included in your MRE. If your code relies on external libraries, specify the required packages and their versions. Ideally, list the dependencies in your bug report using `yolo checks` if you have `ultralytics` installed or `pip list` for other tools. + +## 4. Write a Clear Description of the Issue + +Provide a clear and concise description of the issue you're experiencing. Explain the expected behavior and the actual behavior you're encountering. If applicable, include any relevant error messages or logs. + +## 5. Format Your Code Properly + +Format your code properly using code blocks in the issue description. This makes it easier for others to read and understand your code. In GitHub, you can create a code block by wrapping your code with triple backticks (\```) and specifying the language: + +````bash +```python +# Your Python code goes here +``` +```` + +## 6. Test Your MRE + +Before submitting your MRE, test it to ensure that it accurately reproduces the issue. Make sure that others can run your example without any issues or modifications. + +## Example of an MRE + +Here's an example of an MRE for a hypothetical bug report: + +**Bug description:** + +When running inference on a 0-channel image, I get an error related to the dimensions of the input tensor. + +**MRE:** + +```python +import torch + +from ultralytics import YOLO + +# Load the model +model = YOLO("yolov8n.pt") + +# Load a 0-channel image +image = torch.rand(1, 0, 640, 640) + +# Run the model +results = model(image) +``` + +**Error message:** + +``` +RuntimeError: Expected input[1, 0, 640, 640] to have 3 channels, but got 0 channels instead +``` + +**Dependencies:** + +- `torch==2.3.0` +- `ultralytics==8.2.0` + +In this example, the MRE demonstrates the issue with a minimal amount of code, uses a public model (`"yolov8n.pt"`), includes all necessary dependencies, and provides a clear description of the problem along with the error message. + +By following these guidelines, you'll help the maintainers and [contributors](https://github.com/ultralytics/ultralytics/graphs/contributors) of Ultralytics YOLO repositories to understand and resolve your issue more efficiently. + +## FAQ + +### How do I create an effective Minimum Reproducible Example (MRE) for bug reports in Ultralytics YOLO repositories? + +To create an effective Minimum Reproducible Example (MRE) for bug reports in Ultralytics YOLO repositories, follow these steps: + +1. **Isolate the Problem**: Remove any code or dependencies that are not directly related to the issue. +2. **Use Public Models and Datasets**: Utilize public resources like `yolov8n.pt` and `coco8.yaml` for easier reproducibility. +3. **Include All Necessary Dependencies**: Specify required packages and their versions. You can list dependencies using `yolo checks` if you have `ultralytics` installed or `pip list`. +4. **Write a Clear Description of the Issue**: Explain the expected and actual behavior, including any error messages or logs. +5. **Format Your Code Properly**: Use code blocks to format your code, making it easier to read. +6. **Test Your MRE**: Ensure your MRE reproduces the issue without modifications. + +For a detailed guide, see [Creating a Minimum Reproducible Example](#creating-a-minimum-reproducible-example-for-bug-reports-in-ultralytics-yolo-repositories). + +### Why should I use publicly available models and datasets in my MRE for Ultralytics YOLO bug reports? + +Using publicly available models and datasets in your MRE ensures that maintainers can easily run your example without needing access to proprietary data. This allows for quicker and more efficient issue resolution. For instance, using the `yolov8n.pt` model and `coco8.yaml` dataset helps standardize and simplify the debugging process. Learn more about public models and datasets in the [Use Public Models and Datasets](#2-use-public-models-and-datasets) section. + +### What information should I include in my bug report for Ultralytics YOLO? + +A comprehensive bug report for Ultralytics YOLO should include: + +- **Clear Description**: Explain the issue, expected behavior, and actual behavior. +- **Error Messages**: Include any relevant error messages or logs. +- **Dependencies**: List required dependencies and their versions. +- **MRE**: Provide a Minimum Reproducible Example. +- **Steps to Reproduce**: Outline the steps needed to reproduce the issue. + +For a complete checklist, refer to the [Write a Clear Description of the Issue](#4-write-a-clear-description-of-the-issue) section. + +### How can I format my code properly when submitting a bug report on GitHub? + +To format your code properly when submitting a bug report on GitHub: + +- Use triple backticks (\```) to create code blocks. +- Specify the programming language for syntax highlighting, e.g., \```python. +- Ensure your code is indented correctly for readability. + +Example: + +````bash +```python +# Your Python code goes here +``` +```` + +For more tips on code formatting, see [Format Your Code Properly](#5-format-your-code-properly). + +### What are some common errors to check before submitting my MRE for a bug report? + +Before submitting your MRE, make sure to: + +- Verify the issue is reproducible. +- Ensure all dependencies are listed and correct. +- Remove any unnecessary code. +- Test the MRE to ensure it reproduces the issue without modifications. + +For a detailed checklist, visit the [Test Your MRE](#6-test-your-mre) section. diff --git a/ultralytics/docs/en/help/privacy.md b/ultralytics/docs/en/help/privacy.md new file mode 100644 index 0000000000000000000000000000000000000000..b3a6b3210babd18a37999189d298f6c11a90e198 --- /dev/null +++ b/ultralytics/docs/en/help/privacy.md @@ -0,0 +1,216 @@ +--- +description: Discover how Ultralytics collects and uses anonymized data to enhance the YOLO Python package while prioritizing user privacy and control. +keywords: Ultralytics, data collection, YOLO, Python package, Google Analytics, Sentry, privacy, anonymized data, user control, crash reporting +--- + +# Data Collection for Ultralytics Python Package + +## Overview + +[Ultralytics](https://ultralytics.com) is dedicated to the continuous enhancement of the user experience and the capabilities of our Python package, including the advanced YOLO models we develop. Our approach involves the gathering of anonymized usage statistics and crash reports, helping us identify opportunities for improvement and ensuring the reliability of our software. This transparency document outlines what data we collect, its purpose, and the choice you have regarding this data collection. + +## Anonymized Google Analytics + +[Google Analytics](https://developers.google.com/analytics) is a web analytics service offered by Google that tracks and reports website traffic. It allows us to collect data about how our Python package is used, which is crucial for making informed decisions about design and functionality. + +### What We Collect + +- **Usage Metrics**: These metrics help us understand how frequently and in what ways the package is utilized, what features are favored, and the typical command-line arguments that are used. +- **System Information**: We collect general non-identifiable information about your computing environment to ensure our package performs well across various systems. +- **Performance Data**: Understanding the performance of our models during training, validation, and inference helps us in identifying optimization opportunities. + +For more information about Google Analytics and data privacy, visit [Google Analytics Privacy](https://support.google.com/analytics/answer/6004245). + +### How We Use This Data + +- **Feature Improvement**: Insights from usage metrics guide us in enhancing user satisfaction and interface design. +- **Optimization**: Performance data assist us in fine-tuning our models for better efficiency and speed across diverse hardware and software configurations. +- **Trend Analysis**: By studying usage trends, we can predict and respond to the evolving needs of our community. + +### Privacy Considerations + +We take several measures to ensure the privacy and security of the data you entrust to us: + +- **Anonymization**: We configure Google Analytics to anonymize the data collected, which means no personally identifiable information (PII) is gathered. You can use our services with the assurance that your personal details remain private. +- **Aggregation**: Data is analyzed only in aggregate form. This practice ensures that patterns can be observed without revealing any individual user's activity. +- **No Image Data Collection**: Ultralytics does not collect, process, or view any training or inference images. + +## Sentry Crash Reporting + +[Sentry](https://sentry.io/) is a developer-centric error tracking software that aids in identifying, diagnosing, and resolving issues in real-time, ensuring the robustness and reliability of applications. Within our package, it plays a crucial role by providing insights through crash reporting, significantly contributing to the stability and ongoing refinement of our software. + +!!! Note + + Crash reporting via Sentry is activated only if the `sentry-sdk` Python package is pre-installed on your system. This package isn't included in the `ultralytics` prerequisites and won't be installed automatically by Ultralytics. + +### What We Collect + +If the `sentry-sdk` Python package is pre-installed on your system a crash event may send the following information: + +- **Crash Logs**: Detailed reports on the application's condition at the time of a crash, which are vital for our debugging efforts. +- **Error Messages**: We record error messages generated during the operation of our package to understand and resolve potential issues quickly. + +To learn more about how Sentry handles data, please visit [Sentry's Privacy Policy](https://sentry.io/privacy/). + +### How We Use This Data + +- **Debugging**: Analyzing crash logs and error messages enables us to swiftly identify and correct software bugs. +- **Stability Metrics**: By constantly monitoring for crashes, we aim to improve the stability and reliability of our package. + +### Privacy Considerations + +- **Sensitive Information**: We ensure that crash logs are scrubbed of any personally identifiable or sensitive user data, safeguarding the confidentiality of your information. +- **Controlled Collection**: Our crash reporting mechanism is meticulously calibrated to gather only what is essential for troubleshooting while respecting user privacy. + +By detailing the tools used for data collection and offering additional background information with URLs to their respective privacy pages, users are provided with a comprehensive view of our practices, emphasizing transparency and respect for user privacy. + +## Disabling Data Collection + +We believe in providing our users with full control over their data. By default, our package is configured to collect analytics and crash reports to help improve the experience for all users. However, we respect that some users may prefer to opt out of this data collection. + +To opt out of sending analytics and crash reports, you can simply set `sync=False` in your YOLO settings. This ensures that no data is transmitted from your machine to our analytics tools. + +### Inspecting Settings + +To gain insight into the current configuration of your settings, you can view them directly: + +!!! Example "View settings" + + === "Python" + + You can use Python to view your settings. Start by importing the `settings` object from the `ultralytics` module. Print and return settings using the following commands: + ```python + from ultralytics import settings + + # View all settings + print(settings) + + # Return analytics and crash reporting setting + value = settings["sync"] + ``` + + === "CLI" + + Alternatively, the command-line interface allows you to check your settings with a simple command: + ```bash + yolo settings + ``` + +### Modifying Settings + +Ultralytics allows users to easily modify their settings. Changes can be performed in the following ways: + +!!! Example "Update settings" + + === "Python" + + Within the Python environment, call the `update` method on the `settings` object to change your settings: + ```python + from ultralytics import settings + + # Disable analytics and crash reporting + settings.update({"sync": False}) + + # Reset settings to default values + settings.reset() + ``` + + === "CLI" + + If you prefer using the command-line interface, the following commands will allow you to modify your settings: + ```bash + # Disable analytics and crash reporting + yolo settings sync=False + + # Reset settings to default values + yolo settings reset + ``` + +The `sync=False` setting will prevent any data from being sent to Google Analytics or Sentry. Your settings will be respected across all sessions using the Ultralytics package and saved to disk for future sessions. + +## Commitment to Privacy + +Ultralytics takes user privacy seriously. We design our data collection practices with the following principles: + +- **Transparency**: We are open about the data we collect and how it is used. +- **Control**: We give users full control over their data. +- **Security**: We employ industry-standard security measures to protect the data we collect. + +## Questions or Concerns + +If you have any questions or concerns about our data collection practices, please reach out to us via our [contact form](https://ultralytics.com/contact) or via [support@ultralytics.com](mailto:support@ultralytics.com). We are dedicated to ensuring our users feel informed and confident in their privacy when using our package. + +## FAQ + +### How does Ultralytics ensure the privacy of the data it collects? + +Ultralytics prioritizes user privacy through several key measures. First, all data collected via Google Analytics and Sentry is anonymized to ensure that no personally identifiable information (PII) is gathered. Secondly, data is analyzed in aggregate form, allowing us to observe patterns without identifying individual user activities. Finally, we do not collect any training or inference images, further protecting user data. These measures align with our commitment to transparency and privacy. For more details, visit our [Privacy Considerations](#privacy-considerations) section. + +### What types of data does Ultralytics collect with Google Analytics? + +Ultralytics collects three primary types of data using Google Analytics: + +- **Usage Metrics**: These include how often and in what ways the YOLO Python package is used, preferred features, and typical command-line arguments. +- **System Information**: General non-identifiable information about the computing environments where the package is run. +- **Performance Data**: Metrics related to the performance of models during training, validation, and inference. + This data helps us enhance user experience and optimize software performance. Learn more in the [Anonymized Google Analytics](#anonymized-google-analytics) section. + +### How can I disable data collection in the Ultralytics YOLO package? + +To opt out of data collection, you can simply set `sync=False` in your YOLO settings. This action stops the transmission of any analytics or crash reports. You can disable data collection using Python or CLI methods: + +!!! Example "Update settings" + + === "Python" + + ```python + from ultralytics import settings + + # Disable analytics and crash reporting + settings.update({"sync": False}) + + # Reset settings to default values + settings.reset() + ``` + + === "CLI" + + ```bash + # Disable analytics and crash reporting + yolo settings sync=False + + # Reset settings to default values + yolo settings reset + ``` + +For more details on modifying your settings, refer to the [Modifying Settings](#modifying-settings) section. + +### How does crash reporting with Sentry work in Ultralytics YOLO? + +If the `sentry-sdk` package is pre-installed, Sentry collects detailed crash logs and error messages whenever a crash event occurs. This data helps us diagnose and resolve issues promptly, improving the robustness and reliability of the YOLO Python package. The collected crash logs are scrubbed of any personally identifiable information to protect user privacy. For more information, check the [Sentry Crash Reporting](#sentry-crash-reporting) section. + +### Can I inspect my current data collection settings in Ultralytics YOLO? + +Yes, you can easily view your current settings to understand the configuration of your data collection preferences. Use the following methods to inspect these settings: + +!!! Example "View settings" + + === "Python" + + ```python + from ultralytics import settings + + # View all settings + print(settings) + + # Return analytics and crash reporting setting + value = settings["sync"] + ``` + + === "CLI" + + ```bash + yolo settings + ``` + +For further details, refer to the [Inspecting Settings](#inspecting-settings) section. diff --git a/ultralytics/docs/en/help/security.md b/ultralytics/docs/en/help/security.md new file mode 100644 index 0000000000000000000000000000000000000000..517fedbc4b7c89ece6d8b39f5891ed91afbd6ffa --- /dev/null +++ b/ultralytics/docs/en/help/security.md @@ -0,0 +1,74 @@ +--- +description: Learn about the security measures and tools used by Ultralytics to protect user data and systems. Discover how we address vulnerabilities with Snyk, CodeQL, Dependabot, and more. +keywords: Ultralytics security policy, Snyk scanning, CodeQL scanning, Dependabot alerts, secret scanning, vulnerability reporting, GitHub security, open-source security +--- + +# Ultralytics Security Policy + +At [Ultralytics](https://ultralytics.com), the security of our users' data and systems is of utmost importance. To ensure the safety and security of our [open-source projects](https://github.com/ultralytics), we have implemented several measures to detect and prevent security vulnerabilities. + +## Snyk Scanning + +We utilize [Snyk](https://snyk.io/advisor/python/ultralytics) to conduct comprehensive security scans on Ultralytics repositories. Snyk's robust scanning capabilities extend beyond dependency checks; it also examines our code and Dockerfiles for various vulnerabilities. By identifying and addressing these issues proactively, we ensure a higher level of security and reliability for our users. + +[![ultralytics](https://snyk.io/advisor/python/ultralytics/badge.svg)](https://snyk.io/advisor/python/ultralytics) + +## GitHub CodeQL Scanning + +Our security strategy includes GitHub's [CodeQL](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning-with-codeql) scanning. CodeQL delves deep into our codebase, identifying complex vulnerabilities like SQL injection and XSS by analyzing the code's semantic structure. This advanced level of analysis ensures early detection and resolution of potential security risks. + +[![CodeQL](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml/badge.svg)](https://github.com/ultralytics/ultralytics/actions/workflows/codeql.yaml) + +## GitHub Dependabot Alerts + +[Dependabot](https://docs.github.com/en/code-security/dependabot) is integrated into our workflow to monitor dependencies for known vulnerabilities. When a vulnerability is identified in one of our dependencies, Dependabot alerts us, allowing for swift and informed remediation actions. + +## GitHub Secret Scanning Alerts + +We employ GitHub [secret scanning](https://docs.github.com/en/code-security/secret-scanning/managing-alerts-from-secret-scanning) alerts to detect sensitive data, such as credentials and private keys, accidentally pushed to our repositories. This early detection mechanism helps prevent potential security breaches and data exposures. + +## Private Vulnerability Reporting + +We enable private vulnerability reporting, allowing users to discreetly report potential security issues. This approach facilitates responsible disclosure, ensuring vulnerabilities are handled securely and efficiently. + +If you suspect or discover a security vulnerability in any of our repositories, please let us know immediately. You can reach out to us directly via our [contact form](https://ultralytics.com/contact) or via [security@ultralytics.com](mailto:security@ultralytics.com). Our security team will investigate and respond as soon as possible. + +We appreciate your help in keeping all Ultralytics open-source projects secure and safe for everyone 🙏. + +## FAQ + +### What are the security measures implemented by Ultralytics to protect user data? + +Ultralytics employs a comprehensive security strategy to protect user data and systems. Key measures include: + +- **Snyk Scanning**: Conducts security scans to detect vulnerabilities in code and Dockerfiles. +- **GitHub CodeQL**: Analyzes code semantics to detect complex vulnerabilities such as SQL injection. +- **Dependabot Alerts**: Monitors dependencies for known vulnerabilities and sends alerts for swift remediation. +- **Secret Scanning**: Detects sensitive data like credentials or private keys in code repositories to prevent data breaches. +- **Private Vulnerability Reporting**: Offers a secure channel for users to report potential security issues discreetly. + +These tools ensure proactive identification and resolution of security issues, enhancing overall system security. For more details, visit our [export documentation](../modes/export.md). + +### How does Ultralytics use Snyk for security scanning? + +Ultralytics utilizes [Snyk](https://snyk.io/advisor/python/ultralytics) to conduct thorough security scans on its repositories. Snyk extends beyond basic dependency checks, examining the code and Dockerfiles for various vulnerabilities. By proactively identifying and resolving potential security issues, Snyk helps ensure that Ultralytics' open-source projects remain secure and reliable. + +To see the Snyk badge and learn more about its deployment, check the [Snyk Scanning section](#snyk-scanning). + +### What is CodeQL and how does it enhance security for Ultralytics? + +[CodeQL](https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/about-code-scanning-with-codeql) is a security analysis tool integrated into Ultralytics' workflow via GitHub. It delves deep into the codebase to identify complex vulnerabilities such as SQL injection and Cross-Site Scripting (XSS). CodeQL analyzes the semantic structure of the code to provide an advanced level of security, ensuring early detection and mitigation of potential risks. + +For more information on how CodeQL is used, visit the [GitHub CodeQL Scanning section](#github-codeql-scanning). + +### How does Dependabot help maintain Ultralytics' code security? + +[Dependabot](https://docs.github.com/en/code-security/dependabot) is an automated tool that monitors and manages dependencies for known vulnerabilities. When Dependabot detects a vulnerability in an Ultralytics project dependency, it sends an alert, allowing the team to quickly address and mitigate the issue. This ensures that dependencies are kept secure and up-to-date, minimizing potential security risks. + +For more details, explore the [GitHub Dependabot Alerts section](#github-dependabot-alerts). + +### How does Ultralytics handle private vulnerability reporting? + +Ultralytics encourages users to report potential security issues through private channels. Users can report vulnerabilities discreetly via the [contact form](https://ultralytics.com/contact) or by emailing [security@ultralytics.com](mailto:security@ultralytics.com). This ensures responsible disclosure and allows the security team to investigate and address vulnerabilities securely and efficiently. + +For more information on private vulnerability reporting, refer to the [Private Vulnerability Reporting section](#private-vulnerability-reporting). diff --git a/ultralytics/docs/en/hub/api/index.md b/ultralytics/docs/en/hub/api/index.md new file mode 100644 index 0000000000000000000000000000000000000000..aaead37950fd3dbbcfc976f41f45faa7c7a52b70 --- /dev/null +++ b/ultralytics/docs/en/hub/api/index.md @@ -0,0 +1,34 @@ +--- +description: Discover what's next for Ultralytics with our under-construction page, previewing new, groundbreaking AI and ML features coming soon. +keywords: Ultralytics, coming soon, under construction, new features, AI updates, ML advancements, YOLO, technology preview +--- + +# Under Construction 🏗️🌟 + +Welcome to the Ultralytics "Under Construction" page! Here, we're hard at work developing the next generation of AI and ML innovations. This page serves as a teaser for the exciting updates and new features we're eager to share with you! + +## Exciting New Features on the Way 🎉 + +- **Innovative Breakthroughs:** Get ready for advanced features and services that will transform your AI and ML experience. +- **New Horizons:** Anticipate novel products that redefine AI and ML capabilities. +- **Enhanced Services:** We're upgrading our services for greater efficiency and user-friendliness. + +## Stay Updated 🚧 + +This placeholder page is your first stop for upcoming developments. Keep an eye out for: + +- **Newsletter:** Subscribe [here](https://ultralytics.com/#newsletter) for the latest news. +- **Social Media:** Follow us [here](https://www.linkedin.com/company/ultralytics) for updates and teasers. +- **Blog:** Visit our [blog](https://ultralytics.com/blog) for detailed insights. + +## We Value Your Input 🗣️ + +Your feedback shapes our future releases. Share your thoughts and suggestions [here](https://ultralytics.com/contact). + +## Thank You, Community! 🌍 + +Your [contributions](../../help/contributing.md) inspire our continuous [innovation](https://github.com/ultralytics/ultralytics). Stay tuned for the big reveal of what's next in AI and ML at Ultralytics! + +--- + +Excited for what's coming? Bookmark this page and get ready for a transformative AI and ML journey with Ultralytics! 🛠️🤖 diff --git a/ultralytics/docs/en/hub/app/android.md b/ultralytics/docs/en/hub/app/android.md new file mode 100644 index 0000000000000000000000000000000000000000..89517929e0f403c0d42f8484f50053b676797030 --- /dev/null +++ b/ultralytics/docs/en/hub/app/android.md @@ -0,0 +1,100 @@ +--- +comments: true +description: Experience real-time object detection on Android with Ultralytics. Leverage YOLO models for efficient and fast object identification. Download now!. +keywords: Ultralytics, Android app, real-time object detection, YOLO models, TensorFlow Lite, FP16 quantization, INT8 quantization, hardware delegates, mobile AI, download app +--- + +# Ultralytics Android App: Real-time Object Detection with YOLO Models + + + +
+
+
+
+ Watch: Getting Started with the Ultralytics HUB App (IOS & Android)
+
+
+
+
+ Watch: Getting Started with the Ultralytics HUB App (IOS & Android)
+
+
+
+ Watch: New Feature 🌟 Introducing Ultralytics HUB Cloud Training
+
+
+
+ Watch: Pause and Resume Model Training Using Ultralytics HUB
+
+
+
+ Watch: Watch: Upload Datasets to Ultralytics HUB | Complete Walkthrough of Dataset Upload Feature
+
+ +
+ +The dataset YAML is the same standard YOLOv5 and YOLOv8 YAML format. + +!!! Example "coco8.yaml" + + ```yaml + --8<-- "ultralytics/cfg/datasets/coco8.yaml" + ``` + +After zipping your dataset, you should [validate it](https://docs.ultralytics.com/reference/hub/__init__/#ultralytics.hub.check_dataset) before uploading it to [Ultralytics HUB](https://ultralytics.com/hub). [Ultralytics HUB](https://ultralytics.com/hub) conducts the dataset validation check post-upload, so by ensuring your dataset is correctly formatted and error-free ahead of time, you can forestall any setbacks due to dataset rejection. + +```py +from ultralytics.hub import check_dataset + +check_dataset("path/to/dataset.zip", task="detect") +``` + +Once your dataset ZIP is ready, navigate to the [Datasets](https://hub.ultralytics.com/datasets) page by clicking on the **Datasets** button in the sidebar and click on the **Upload Dataset** button on the top right of the page. + +![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to the Datasets button in the sidebar and one to the Upload Dataset button](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_2.jpg) + +??? tip "Tip" + + You can upload a dataset directly from the [Home](https://hub.ultralytics.com/home) page. + + ![Ultralytics HUB screenshot of the Home page with an arrow pointing to the Upload Dataset card](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_1.jpg) + +This action will trigger the **Upload Dataset** dialog. + +Select the dataset task of your dataset and upload it in the _Dataset .zip file_ field. + +You have the additional option to set a custom name and description for your [Ultralytics HUB](https://ultralytics.com/hub) dataset. + +When you're happy with your dataset configuration, click **Upload**. + +![Ultralytics HUB screenshot of the Upload Dataset dialog with arrows pointing to dataset task, dataset file and Upload button](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_3.jpg) + +After your dataset is uploaded and processed, you will be able to access it from the [Datasets](https://hub.ultralytics.com/datasets) page. + +![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_4.jpg) + +You can view the images in your dataset grouped by splits (Train, Validation, Test). + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Images tab](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_5.jpg) + +??? tip "Tip" + + Each image can be enlarged for better visualization. + + ![Ultralytics HUB screenshot of the Images tab inside the Dataset page with an arrow pointing to the expand icon](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_6.jpg) + + ![Ultralytics HUB screenshot of the Images tab inside the Dataset page with one of the images expanded](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_7.jpg) + +Also, you can analyze your dataset by click on the **Overview** tab. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Overview tab](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_8.jpg) + +Next, [train a model](./models.md#train-model) on your dataset. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Train Model button](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_upload_dataset_9.jpg) + +## Download Dataset + +Navigate to the Dataset page of the dataset you want to download, open the dataset actions dropdown and click on the **Download** option. This action will start downloading your dataset. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Download option](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_download_dataset_1.jpg) + +??? tip "Tip" + + You can download a dataset directly from the [Datasets](https://hub.ultralytics.com/datasets) page. + + ![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to the Download option of one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_download_dataset_2.jpg) + +## Share Dataset + +!!! info "Info" + + [Ultralytics HUB](https://ultralytics.com/hub)'s sharing functionality provides a convenient way to share datasets with others. This feature is designed to accommodate both existing [Ultralytics HUB](https://ultralytics.com/hub) users and those who have yet to create an account. + +!!! note "Note" + + You have control over the general access of your datasets. + + You can choose to set the general access to "Private", in which case, only you will have access to it. Alternatively, you can set the general access to "Unlisted" which grants viewing access to anyone who has the direct link to the dataset, regardless of whether they have an [Ultralytics HUB](https://ultralytics.com/hub) account or not. + +Navigate to the Dataset page of the dataset you want to share, open the dataset actions dropdown and click on the **Share** option. This action will trigger the **Share Dataset** dialog. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Share option](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_share_dataset_1.jpg) + +??? tip "Tip" + + You can share a dataset directly from the [Datasets](https://hub.ultralytics.com/datasets) page. + + ![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to the Share option of one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_share_dataset_2.jpg) + +Set the general access to "Unlisted" and click **Save**. + +![Ultralytics HUB screenshot of the Share Dataset dialog with an arrow pointing to the dropdown and one to the Save button](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_share_dataset_3.jpg) + +Now, anyone who has the direct link to your dataset can view it. + +??? tip "Tip" + + You can easily click on the dataset's link shown in the **Share Dataset** dialog to copy it. + + ![Ultralytics HUB screenshot of the Share Dataset dialog with an arrow pointing to the dataset's link](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_share_dataset_4.jpg) + +## Edit Dataset + +Navigate to the Dataset page of the dataset you want to edit, open the dataset actions dropdown and click on the **Edit** option. This action will trigger the **Update Dataset** dialog. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Edit option](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_edit_dataset_1.jpg) + +??? tip "Tip" + + You can edit a dataset directly from the [Datasets](https://hub.ultralytics.com/datasets) page. + + ![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to the Edit option of one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_edit_dataset_2.jpg) + +Apply the desired modifications to your dataset and then confirm the changes by clicking **Save**. + +![Ultralytics HUB screenshot of the Update Dataset dialog with an arrow pointing to the Save button](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_edit_dataset_3.jpg) + +## Delete Dataset + +Navigate to the Dataset page of the dataset you want to delete, open the dataset actions dropdown and click on the **Delete** option. This action will delete the dataset. + +![Ultralytics HUB screenshot of the Dataset page with an arrow pointing to the Delete option](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_delete_dataset_1.jpg) + +??? tip "Tip" + + You can delete a dataset directly from the [Datasets](https://hub.ultralytics.com/datasets) page. + + ![Ultralytics HUB screenshot of the Datasets page with an arrow pointing to the Delete option of one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_delete_dataset_2.jpg) + +!!! note "Note" + + If you change your mind, you can restore the dataset from the [Trash](https://hub.ultralytics.com/trash) page. + + ![Ultralytics HUB screenshot of the Trash page with an arrow pointing to Trash button in the sidebar and one to the Restore option of one of the datasets](https://raw.githubusercontent.com/ultralytics/assets/main/docs/hub/datasets/hub_delete_dataset_3.jpg) diff --git a/ultralytics/docs/en/hub/index.md b/ultralytics/docs/en/hub/index.md new file mode 100644 index 0000000000000000000000000000000000000000..ce7978236325fe7f775a1ebdbbb140877d3c1e72 --- /dev/null +++ b/ultralytics/docs/en/hub/index.md @@ -0,0 +1,126 @@ +--- +comments: true +description: Discover Ultralytics HUB, the all-in-one web tool for training and deploying YOLOv5 and YOLOv8 models. Get started quickly with pre-trained models and user-friendly features. +keywords: Ultralytics HUB, YOLO models, train YOLO, YOLOv5, YOLOv8, object detection, model deployment, machine learning, deep learning, AI tools, dataset upload, model training +--- + +# Ultralytics HUB + +
+
+
+ Watch: Train Your Custom YOLO Models In A Few Clicks with Ultralytics HUB
+
+
+
+ Watch: Ultralytics HUB Inference API Walkthrough
+
+
+
+ Watch: Ultralytics HUB Training and Validation Overview
+
+
+
+
+ Watch: How to Configure Ultralytics YOLOv8 Training Parameters in Ultralytics HUB
+
+
+
+ Watch: Bring your Own Agent model training using Ultralytics HUB
+
+
+
+ Watch: Train YOLOv8 Pose Model on Tiger-Pose Dataset Using Ultralytics HUB
+
+
+
+ Watch: Train Your Custom YOLO Models In A Few Clicks with Ultralytics HUB
+
+
+
+
+ Watch: How to Train a YOLOv8 model on Your Custom Dataset in Google Colab.
+
+ +
+ +[Amazon SageMaker](https://aws.amazon.com/sagemaker/) is a machine learning service from Amazon Web Services (AWS) that simplifies the process of building, training, and deploying machine learning models. It provides a broad range of tools for handling various aspects of machine learning workflows. This includes automated features for tuning models, options for training models at scale, and straightforward methods for deploying models into production. SageMaker supports popular machine learning frameworks, offering the flexibility needed for diverse projects. Its features also cover data labeling, workflow management, and performance analysis. + +## Deploying YOLOv8 on Amazon SageMaker Endpoints + +Deploying YOLOv8 on Amazon SageMaker lets you use its managed environment for real-time inference and take advantage of features like autoscaling. Take a look at the AWS architecture below. + ++ +
+ +### Step 1: Setup Your AWS Environment + +First, ensure you have the following prerequisites in place: + +- An AWS Account: If you don't already have one, sign up for an AWS account. + +- Configured IAM Roles: You'll need an IAM role with the necessary permissions for Amazon SageMaker, AWS CloudFormation, and Amazon S3. This role should have policies that allow it to access these services. + +- AWS CLI: If not already installed, download and install the AWS Command Line Interface (CLI) and configure it with your account details. Follow [the AWS CLI instructions](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) for installation. + +- AWS CDK: If not already installed, install the AWS Cloud Development Kit (CDK), which will be used for scripting the deployment. Follow [the AWS CDK instructions](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install) for installation. + +- Adequate Service Quota: Confirm that you have sufficient quotas for two separate resources in Amazon SageMaker: one for `ml.m5.4xlarge` for endpoint usage and another for `ml.m5.4xlarge` for notebook instance usage. Each of these requires a minimum of one quota value. If your current quotas are below this requirement, it's important to request an increase for each. You can request a quota increase by following the detailed instructions in the [AWS Service Quotas documentation](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html#quota-console-increase). + +### Step 2: Clone the YOLOv8 SageMaker Repository + +The next step is to clone the specific AWS repository that contains the resources for deploying YOLOv8 on SageMaker. This repository, hosted on GitHub, includes the necessary CDK scripts and configuration files. + +- Clone the GitHub Repository: Execute the following command in your terminal to clone the host-yolov8-on-sagemaker-endpoint repository: + +```bash +git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git +``` + +- Navigate to the Cloned Directory: Change your directory to the cloned repository: + +```bash +cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk +``` + +### Step 3: Set Up the CDK Environment + +Now that you have the necessary code, set up your environment for deploying with AWS CDK. + +- Create a Python Virtual Environment: This isolates your Python environment and dependencies. Run: + +```bash +python3 -m venv .venv +``` + +- Activate the Virtual Environment: + +```bash +source .venv/bin/activate +``` + +- Install Dependencies: Install the required Python dependencies for the project: + +```bash +pip3 install -r requirements.txt +``` + +- Upgrade AWS CDK Library: Ensure you have the latest version of the AWS CDK library: + +```bash +pip install --upgrade aws-cdk-lib +``` + +### Step 4: Create the AWS CloudFormation Stack + +- Synthesize the CDK Application: Generate the AWS CloudFormation template from your CDK code: + +```bash +cdk synth +``` + +- Bootstrap the CDK Application: Prepare your AWS environment for CDK deployment: + +```bash +cdk bootstrap +``` + +- Deploy the Stack: This will create the necessary AWS resources and deploy your model: + +```bash +cdk deploy +``` + +### Step 5: Deploy the YOLOv8 Model + +Before diving into the deployment instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +After creating the AWS CloudFormation Stack, the next step is to deploy YOLOv8. + +- Open the Notebook Instance: Go to the AWS Console and navigate to the Amazon SageMaker service. Select "Notebook Instances" from the dashboard, then locate the notebook instance that was created by your CDK deployment script. Open the notebook instance to access the Jupyter environment. + +- Access and Modify inference.py: After opening the SageMaker notebook instance in Jupyter, locate the inference.py file. Edit the output_fn function in inference.py as shown below and save your changes to the script, ensuring that there are no syntax errors. + +```python +import json + + +def output_fn(prediction_output): + """Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints.""" + print("Executing output_fn from inference.py ...") + infer = {} + for result in prediction_output: + if result.boxes is not None: + infer["boxes"] = result.boxes.numpy().data.tolist() + if result.masks is not None: + infer["masks"] = result.masks.numpy().data.tolist() + if result.keypoints is not None: + infer["keypoints"] = result.keypoints.numpy().data.tolist() + if result.obb is not None: + infer["obb"] = result.obb.numpy().data.tolist() + if result.probs is not None: + infer["probs"] = result.probs.numpy().data.tolist() + return json.dumps(infer) +``` + +- Deploy the Endpoint Using 1_DeployEndpoint.ipynb: In the Jupyter environment, open the 1_DeployEndpoint.ipynb notebook located in the sm-notebook directory. Follow the instructions in the notebook and run the cells to download the YOLOv8 model, package it with the updated inference code, and upload it to an Amazon S3 bucket. The notebook will guide you through creating and deploying a SageMaker endpoint for the YOLOv8 model. + +### Step 6: Testing Your Deployment + +Now that your YOLOv8 model is deployed, it's important to test its performance and functionality. + +- Open the Test Notebook: In the same Jupyter environment, locate and open the 2_TestEndpoint.ipynb notebook, also in the sm-notebook directory. + +- Run the Test Notebook: Follow the instructions within the notebook to test the deployed SageMaker endpoint. This includes sending an image to the endpoint and running inferences. Then, you'll plot the output to visualize the model's performance and accuracy, as shown below. + ++ +
+ +- Clean-Up Resources: The test notebook will also guide you through the process of cleaning up the endpoint and the hosted model. This is an important step to manage costs and resources effectively, especially if you do not plan to use the deployed model immediately. + +### Step 7: Monitoring and Management + +After testing, continuous monitoring and management of your deployed model are essential. + +- Monitor with Amazon CloudWatch: Regularly check the performance and health of your SageMaker endpoint using [Amazon CloudWatch](https://aws.amazon.com/cloudwatch/). + +- Manage the Endpoint: Use the SageMaker console for ongoing management of the endpoint. This includes scaling, updating, or redeploying the model as required. + +By completing these steps, you will have successfully deployed and tested a YOLOv8 model on Amazon SageMaker Endpoints. This process not only equips you with practical experience in using AWS services for machine learning deployment but also lays the foundation for deploying other advanced models in the future. + +## Summary + +This guide took you step by step through deploying YOLOv8 on Amazon SageMaker Endpoints using AWS CloudFormation and the AWS Cloud Development Kit (CDK). The process includes cloning the necessary GitHub repository, setting up the CDK environment, deploying the model using AWS services, and testing its performance on SageMaker. + +For more technical details, refer to [this article](https://aws.amazon.com/blogs/machine-learning/hosting-yolov8-pytorch-model-on-amazon-sagemaker-endpoints/) on the AWS Machine Learning Blog. You can also check out the official [Amazon SageMaker Documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) for more insights into various features and functionalities. + +Are you interested in learning more about different YOLOv8 integrations? Visit the [Ultralytics integrations guide page](../integrations/index.md) to discover additional tools and capabilities that can enhance your machine-learning projects. + +## FAQ + +### How do I deploy the Ultralytics YOLOv8 model on Amazon SageMaker Endpoints? + +To deploy the Ultralytics YOLOv8 model on Amazon SageMaker Endpoints, follow these steps: + +1. **Set Up Your AWS Environment**: Ensure you have an AWS Account, IAM roles with necessary permissions, and the AWS CLI configured. Install AWS CDK if not already done (refer to the [AWS CDK instructions](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install)). +2. **Clone the YOLOv8 SageMaker Repository**: + ```bash + git clone https://github.com/aws-samples/host-yolov8-on-sagemaker-endpoint.git + cd host-yolov8-on-sagemaker-endpoint/yolov8-pytorch-cdk + ``` +3. **Set Up the CDK Environment**: Create a Python virtual environment, activate it, install dependencies, and upgrade AWS CDK library. + ```bash + python3 -m venv .venv + source .venv/bin/activate + pip3 install -r requirements.txt + pip install --upgrade aws-cdk-lib + ``` +4. **Deploy using AWS CDK**: Synthesize and deploy the CloudFormation stack, bootstrap the environment. + ```bash + cdk synth + cdk bootstrap + cdk deploy + ``` + +For further details, review the [documentation section](#step-5-deploy-the-yolov8-model). + +### What are the prerequisites for deploying YOLOv8 on Amazon SageMaker? + +To deploy YOLOv8 on Amazon SageMaker, ensure you have the following prerequisites: + +1. **AWS Account**: Active AWS account ([sign up here](https://aws.amazon.com/)). +2. **IAM Roles**: Configured IAM roles with permissions for SageMaker, CloudFormation, and Amazon S3. +3. **AWS CLI**: Installed and configured AWS Command Line Interface ([AWS CLI installation guide](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)). +4. **AWS CDK**: Installed AWS Cloud Development Kit ([CDK setup guide](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install)). +5. **Service Quotas**: Sufficient quotas for `ml.m5.4xlarge` instances for both endpoint and notebook usage ([request a quota increase](https://docs.aws.amazon.com/servicequotas/latest/userguide/request-quota-increase.html#quota-console-increase)). + +For detailed setup, refer to [this section](#step-1-setup-your-aws-environment). + +### Why should I use Ultralytics YOLOv8 on Amazon SageMaker? + +Using Ultralytics YOLOv8 on Amazon SageMaker offers several advantages: + +1. **Scalability and Management**: SageMaker provides a managed environment with features like autoscaling, which helps in real-time inference needs. +2. **Integration with AWS Services**: Seamlessly integrate with other AWS services, such as S3 for data storage, CloudFormation for infrastructure as code, and CloudWatch for monitoring. +3. **Ease of Deployment**: Simplified setup using AWS CDK scripts and streamlined deployment processes. +4. **Performance**: Leverage Amazon SageMaker's high-performance infrastructure for running large scale inference tasks efficiently. + +Explore more about the advantages of using SageMaker in the [introduction section](#amazon-sagemaker). + +### Can I customize the inference logic for YOLOv8 on Amazon SageMaker? + +Yes, you can customize the inference logic for YOLOv8 on Amazon SageMaker: + +1. **Modify `inference.py`**: Locate and customize the `output_fn` function in the `inference.py` file to tailor output formats. + + ```python + import json + + + def output_fn(prediction_output): + """Formats model outputs as JSON string, extracting attributes like boxes, masks, keypoints.""" + infer = {} + for result in prediction_output: + if result.boxes is not None: + infer["boxes"] = result.boxes.numpy().data.tolist() + # Add more processing logic if necessary + return json.dumps(infer) + ``` + +2. **Deploy Updated Model**: Ensure you redeploy the model using Jupyter notebooks provided (`1_DeployEndpoint.ipynb`) to include these changes. + +Refer to the [detailed steps](#step-5-deploy-the-yolov8-model) for deploying the modified model. + +### How can I test the deployed YOLOv8 model on Amazon SageMaker? + +To test the deployed YOLOv8 model on Amazon SageMaker: + +1. **Open the Test Notebook**: Locate the `2_TestEndpoint.ipynb` notebook in the SageMaker Jupyter environment. +2. **Run the Notebook**: Follow the notebook's instructions to send an image to the endpoint, perform inference, and display results. +3. **Visualize Results**: Use built-in plotting functionalities to visualize performance metrics, such as bounding boxes around detected objects. + +For comprehensive testing instructions, visit the [testing section](#step-6-testing-your-deployment). diff --git a/ultralytics/docs/en/integrations/clearml.md b/ultralytics/docs/en/integrations/clearml.md new file mode 100644 index 0000000000000000000000000000000000000000..139bac3778e438cbf43c900de3b0eb912efb77ee --- /dev/null +++ b/ultralytics/docs/en/integrations/clearml.md @@ -0,0 +1,246 @@ +--- +comments: true +description: Discover how to integrate YOLOv8 with ClearML to streamline your MLOps workflow, automate experiments, and enhance model management effortlessly. +keywords: YOLOv8, ClearML, MLOps, Ultralytics, machine learning, object detection, model training, automation, experiment management +--- + +# Training YOLOv8 with ClearML: Streamlining Your MLOps Workflow + +MLOps bridges the gap between creating and deploying machine learning models in real-world settings. It focuses on efficient deployment, scalability, and ongoing management to ensure models perform well in practical applications. + +[Ultralytics YOLOv8](https://ultralytics.com) effortlessly integrates with ClearML, streamlining and enhancing your object detection model's training and management. This guide will walk you through the integration process, detailing how to set up ClearML, manage experiments, automate model management, and collaborate effectively. + +## ClearML + ++ +
+ +[ClearML](https://clear.ml/) is an innovative open-source MLOps platform that is skillfully designed to automate, monitor, and orchestrate machine learning workflows. Its key features include automated logging of all training and inference data for full experiment reproducibility, an intuitive web UI for easy data visualization and analysis, advanced hyperparameter optimization algorithms, and robust model management for efficient deployment across various platforms. + +## YOLOv8 Training with ClearML + +You can bring automation and efficiency to your machine learning workflow by improving your training process by integrating YOLOv8 with ClearML. + +## Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages for YOLOv8 and ClearML + pip install ultralytics clearml + ``` + +For detailed instructions and best practices related to the installation process, be sure to check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +## Configuring ClearML + +Once you have installed the necessary packages, the next step is to initialize and configure your ClearML SDK. This involves setting up your ClearML account and obtaining the necessary credentials for a seamless connection between your development environment and the ClearML server. + +Begin by initializing the ClearML SDK in your environment. The 'clearml-init' command starts the setup process and prompts you for the necessary credentials. + +!!! Tip "Initial SDK Setup" + + === "CLI" + + ```bash + # Initialize your ClearML SDK setup process + clearml-init + ``` + +After executing this command, visit the [ClearML Settings page](https://app.clear.ml/settings/workspace-configuration). Navigate to the top right corner and select "Settings." Go to the "Workspace" section and click on "Create new credentials." Use the credentials provided in the "Create Credentials" pop-up to complete the setup as instructed, depending on whether you are configuring ClearML in a Jupyter Notebook or a local Python environment. + +## Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage" + + === "Python" + + ```python + from clearml import Task + + from ultralytics import YOLO + + # Step 1: Creating a ClearML Task + task = Task.init(project_name="my_project", task_name="my_yolov8_task") + + # Step 2: Selecting the YOLOv8 Model + model_variant = "yolov8n" + task.set_parameter("model_variant", model_variant) + + # Step 3: Loading the YOLOv8 Model + model = YOLO(f"{model_variant}.pt") + + # Step 4: Setting Up Training Arguments + args = dict(data="coco8.yaml", epochs=16) + task.connect(args) + + # Step 5: Initiating Model Training + results = model.train(**args) + ``` + +### Understanding the Code + +Let's understand the steps showcased in the usage code snippet above. + +**Step 1: Creating a ClearML Task**: A new task is initialized in ClearML, specifying your project and task names. This task will track and manage your model's training. + +**Step 2: Selecting the YOLOv8 Model**: The `model_variant` variable is set to 'yolov8n', one of the YOLOv8 models. This variant is then logged in ClearML for tracking. + +**Step 3: Loading the YOLOv8 Model**: The selected YOLOv8 model is loaded using Ultralytics' YOLO class, preparing it for training. + +**Step 4: Setting Up Training Arguments**: Key training arguments like the dataset (`coco8.yaml`) and the number of epochs (`16`) are organized in a dictionary and connected to the ClearML task. This allows for tracking and potential modification via the ClearML UI. For a detailed understanding of the model training process and best practices, refer to our [YOLOv8 Model Training guide](../modes/train.md). + +**Step 5: Initiating Model Training**: The model training is started with the specified arguments. The results of the training process are captured in the `results` variable. + +### Understanding the Output + +Upon running the usage code snippet above, you can expect the following output: + +- A confirmation message indicating the creation of a new ClearML task, along with its unique ID. +- An informational message about the script code being stored, indicating that the code execution is being tracked by ClearML. +- A URL link to the ClearML results page where you can monitor the training progress and view detailed logs. +- Download progress for the YOLOv8 model and the specified dataset, followed by a summary of the model architecture and training configuration. +- Initialization messages for various training components like TensorBoard, Automatic Mixed Precision (AMP), and dataset preparation. +- Finally, the training process starts, with progress updates as the model trains on the specified dataset. For an in-depth understanding of the performance metrics used during training, read [our guide on performance metrics](../guides/yolo-performance-metrics.md). + +### Viewing the ClearML Results Page + +By clicking on the URL link to the ClearML results page in the output of the usage code snippet, you can access a comprehensive view of your model's training process. + +#### Key Features of the ClearML Results Page + +- **Real-Time Metrics Tracking** + + - Track critical metrics like loss, accuracy, and validation scores as they occur. + - Provides immediate feedback for timely model performance adjustments. + +- **Experiment Comparison** + + - Compare different training runs side-by-side. + - Essential for hyperparameter tuning and identifying the most effective models. + +- **Detailed Logs and Outputs** + + - Access comprehensive logs, graphical representations of metrics, and console outputs. + - Gain a deeper understanding of model behavior and issue resolution. + +- **Resource Utilization Monitoring** + + - Monitor the utilization of computational resources, including CPU, GPU, and memory. + - Key to optimizing training efficiency and costs. + +- **Model Artifacts Management** + + - View, download, and share model artifacts like trained models and checkpoints. + - Enhances collaboration and streamlines model deployment and sharing. + +For a visual walkthrough of what the ClearML Results Page looks like, watch the video below: + +
+
+
+
+ Watch: YOLOv8 MLOps Integration using ClearML
+
+
+
+ +
+ +[Comet ML](https://www.comet.ml/) is a platform for tracking, comparing, explaining, and optimizing machine learning models and experiments. It allows you to log metrics, parameters, media, and more during your model training and monitor your experiments through an aesthetically pleasing web interface. Comet ML helps data scientists iterate more rapidly, enhances transparency and reproducibility, and aids in the development of production models. + +## Harnessing the Power of YOLOv8 and Comet ML + +By combining Ultralytics YOLOv8 with Comet ML, you unlock a range of benefits. These include simplified experiment management, real-time insights for quick adjustments, flexible and tailored logging options, and the ability to log experiments offline when internet access is limited. This integration empowers you to make data-driven decisions, analyze performance metrics, and achieve exceptional results. + +## Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages for YOLOv8 and Comet ML + pip install ultralytics comet_ml torch torchvision + ``` + +## Configuring Comet ML + +After installing the required packages, you'll need to sign up, get a [Comet API Key](https://www.comet.com/signup), and configure it. + +!!! Tip "Configuring Comet ML" + + === "CLI" + + ```bash + # Set your Comet Api Key + export COMET_API_KEY=+ +
+ +**Metrics** + +In the metrics section, you have the option to examine the metrics in a tabular format as well, which is displayed in a dedicated pane as illustrated here. + ++ +
+ +**Interactive Confusion Matrix** + +The confusion matrix, found in the Confusion Matrix tab, provides an interactive way to assess the model's classification accuracy. It details the correct and incorrect predictions, allowing you to understand the model's strengths and weaknesses. + ++ +
+ +**System Metrics** + +Comet ML logs system metrics to help identify any bottlenecks in the training process. It includes metrics such as GPU utilization, GPU memory usage, CPU utilization, and RAM usage. These are essential for monitoring the efficiency of resource usage during model training. + ++ +
+ +## Customizing Comet ML Logging + +Comet ML offers the flexibility to customize its logging behavior by setting environment variables. These configurations allow you to tailor Comet ML to your specific needs and preferences. Here are some helpful customization options: + +### Logging Image Predictions + +You can control the number of image predictions that Comet ML logs during your experiments. By default, Comet ML logs 100 image predictions from the validation set. However, you can change this number to better suit your requirements. For example, to log 200 image predictions, use the following code: + +```python +import os + +os.environ["COMET_MAX_IMAGE_PREDICTIONS"] = "200" +``` + +### Batch Logging Interval + +Comet ML allows you to specify how often batches of image predictions are logged. The `COMET_EVAL_BATCH_LOGGING_INTERVAL` environment variable controls this frequency. The default setting is 1, which logs predictions from every validation batch. You can adjust this value to log predictions at a different interval. For instance, setting it to 4 will log predictions from every fourth batch. + +```python +import os + +os.environ["COMET_EVAL_BATCH_LOGGING_INTERVAL"] = "4" +``` + +### Disabling Confusion Matrix Logging + +In some cases, you may not want to log the confusion matrix from your validation set after every epoch. You can disable this feature by setting the `COMET_EVAL_LOG_CONFUSION_MATRIX` environment variable to "false." The confusion matrix will only be logged once, after the training is completed. + +```python +import os + +os.environ["COMET_EVAL_LOG_CONFUSION_MATRIX"] = "false" +``` + +### Offline Logging + +If you find yourself in a situation where internet access is limited, Comet ML provides an offline logging option. You can set the `COMET_MODE` environment variable to "offline" to enable this feature. Your experiment data will be saved locally in a directory that you can later upload to Comet ML when internet connectivity is available. + +```python +import os + +os.environ["COMET_MODE"] = "offline" +``` + +## Summary + +This guide has walked you through integrating Comet ML with Ultralytics' YOLOv8. From installation to customization, you've learned to streamline experiment management, gain real-time insights, and adapt logging to your project's needs. + +Explore [Comet ML's official documentation](https://www.comet.com/docs/v2/integrations/third-party-tools/yolov8/) for more insights on integrating with YOLOv8. + +Furthermore, if you're looking to dive deeper into the practical applications of YOLOv8, specifically for image segmentation tasks, this detailed guide on [fine-tuning YOLOv8 with Comet ML](https://www.comet.com/site/blog/fine-tuning-yolov8-for-image-segmentation-with-comet/) offers valuable insights and step-by-step instructions to enhance your model's performance. + +Additionally, to explore other exciting integrations with Ultralytics, check out the [integration guide page](../integrations/index.md), which offers a wealth of resources and information. + +## FAQ + +### How do I integrate Comet ML with Ultralytics YOLOv8 for training? + +To integrate Comet ML with Ultralytics YOLOv8, follow these steps: + +1. **Install the required packages**: + + ```bash + pip install ultralytics comet_ml torch torchvision + ``` + +2. **Set up your Comet API Key**: + + ```bash + export COMET_API_KEY=+ +
+ +[CoreML](https://developer.apple.com/documentation/coreml) is Apple's foundational machine learning framework that builds upon Accelerate, BNNS, and Metal Performance Shaders. It provides a machine-learning model format that seamlessly integrates into iOS applications and supports tasks such as image analysis, natural language processing, audio-to-text conversion, and sound analysis. + +Applications can take advantage of Core ML without the need to have a network connection or API calls because the Core ML framework works using on-device computing. This means model inference can be performed locally on the user's device. + +## Key Features of CoreML Models + +Apple's CoreML framework offers robust features for on-device machine learning. Here are the key features that make CoreML a powerful tool for developers: + +- **Comprehensive Model Support**: Converts and runs models from popular frameworks like TensorFlow, PyTorch, scikit-learn, XGBoost, and LibSVM. + ++ +
+ +- **On-device Machine Learning**: Ensures data privacy and swift processing by executing models directly on the user's device, eliminating the need for network connectivity. + +- **Performance and Optimization**: Uses the device's CPU, GPU, and Neural Engine for optimal performance with minimal power and memory usage. Offers tools for model compression and optimization while maintaining accuracy. + +- **Ease of Integration**: Provides a unified format for various model types and a user-friendly API for seamless integration into apps. Supports domain-specific tasks through frameworks like Vision and Natural Language. + +- **Advanced Features**: Includes on-device training capabilities for personalized experiences, asynchronous predictions for interactive ML experiences, and model inspection and validation tools. + +## CoreML Deployment Options + +Before we look at the code for exporting YOLOv8 models to the CoreML format, let's understand where CoreML models are usually used. + +CoreML offers various deployment options for machine learning models, including: + +- **On-Device Deployment**: This method directly integrates CoreML models into your iOS app. It's particularly advantageous for ensuring low latency, enhanced privacy (since data remains on the device), and offline functionality. This approach, however, may be limited by the device's hardware capabilities, especially for larger and more complex models. On-device deployment can be executed in the following two ways. + + - **Embedded Models**: These models are included in the app bundle and are immediately accessible. They are ideal for small models that do not require frequent updates. + + - **Downloaded Models**: These models are fetched from a server as needed. This approach is suitable for larger models or those needing regular updates. It helps keep the app bundle size smaller. + +- **Cloud-Based Deployment**: CoreML models are hosted on servers and accessed by the iOS app through API requests. This scalable and flexible option enables easy model updates without app revisions. It's ideal for complex models or large-scale apps requiring regular updates. However, it does require an internet connection and may pose latency and security issues. + +## Exporting YOLOv8 Models to CoreML + +Exporting YOLOv8 to CoreML enables optimized, on-device machine learning performance within Apple's ecosystem, offering benefits in terms of efficiency, security, and seamless integration with iOS, macOS, watchOS, and tvOS platforms. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to CoreML format + model.export(format="coreml") # creates 'yolov8n.mlpackage' + + # Load the exported CoreML model + coreml_model = YOLO("yolov8n.mlpackage") + + # Run inference + results = coreml_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to CoreML format + yolo export model=yolov8n.pt format=coreml # creates 'yolov8n.mlpackage'' + + # Run inference with the exported model + yolo predict model=yolov8n.mlpackage source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +## Deploying Exported YOLOv8 CoreML Models + +Having successfully exported your Ultralytics YOLOv8 models to CoreML, the next critical phase is deploying these models effectively. For detailed guidance on deploying CoreML models in various environments, check out these resources: + +- **[CoreML Tools](https://apple.github.io/coremltools/docs-guides/)**: This guide includes instructions and examples to convert models from TensorFlow, PyTorch, and other libraries to Core ML. + +- **[ML and Vision](https://developer.apple.com/videos/ml-vision)**: A collection of comprehensive videos that cover various aspects of using and implementing CoreML models. + +- **[Integrating a Core ML Model into Your App](https://developer.apple.com/documentation/coreml/integrating_a_core_ml_model_into_your_app)**: A comprehensive guide on integrating a CoreML model into an iOS application, detailing steps from preparing the model to implementing it in the app for various functionalities. + +## Summary + +In this guide, we went over how to export Ultralytics YOLOv8 models to CoreML format. By following the steps outlined in this guide, you can ensure maximum compatibility and performance when exporting YOLOv8 models to CoreML. + +For further details on usage, visit the [CoreML official documentation](https://developer.apple.com/documentation/coreml). + +Also, if you'd like to know more about other Ultralytics YOLOv8 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of valuable resources and insights there. + +## FAQ + +### How do I export YOLOv8 models to CoreML format? + +To export your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models to CoreML format, you'll first need to ensure you have the `ultralytics` package installed. You can install it using: + +!!! Example "Installation" + + === "CLI" + + ```bash + pip install ultralytics + ``` + +Next, you can export the model using the following Python or CLI commands: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + model = YOLO("yolov8n.pt") + model.export(format="coreml") + ``` + + === "CLI" + + ```bash + yolo export model=yolov8n.pt format=coreml + ``` + +For further details, refer to the [Exporting YOLOv8 Models to CoreML](../modes/export.md) section of our documentation. + +### What are the benefits of using CoreML for deploying YOLOv8 models? + +CoreML provides numerous advantages for deploying [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models on Apple devices: + +- **On-device Processing**: Enables local model inference on devices, ensuring data privacy and minimizing latency. +- **Performance Optimization**: Leverages the full potential of the device's CPU, GPU, and Neural Engine, optimizing both speed and efficiency. +- **Ease of Integration**: Offers a seamless integration experience with Apple's ecosystems, including iOS, macOS, watchOS, and tvOS. +- **Versatility**: Supports a wide range of machine learning tasks such as image analysis, audio processing, and natural language processing using the CoreML framework. + +For more details on integrating your CoreML model into an iOS app, check out the guide on [Integrating a Core ML Model into Your App](https://developer.apple.com/documentation/coreml/integrating_a_core_ml_model_into_your_app). + +### What are the deployment options for YOLOv8 models exported to CoreML? + +Once you export your YOLOv8 model to CoreML format, you have multiple deployment options: + +1. **On-Device Deployment**: Directly integrate CoreML models into your app for enhanced privacy and offline functionality. This can be done as: + + - **Embedded Models**: Included in the app bundle, accessible immediately. + - **Downloaded Models**: Fetched from a server as needed, keeping the app bundle size smaller. + +2. **Cloud-Based Deployment**: Host CoreML models on servers and access them via API requests. This approach supports easier updates and can handle more complex models. + +For detailed guidance on deploying CoreML models, refer to [CoreML Deployment Options](#coreml-deployment-options). + +### How does CoreML ensure optimized performance for YOLOv8 models? + +CoreML ensures optimized performance for [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models by utilizing various optimization techniques: + +- **Hardware Acceleration**: Uses the device's CPU, GPU, and Neural Engine for efficient computation. +- **Model Compression**: Provides tools for compressing models to reduce their footprint without compromising accuracy. +- **Adaptive Inference**: Adjusts inference based on the device's capabilities to maintain a balance between speed and performance. + +For more information on performance optimization, visit the [CoreML official documentation](https://developer.apple.com/documentation/coreml). + +### Can I run inference directly with the exported CoreML model? + +Yes, you can run inference directly using the exported CoreML model. Below are the commands for Python and CLI: + +!!! Example "Running Inference" + + === "Python" + + ```python + from ultralytics import YOLO + + coreml_model = YOLO("yolov8n.mlpackage") + results = coreml_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + yolo predict model=yolov8n.mlpackage source='https://ultralytics.com/images/bus.jpg' + ``` + +For additional information, refer to the [Usage section](#usage) of the CoreML export guide. diff --git a/ultralytics/docs/en/integrations/dvc.md b/ultralytics/docs/en/integrations/dvc.md new file mode 100644 index 0000000000000000000000000000000000000000..0810220f70bbb1bc272ed0b8be48d9caac31aca8 --- /dev/null +++ b/ultralytics/docs/en/integrations/dvc.md @@ -0,0 +1,278 @@ +--- +comments: true +description: Unlock seamless YOLOv8 tracking with DVCLive. Discover how to log, visualize, and analyze experiments for optimized ML model performance. +keywords: YOLOv8, DVCLive, experiment tracking, machine learning, model training, data visualization, Git integration +--- + +# Advanced YOLOv8 Experiment Tracking with DVCLive + +Experiment tracking in machine learning is critical to model development and evaluation. It involves recording and analyzing various parameters, metrics, and outcomes from numerous training runs. This process is essential for understanding model performance and making data-driven decisions to refine and optimize models. + +Integrating DVCLive with [Ultralytics YOLOv8](https://ultralytics.com) transforms the way experiments are tracked and managed. This integration offers a seamless solution for automatically logging key experiment details, comparing results across different runs, and visualizing data for in-depth analysis. In this guide, we'll understand how DVCLive can be used to streamline the process. + +## DVCLive + ++ +
+ +[DVCLive](https://dvc.org/doc/dvclive), developed by DVC, is an innovative open-source tool for experiment tracking in machine learning. Integrating seamlessly with Git and DVC, it automates the logging of crucial experiment data like model parameters and training metrics. Designed for simplicity, DVCLive enables effortless comparison and analysis of multiple runs, enhancing the efficiency of machine learning projects with intuitive data visualization and analysis tools. + +## YOLOv8 Training with DVCLive + +YOLOv8 training sessions can be effectively monitored with DVCLive. Additionally, DVC provides integral features for visualizing these experiments, including the generation of a report that enables the comparison of metric plots across all tracked experiments, offering a comprehensive view of the training process. + +## Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages for YOLOv8 and DVCLive + pip install ultralytics dvclive + ``` + +For detailed instructions and best practices related to the installation process, be sure to check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +## Configuring DVCLive + +Once you have installed the necessary packages, the next step is to set up and configure your environment with the necessary credentials. This setup ensures a smooth integration of DVCLive into your existing workflow. + +Begin by initializing a Git repository, as Git plays a crucial role in version control for both your code and DVCLive configurations. + +!!! Tip "Initial Environment Setup" + + === "CLI" + + ```bash + # Initialize a Git repository + git init -q + + # Configure Git with your details + git config --local user.email "you@example.com" + git config --local user.name "Your Name" + + # Initialize DVCLive in your project + dvc init -q + + # Commit the DVCLive setup to your Git repository + git commit -m "DVC init" + ``` + +In these commands, ensure to replace "you@example.com" with the email address associated with your Git account, and "Your Name" with your Git account username. + +## Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +### Training YOLOv8 Models with DVCLive + +Start by running your YOLOv8 training sessions. You can use different model configurations and training parameters to suit your project needs. For instance: + +```bash +# Example training commands for YOLOv8 with varying configurations +yolo train model=yolov8n.pt data=coco8.yaml epochs=5 imgsz=512 +yolo train model=yolov8n.pt data=coco8.yaml epochs=5 imgsz=640 +``` + +Adjust the model, data, epochs, and imgsz parameters according to your specific requirements. For a detailed understanding of the model training process and best practices, refer to our [YOLOv8 Model Training guide](../modes/train.md). + +### Monitoring Experiments with DVCLive + +DVCLive enhances the training process by enabling the tracking and visualization of key metrics. When installed, Ultralytics YOLOv8 automatically integrates with DVCLive for experiment tracking, which you can later analyze for performance insights. For a comprehensive understanding of the specific performance metrics used during training, be sure to explore [our detailed guide on performance metrics](../guides/yolo-performance-metrics.md). + +### Analyzing Results + +After your YOLOv8 training sessions are complete, you can leverage DVCLive's powerful visualization tools for in-depth analysis of the results. DVCLive's integration ensures that all training metrics are systematically logged, facilitating a comprehensive evaluation of your model's performance. + +To start the analysis, you can extract the experiment data using DVC's API and process it with Pandas for easier handling and visualization: + +```python +import dvc.api +import pandas as pd + +# Define the columns of interest +columns = ["Experiment", "epochs", "imgsz", "model", "metrics.mAP50-95(B)"] + +# Retrieve experiment data +df = pd.DataFrame(dvc.api.exp_show(), columns=columns) + +# Clean the data +df.dropna(inplace=True) +df.reset_index(drop=True, inplace=True) + +# Display the DataFrame +print(df) +``` + +The output of the code snippet above provides a clear tabular view of the different experiments conducted with YOLOv8 models. Each row represents a different training run, detailing the experiment's name, the number of epochs, image size (imgsz), the specific model used, and the mAP50-95(B) metric. This metric is crucial for evaluating the model's accuracy, with higher values indicating better performance. + +#### Visualizing Results with Plotly + +For a more interactive and visual analysis of your experiment results, you can use Plotly's parallel coordinates plot. This type of plot is particularly useful for understanding the relationships and trade-offs between different parameters and metrics. + +```python +from plotly.express import parallel_coordinates + +# Create a parallel coordinates plot +fig = parallel_coordinates(df, columns, color="metrics.mAP50-95(B)") + +# Display the plot +fig.show() +``` + +The output of the code snippet above generates a plot that will visually represent the relationships between epochs, image size, model type, and their corresponding mAP50-95(B) scores, enabling you to spot trends and patterns in your experiment data. + +#### Generating Comparative Visualizations with DVC + +DVC provides a useful command to generate comparative plots for your experiments. This can be especially helpful to compare the performance of different models over various training runs. + +```bash +# Generate DVC comparative plots +dvc plots diff $(dvc exp list --names-only) +``` + +After executing this command, DVC generates plots comparing the metrics across different experiments, which are saved as HTML files. Below is an example image illustrating typical plots generated by this process. The image showcases various graphs, including those representing mAP, recall, precision, loss values, and more, providing a visual overview of key performance metrics: + ++ +
+ +### Displaying DVC Plots + +If you are using a Jupyter Notebook and you want to display the generated DVC plots, you can use the IPython display functionality. + +```python +from IPython.display import HTML + +# Display the DVC plots as HTML +HTML(filename="./dvc_plots/index.html") +``` + +This code will render the HTML file containing the DVC plots directly in your Jupyter Notebook, providing an easy and convenient way to analyze the visualized experiment data. + +### Making Data-Driven Decisions + +Use the insights gained from these visualizations to make informed decisions about model optimizations, hyperparameter tuning, and other modifications to enhance your model's performance. + +### Iterating on Experiments + +Based on your analysis, iterate on your experiments. Adjust model configurations, training parameters, or even the data inputs, and repeat the training and analysis process. This iterative approach is key to refining your model for the best possible performance. + +## Summary + +This guide has led you through the process of integrating DVCLive with Ultralytics' YOLOv8. You have learned how to harness the power of DVCLive for detailed experiment monitoring, effective visualization, and insightful analysis in your machine learning endeavors. + +For further details on usage, visit [DVCLive's official documentation](https://dvc.org/doc/dvclive/ml-frameworks/yolo). + +Additionally, explore more integrations and capabilities of Ultralytics by visiting the [Ultralytics integration guide page](../integrations/index.md), which is a collection of great resources and insights. + +## FAQ + +### How do I integrate DVCLive with Ultralytics YOLOv8 for experiment tracking? + +Integrating DVCLive with Ultralytics YOLOv8 is straightforward. Start by installing the necessary packages: + +!!! Example "Installation" + + === "CLI" + + ```bash + pip install ultralytics dvclive + ``` + +Next, initialize a Git repository and configure DVCLive in your project: + +!!! Example "Initial Environment Setup" + + === "CLI" + + ```bash + git init -q + git config --local user.email "you@example.com" + git config --local user.name "Your Name" + dvc init -q + git commit -m "DVC init" + ``` + +Follow our [YOLOv8 Installation guide](../quickstart.md) for detailed setup instructions. + +### Why should I use DVCLive for tracking YOLOv8 experiments? + +Using DVCLive with YOLOv8 provides several advantages, such as: + +- **Automated Logging**: DVCLive automatically records key experiment details like model parameters and metrics. +- **Easy Comparison**: Facilitates comparison of results across different runs. +- **Visualization Tools**: Leverages DVCLive's robust data visualization capabilities for in-depth analysis. + +For further details, refer to our guide on [YOLOv8 Model Training](../modes/train.md) and [YOLO Performance Metrics](../guides/yolo-performance-metrics.md) to maximize your experiment tracking efficiency. + +### How can DVCLive improve my results analysis for YOLOv8 training sessions? + +After completing your YOLOv8 training sessions, DVCLive helps in visualizing and analyzing the results effectively. Example code for loading and displaying experiment data: + +```python +import dvc.api +import pandas as pd + +# Define columns of interest +columns = ["Experiment", "epochs", "imgsz", "model", "metrics.mAP50-95(B)"] + +# Retrieve experiment data +df = pd.DataFrame(dvc.api.exp_show(), columns=columns) + +# Clean data +df.dropna(inplace=True) +df.reset_index(drop=True, inplace=True) + +# Display DataFrame +print(df) +``` + +To visualize results interactively, use Plotly's parallel coordinates plot: + +```python +from plotly.express import parallel_coordinates + +fig = parallel_coordinates(df, columns, color="metrics.mAP50-95(B)") +fig.show() +``` + +Refer to our guide on [YOLOv8 Training with DVCLive](#yolov8-training-with-dvclive) for more examples and best practices. + +### What are the steps to configure my environment for DVCLive and YOLOv8 integration? + +To configure your environment for a smooth integration of DVCLive and YOLOv8, follow these steps: + +1. **Install Required Packages**: Use `pip install ultralytics dvclive`. +2. **Initialize Git Repository**: Run `git init -q`. +3. **Setup DVCLive**: Execute `dvc init -q`. +4. **Commit to Git**: Use `git commit -m "DVC init"`. + +These steps ensure proper version control and setup for experiment tracking. For in-depth configuration details, visit our [Configuration guide](../quickstart.md). + +### How do I visualize YOLOv8 experiment results using DVCLive? + +DVCLive offers powerful tools to visualize the results of YOLOv8 experiments. Here's how you can generate comparative plots: + +!!! Example "Generate Comparative Plots" + + === "CLI" + + ```bash + dvc plots diff $(dvc exp list --names-only) + ``` + +To display these plots in a Jupyter Notebook, use: + +```python +from IPython.display import HTML + +# Display plots as HTML +HTML(filename="./dvc_plots/index.html") +``` + +These visualizations help identify trends and optimize model performance. Check our detailed guides on [YOLOv8 Experiment Analysis](#analyzing-results) for comprehensive steps and examples. diff --git a/ultralytics/docs/en/integrations/edge-tpu.md b/ultralytics/docs/en/integrations/edge-tpu.md new file mode 100644 index 0000000000000000000000000000000000000000..ef4dc84689aea997dd6cfb252d15fafdf5e1f859 --- /dev/null +++ b/ultralytics/docs/en/integrations/edge-tpu.md @@ -0,0 +1,185 @@ +--- +comments: true +description: Learn how to export YOLOv8 models to TFLite Edge TPU format for high-speed, low-power inferencing on mobile and embedded devices. +keywords: YOLOv8, TFLite Edge TPU, TensorFlow Lite, model export, machine learning, edge computing, neural networks, Ultralytics +--- + +# Learn to Export to TFLite Edge TPU Format From YOLOv8 Model + +Deploying computer vision models on devices with limited computational power, such as mobile or embedded systems, can be tricky. Using a model format that is optimized for faster performance simplifies the process. The [TensorFlow Lite](https://www.tensorflow.org/lite) [Edge TPU](https://coral.ai/docs/edgetpu/models-intro/) or TFLite Edge TPU model format is designed to use minimal power while delivering fast performance for neural networks. + +The export to TFLite Edge TPU format feature allows you to optimize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for high-speed and low-power inferencing. In this guide, we'll walk you through converting your models to the TFLite Edge TPU format, making it easier for your models to perform well on various mobile and embedded devices. + +## Why Should You Export to TFLite Edge TPU? + +Exporting models to TensorFlow Edge TPU makes machine learning tasks fast and efficient. This technology suits applications with limited power, computing resources, and connectivity. The Edge TPU is a hardware accelerator by Google. It speeds up TensorFlow Lite models on edge devices. The image below shows an example of the process involved. + ++ +
+ +The Edge TPU works with quantized models. Quantization makes models smaller and faster without losing much accuracy. It is ideal for the limited resources of edge computing, allowing applications to respond quickly by reducing latency and allowing for quick data processing locally, without cloud dependency. Local processing also keeps user data private and secure since it's not sent to a remote server. + +## Key Features of TFLite Edge TPU + +Here are the key features that make TFLite Edge TPU a great model format choice for developers: + +- **Optimized Performance on Edge Devices**: The TFLite Edge TPU achieves high-speed neural networking performance through quantization, model optimization, hardware acceleration, and compiler optimization. Its minimalistic architecture contributes to its smaller size and cost-efficiency. + +- **High Computational Throughput**: TFLite Edge TPU combines specialized hardware acceleration and efficient runtime execution to achieve high computational throughput. It is well-suited for deploying machine learning models with stringent performance requirements on edge devices. + +- **Efficient Matrix Computations**: The TensorFlow Edge TPU is optimized for matrix operations, which are crucial for neural network computations. This efficiency is key in machine learning models, particularly those requiring numerous and complex matrix multiplications and transformations. + +## Deployment Options with TFLite Edge TPU + +Before we jump into how to export YOLOv8 models to the TFLite Edge TPU format, let's understand where TFLite Edge TPU models are usually used. + +TFLite Edge TPU offers various deployment options for machine learning models, including: + +- **On-Device Deployment**: TensorFlow Edge TPU models can be directly deployed on mobile and embedded devices. On-device deployment allows the models to execute directly on the hardware, eliminating the need for cloud connectivity. + +- **Edge Computing with Cloud TensorFlow TPUs**: In scenarios where edge devices have limited processing capabilities, TensorFlow Edge TPUs can offload inference tasks to cloud servers equipped with TPUs. + +- **Hybrid Deployment**: A hybrid approach combines on-device and cloud deployment and offers a versatile and scalable solution for deploying machine learning models. Advantages include on-device processing for quick responses and cloud computing for more complex computations. + +## Exporting YOLOv8 Models to TFLite Edge TPU + +You can expand model compatibility and deployment flexibility by converting YOLOv8 models to TensorFlow Edge TPU. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TFLite Edge TPU format + model.export(format="edgetpu") # creates 'yolov8n_full_integer_quant_edgetpu.tflite' + + # Load the exported TFLite Edge TPU model + edgetpu_model = YOLO("yolov8n_full_integer_quant_edgetpu.tflite") + + # Run inference + results = edgetpu_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TFLite Edge TPU format + yolo export model=yolov8n.pt format=edgetpu # creates 'yolov8n_full_integer_quant_edgetpu.tflite' + + # Run inference with the exported model + yolo predict model=yolov8n_full_integer_quant_edgetpu.tflite source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 TFLite Edge TPU Models + +After successfully exporting your Ultralytics YOLOv8 models to TFLite Edge TPU format, you can now deploy them. The primary and recommended first step for running a TFLite Edge TPU model is to use the YOLO("model_edgetpu.tflite") method, as outlined in the previous usage code snippet. + +However, for in-depth instructions on deploying your TFLite Edge TPU models, take a look at the following resources: + +- **[Coral Edge TPU on a Raspberry Pi with Ultralytics YOLOv8](../guides/coral-edge-tpu-on-raspberry-pi.md)**: Discover how to integrate Coral Edge TPUs with Raspberry Pi for enhanced machine learning capabilities. + +- **[Code Examples](https://coral.ai/docs/edgetpu/compiler/)**: Access practical TensorFlow Edge TPU deployment examples to kickstart your projects. + +- **[Run Inference on the Edge TPU with Python](https://coral.ai/docs/edgetpu/tflite-python/#overview)**: Explore how to use the TensorFlow Lite Python API for Edge TPU applications, including setup and usage guidelines. + +## Summary + +In this guide, we've learned how to export Ultralytics YOLOv8 models to TFLite Edge TPU format. By following the steps mentioned above, you can increase the speed and power of your computer vision applications. + +For further details on usage, visit the [Edge TPU official website](https://cloud.google.com/edge-tpu). + +Also, for more information on other Ultralytics YOLOv8 integrations, please visit our [integration guide page](index.md). There, you'll discover valuable resources and insights. + +## FAQ + +### How do I export a YOLOv8 model to TFLite Edge TPU format? + +To export a YOLOv8 model to TFLite Edge TPU format, you can follow these steps: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TFLite Edge TPU format + model.export(format="edgetpu") # creates 'yolov8n_full_integer_quant_edgetpu.tflite' + + # Load the exported TFLite Edge TPU model + edgetpu_model = YOLO("yolov8n_full_integer_quant_edgetpu.tflite") + + # Run inference + results = edgetpu_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TFLite Edge TPU format + yolo export model=yolov8n.pt format=edgetpu # creates 'yolov8n_full_integer_quant_edgetpu.tflite' + + # Run inference with the exported model + yolo predict model=yolov8n_full_integer_quant_edgetpu.tflite source='https://ultralytics.com/images/bus.jpg' + ``` + +For complete details on exporting models to other formats, refer to our [export guide](../modes/export.md). + +### What are the benefits of exporting YOLOv8 models to TFLite Edge TPU? + +Exporting YOLOv8 models to TFLite Edge TPU offers several benefits: + +- **Optimized Performance**: Achieve high-speed neural network performance with minimal power consumption. +- **Reduced Latency**: Quick local data processing without the need for cloud dependency. +- **Enhanced Privacy**: Local processing keeps user data private and secure. + +This makes it ideal for applications in edge computing, where devices have limited power and computational resources. Learn more about [why you should export](#why-should-you-export-to-tflite-edge-tpu). + +### Can I deploy TFLite Edge TPU models on mobile and embedded devices? + +Yes, TensorFlow Lite Edge TPU models can be deployed directly on mobile and embedded devices. This deployment approach allows models to execute directly on the hardware, offering faster and more efficient inferencing. For integration examples, check our [guide on deploying Coral Edge TPU on Raspberry Pi](../guides/coral-edge-tpu-on-raspberry-pi.md). + +### What are some common use cases for TFLite Edge TPU models? + +Common use cases for TFLite Edge TPU models include: + +- **Smart Cameras**: Enhancing real-time image and video analysis. +- **IoT Devices**: Enabling smart home and industrial automation. +- **Healthcare**: Accelerating medical imaging and diagnostics. +- **Retail**: Improving inventory management and customer behavior analysis. + +These applications benefit from the high performance and low power consumption of TFLite Edge TPU models. Discover more about [usage scenarios](#deployment-options-with-tflite-edge-tpu). + +### How can I troubleshoot issues while exporting or deploying TFLite Edge TPU models? + +If you encounter issues while exporting or deploying TFLite Edge TPU models, refer to our [Common Issues guide](../guides/yolo-common-issues.md) for troubleshooting tips. This guide covers common problems and solutions to help you ensure smooth operation. For additional support, visit our [Help Center](https://docs.ultralytics.com/help/). diff --git a/ultralytics/docs/en/integrations/google-colab.md b/ultralytics/docs/en/integrations/google-colab.md new file mode 100644 index 0000000000000000000000000000000000000000..3f0919bcef9c65fc77f9ed965cc33c1d6ac5ecc8 --- /dev/null +++ b/ultralytics/docs/en/integrations/google-colab.md @@ -0,0 +1,151 @@ +--- +comments: true +description: Learn how to efficiently train Ultralytics YOLOv8 models using Google Colab's powerful cloud-based environment. Start your project with ease. +keywords: YOLOv8, Google Colab, machine learning, deep learning, model training, GPU, TPU, cloud computing, Jupyter Notebook, Ultralytics +--- + +# Accelerating YOLOv8 Projects with Google Colab + +Many developers lack the powerful computing resources needed to build deep learning models. Acquiring high-end hardware or renting a decent GPU can be expensive. Google Colab is a great solution to this. It's a browser-based platform that allows you to work with large datasets, develop complex models, and share your work with others without a huge cost. + +You can use Google Colab to work on projects related to [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models. Google Colab's user-friendly environment is well suited for efficient model development and experimentation. Let's learn more about Google Colab, its key features, and how you can use it to train YOLOv8 models. + +## Google Colaboratory + +Google Colaboratory, commonly known as Google Colab, was developed by Google Research in 2017. It is a free online cloud-based Jupyter Notebook environment that allows you to train your machine learning and deep learning models on CPUs, GPUs, and TPUs. The motivation behind developing Google Colab was Google's broader goals to advance AI technology and educational tools, and encourage the use of cloud services. + +You can use Google Colab regardless of the specifications and configurations of your local computer. All you need is a Google account and a web browser, and you're good to go. + +## Training YOLOv8 Using Google Colaboratory + +Training YOLOv8 models on Google Colab is pretty straightforward. Thanks to the integration, you can access the [Google Colab YOLOv8 Notebook](https://colab.research.google.com/github/ultralytics/ultralytics/blob/main/examples/tutorial.ipynb) and start training your model immediately. For a detailed understanding of the model training process and best practices, refer to our [YOLOv8 Model Training guide](../modes/train.md). + +Sign in to your Google account and run the notebook's cells to train your model. + +![Training YOLOv8 Using Google Colab](https://github.com/ultralytics/ultralytics/assets/25847604/71779530-619e-4518-aced-5d56e1e8eab3) + +Learn how to train a YOLOv8 model with custom data on YouTube with Nicolai. Check out the guide below. + +
+
+
+
+ Watch: How to Train Ultralytics YOLOv8 models on Your Custom Dataset in Google Colab | Episode 3
+
+
+
+
+ Watch: Gradio Integration with Ultralytics YOLOv8
+
+ +
+ +## How to Install the Gradio + +```bash +pip install gradio +``` + +## How to Use the Interface + +1. **Upload Image:** Click on 'Upload Image' to choose an image file for object detection. +2. **Adjust Parameters:** + - **Confidence Threshold:** Slider to set the minimum confidence level for detecting objects. + - **IoU Threshold:** Slider to set the IoU threshold for distinguishing different objects. +3. **View Results:** The processed image with detected objects and their labels will be displayed. + +## Example Use Cases + +- **Sample Image 1:** Bus detection with default thresholds. +- **Sample Image 2:** Detection on a sports image with default thresholds. + +## Usage Example + +This section provides the Python code used to create the Gradio interface with the Ultralytics YOLOv8 model. Supports classification tasks, detection tasks, segmentation tasks, and key point tasks. + +```python +import gradio as gr +import PIL.Image as Image + +from ultralytics import ASSETS, YOLO + +model = YOLO("yolov8n.pt") + + +def predict_image(img, conf_threshold, iou_threshold): + """Predicts objects in an image using a YOLOv8 model with adjustable confidence and IOU thresholds.""" + results = model.predict( + source=img, + conf=conf_threshold, + iou=iou_threshold, + show_labels=True, + show_conf=True, + imgsz=640, + ) + + for r in results: + im_array = r.plot() + im = Image.fromarray(im_array[..., ::-1]) + + return im + + +iface = gr.Interface( + fn=predict_image, + inputs=[ + gr.Image(type="pil", label="Upload Image"), + gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), + gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), + ], + outputs=gr.Image(type="pil", label="Result"), + title="Ultralytics Gradio", + description="Upload images for inference. The Ultralytics YOLOv8n model is used by default.", + examples=[ + [ASSETS / "bus.jpg", 0.25, 0.45], + [ASSETS / "zidane.jpg", 0.25, 0.45], + ], +) + +if __name__ == "__main__": + iface.launch() +``` + +## Parameters Explanation + +| Parameter Name | Type | Description | +| ---------------- | ------- | -------------------------------------------------------- | +| `img` | `Image` | The image on which object detection will be performed. | +| `conf_threshold` | `float` | Confidence threshold for detecting objects. | +| `iou_threshold` | `float` | Intersection-over-union threshold for object separation. | + +### Gradio Interface Components + +| Component | Description | +| ------------ | ---------------------------------------- | +| Image Input | To upload the image for detection. | +| Sliders | To adjust confidence and IoU thresholds. | +| Image Output | To display the detection results. | + +## FAQ + +### How do I use Gradio with Ultralytics YOLOv8 for object detection? + +To use Gradio with Ultralytics YOLOv8 for object detection, you can follow these steps: + +1. **Install Gradio:** Use the command `pip install gradio`. +2. **Create Interface:** Write a Python script to initialize the Gradio interface. You can refer to the provided code example in the [documentation](#usage-example) for details. +3. **Upload and Adjust:** Upload your image and adjust the confidence and IoU thresholds on the Gradio interface to get real-time object detection results. + +Here's a minimal code snippet for reference: + +```python +import gradio as gr + +from ultralytics import YOLO + +model = YOLO("yolov8n.pt") + + +def predict_image(img, conf_threshold, iou_threshold): + results = model.predict( + source=img, + conf=conf_threshold, + iou=iou_threshold, + show_labels=True, + show_conf=True, + ) + return results[0].plot() if results else None + + +iface = gr.Interface( + fn=predict_image, + inputs=[ + gr.Image(type="pil", label="Upload Image"), + gr.Slider(minimum=0, maximum=1, value=0.25, label="Confidence threshold"), + gr.Slider(minimum=0, maximum=1, value=0.45, label="IoU threshold"), + ], + outputs=gr.Image(type="pil", label="Result"), + title="Ultralytics Gradio YOLOv8", + description="Upload images for YOLOv8 object detection.", +) +iface.launch() +``` + +### What are the benefits of using Gradio for Ultralytics YOLOv8 object detection? + +Using Gradio for Ultralytics YOLOv8 object detection offers several benefits: + +- **User-Friendly Interface:** Gradio provides an intuitive interface for users to upload images and visualize detection results without any coding effort. +- **Real-Time Adjustments:** You can dynamically adjust detection parameters such as confidence and IoU thresholds and see the effects immediately. +- **Accessibility:** The web interface is accessible to anyone, making it useful for quick experiments, educational purposes, and demonstrations. + +For more details, you can read this [blog post](https://www.ultralytics.com/blog/ai-and-radiology-a-new-era-of-precision-and-efficiency). + +### Can I use Gradio and Ultralytics YOLOv8 together for educational purposes? + +Yes, Gradio and Ultralytics YOLOv8 can be utilized together for educational purposes effectively. Gradio's intuitive web interface makes it easy for students and educators to interact with state-of-the-art deep learning models like Ultralytics YOLOv8 without needing advanced programming skills. This setup is ideal for demonstrating key concepts in object detection and computer vision, as Gradio provides immediate visual feedback which helps in understanding the impact of different parameters on the detection performance. + +### How do I adjust the confidence and IoU thresholds in the Gradio interface for YOLOv8? + +In the Gradio interface for YOLOv8, you can adjust the confidence and IoU thresholds using the sliders provided. These thresholds help control the prediction accuracy and object separation: + +- **Confidence Threshold:** Determines the minimum confidence level for detecting objects. Slide to increase or decrease the confidence required. +- **IoU Threshold:** Sets the intersection-over-union threshold for distinguishing between overlapping objects. Adjust this value to refine object separation. + +For more information on these parameters, visit the [parameters explanation section](#parameters-explanation). + +### What are some practical applications of using Ultralytics YOLOv8 with Gradio? + +Practical applications of combining Ultralytics YOLOv8 with Gradio include: + +- **Real-Time Object Detection Demonstrations:** Ideal for showcasing how object detection works in real-time. +- **Educational Tools:** Useful in academic settings to teach object detection and computer vision concepts. +- **Prototype Development:** Efficient for developing and testing prototype object detection applications quickly. +- **Community and Collaborations:** Making it easy to share models with the community for feedback and collaboration. + +For examples of similar use cases, check out the [Ultralytics blog](https://www.ultralytics.com/blog/monitoring-animal-behavior-using-ultralytics-yolov8). + +Providing this information within the documentation will help in enhancing the usability and accessibility of Ultralytics YOLOv8, making it more approachable for users at all levels of expertise. diff --git a/ultralytics/docs/en/integrations/ibm-watsonx.md b/ultralytics/docs/en/integrations/ibm-watsonx.md new file mode 100644 index 0000000000000000000000000000000000000000..d81c5bcf4bc73e6cc891acc395d5d71f85e166c3 --- /dev/null +++ b/ultralytics/docs/en/integrations/ibm-watsonx.md @@ -0,0 +1,410 @@ +--- +comments: true +description: Dive into our detailed integration guide on using IBM Watson to train a YOLOv8 model. Uncover key features and step-by-step instructions on model training. +keywords: IBM Watsonx, IBM Watsonx AI, What is Watson?, IBM Watson Integration, IBM Watson Features, YOLOv8, Ultralytics, Model Training, GPU, TPU, cloud computing +--- + +# A Step-by-Step Guide to Training YOLOv8 Models with IBM Watsonx + +Nowadays, scalable [computer vision solutions](../guides/steps-of-a-cv-project.md) are becoming more common and transforming the way we handle visual data. A great example is IBM Watsonx, an advanced AI and data platform that simplifies the development, deployment, and management of AI models. It offers a complete suite for the entire AI lifecycle and seamless integration with IBM Cloud services. + +You can train [Ultralytics YOLOv8 models](https://github.com/ultralytics/ultralytics) using IBM Watsonx. It's a good option for enterprises interested in efficient [model training](../modes/train.md), fine-tuning for specific tasks, and improving [model performance](../guides/model-evaluation-insights.md) with robust tools and a user-friendly setup. In this guide, we'll walk you through the process of training YOLOv8 with IBM Watsonx, covering everything from setting up your environment to evaluating your trained models. Let's get started! + +## What is IBM Watsonx? + +[Watsonx](https://www.ibm.com/watsonx) is IBM's cloud-based platform designed for commercial generative AI and scientific data. IBM Watsonx's three components - watsonx.ai, watsonx.data, and watsonx.governance - come together to create an end-to-end, trustworthy AI platform that can accelerate AI projects aimed at solving business problems. It provides powerful tools for building, training, and [deploying machine learning models](../guides/model-deployment-options.md) and makes it easy to connect with various data sources. + ++ +
+ +Its user-friendly interface and collaborative capabilities streamline the development process and help with efficient model management and deployment. Whether for computer vision, predictive analytics, natural language processing, or other AI applications, IBM Watsonx provides the tools and support needed to drive innovation. + +## Key Features of IBM Watsonx + +IBM Watsonx is made of three main components: watsonx.ai, watsonx.data, and watsonx.governance. Each component offers features that cater to different aspects of AI and data management. Let's take a closer look at them. + +### [Watsonx.ai](https://www.ibm.com/products/watsonx-ai) + +Watsonx.ai provides powerful tools for AI development and offers access to IBM-supported custom models, third-party models like [Llama 3](https://www.ultralytics.com/blog/getting-to-know-metas-llama-3), and IBM's own Granite models. It includes the Prompt Lab for experimenting with AI prompts, the Tuning Studio for improving model performance with labeled data, and the Flows Engine for simplifying generative AI application development. Also, it offers comprehensive tools for automating the AI model lifecycle and connecting to various APIs and libraries. + +### [Watsonx.data](https://www.ibm.com/products/watsonx-data) + +Watsonx.data supports both cloud and on-premises deployments through the IBM Storage Fusion HCI integration. Its user-friendly console provides centralized access to data across environments and makes data exploration easy with common SQL. It optimizes workloads with efficient query engines like Presto and Spark, accelerates data insights with an AI-powered semantic layer, includes a vector database for AI relevance, and supports open data formats for easy sharing of analytics and AI data. + +### [Watsonx.governance](https://www.ibm.com/products/watsonx-governance) + +Watsonx.governance makes compliance easier by automatically identifying regulatory changes and enforcing policies. It links requirements to internal risk data and provides up-to-date AI factsheets. The platform helps manage risk with alerts and tools to detect issues such as [bias and drift](../guides/model-monitoring-and-maintenance.md). It also automates the monitoring and documentation of the AI lifecycle, organizes AI development with a model inventory, and enhances collaboration with user-friendly dashboards and reporting tools. + +## How to Train YOLOv8 Using IBM Watsonx + +You can use IBM Watsonx to accelerate your YOLOv8 model training workflow. + +### Prerequisites + +You need an [IBM Cloud account](https://cloud.ibm.com/registration) to create a [watsonx.ai](https://www.ibm.com/products/watsonx-ai) project, and you'll also need a [Kaggle](./kaggle.md) account to load the data set. + +### Step 1: Set Up Your Environment + +First, you'll need to set up an IBM account to use a Jupyter Notebook. Log in to [watsonx.ai](https://eu-de.dataplatform.cloud.ibm.com/registration/stepone?preselect_region=true) using your IBM Cloud account. + +Then, create a [watsonx.ai project](https://www.ibm.com/docs/en/watsonx/saas?topic=projects-creating-project), and a [Jupyter Notebook](https://www.ibm.com/docs/en/watsonx/saas?topic=editor-creating-managing-notebooks). + +Once you do so, a notebook environment will open for you to load your data set. You can use the code from this tutorial to tackle a simple object detection model training task. + +### Step 2: Install and Import Relevant Libraries + +Next, you can install and import the necessary Python libraries. + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages + pip install torch torchvision torchaudio + pip install opencv-contrib-python-headless + pip install ultralytics==8.0.196 + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +Then, you can import the needed packages. + +!!! Example "Import Relevant Libraries" + + === "Python" + + ```python + # Import ultralytics + import ultralytics + + ultralytics.checks() + + # Import packages to retrieve and display image files + ``` + +### Step 3: Load the Data + +For this tutorial, we will use a [marine litter dataset](https://www.kaggle.com/datasets/atiqishrak/trash-dataset-icra19) available on Kaggle. With this dataset, we will custom-train a YOLOv8 model to detect and classify litter and biological objects in underwater images. + +We can load the dataset directly into the notebook using the Kaggle API. First, create a free Kaggle account. Once you have created an account, you'll need to generate an API key. Directions for generating your key can be found in the [Kaggle API documentation](https://github.com/Kaggle/kaggle-api/blob/main/docs/README.md) under the section "API credentials". + +Copy and paste your Kaggle username and API key into the following code. Then run the code to install the API and load the dataset into Watsonx. + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install kaggle + pip install kaggle + ``` + +After installing Kaggle, we can load the dataset into Watsonx. + +!!! Example "Load the Data" + + === "Python" + + ```python + # Replace "username" string with your username + os.environ["KAGGLE_USERNAME"] = "username" + # Replace "apiKey" string with your key + os.environ["KAGGLE_KEY"] = "apiKey" + + # Load dataset + os.system("kaggle datasets download atiqishrak/trash-dataset-icra19 --unzip") + + # Store working directory path as work_dir + work_dir = os.getcwd() + + # Print work_dir path + print(os.getcwd()) + + # Print work_dir contents + print(os.listdir(f"{work_dir}")) + + # Print trash_ICRA19 subdirectory contents + print(os.listdir(f"{work_dir}/trash_ICRA19")) + ``` + +After loading the dataset, we printed and saved our working directory. We have also printed the contents of our working directory to confirm the "trash_ICRA19" data set was loaded properly. + +If you see "trash_ICRA19" among the directory's contents, then it has loaded successfully. You should see three files/folders: a `config.yaml` file, a `videos_for_testing` directory, and a `dataset` directory. We will ignore the `videos_for_testing` directory, so feel free to delete it. + +We will use the config.yaml file and the contents of the dataset directory to train our object detection model. Here is a sample image from our marine litter data set. + ++ +
+ +### Step 4: Preprocess the Data + +Fortunately, all labels in the marine litter data set are already formatted as YOLO .txt files. However, we need to rearrange the structure of the image and label directories in order to help our model process the image and labels. Right now, our loaded data set directory follows this structure: + ++ +
+ +But, YOLO models by default require separate images and labels in subdirectories within the train/val/test split. We need to reorganize the directory into the following structure: + ++ +
+ +To reorganize the data set directory, we can run the following script: + +!!! Example "Preprocess the Data" + + === "Python" + + ```python + # Function to reorganize dir + def organize_files(directory): + for subdir in ["train", "test", "val"]: + subdir_path = os.path.join(directory, subdir) + if not os.path.exists(subdir_path): + continue + + images_dir = os.path.join(subdir_path, "images") + labels_dir = os.path.join(subdir_path, "labels") + + # Create image and label subdirs if non-existent + os.makedirs(images_dir, exist_ok=True) + os.makedirs(labels_dir, exist_ok=True) + + # Move images and labels to respective subdirs + for filename in os.listdir(subdir_path): + if filename.endswith(".txt"): + shutil.move(os.path.join(subdir_path, filename), os.path.join(labels_dir, filename)) + elif filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"): + shutil.move(os.path.join(subdir_path, filename), os.path.join(images_dir, filename)) + # Delete .xml files + elif filename.endswith(".xml"): + os.remove(os.path.join(subdir_path, filename)) + + + if __name__ == "__main__": + directory = f"{work_dir}/trash_ICRA19/dataset" + organize_files(directory) + ``` + +Next, we need to modify the .yaml file for the data set. This is the setup we will use in our .yaml file. Class ID numbers start from 0: + +```yaml +path: /path/to/dataset/directory # root directory for dataset +train: train/images # train images subdirectory +val: train/images # validation images subdirectory +test: test/images # test images subdirectory + +# Classes +names: + 0: plastic + 1: bio + 2: rov +``` + +Run the following script to delete the current contents of config.yaml and replace it with the above contents that reflect our new data set directory structure. Be certain to replace the work_dir portion of the root directory path in line 4 with your own working directory path we retrieved earlier. Leave the train, val, and test subdirectory definitions. Also, do not change {work_dir} in line 23 of the code. + +!!! Example "Edit the .yaml File" + + === "Python" + + ```python + # Contents of new confg.yaml file + def update_yaml_file(file_path): + data = { + "path": "work_dir/trash_ICRA19/dataset", + "train": "train/images", + "val": "train/images", + "test": "test/images", + "names": {0: "plastic", 1: "bio", 2: "rov"}, + } + + # Ensures the "names" list appears after the sub/directories + names_data = data.pop("names") + with open(file_path, "w") as yaml_file: + yaml.dump(data, yaml_file) + yaml_file.write("\n") + yaml.dump({"names": names_data}, yaml_file) + + + if __name__ == "__main__": + file_path = f"{work_dir}/trash_ICRA19/config.yaml" # .yaml file path + update_yaml_file(file_path) + print(f"{file_path} updated successfully.") + ``` + +### Step 5: Train the YOLOv8 model + +Run the following command-line code to fine tune a pretrained default YOLOv8 model. + +!!! Example "Train the YOLOv8 model" + + === "CLI" + + ```bash + !yolo task=detect mode=train data={work_dir}/trash_ICRA19/config.yaml model=yolov8s.pt epochs=2 batch=32 lr0=.04 plots=True + ``` + +Here's a closer look at the parameters in the model training command: + +- **task**: It specifies the computer vision task for which you are using the specified YOLO model and data set. +- **mode**: Denotes the purpose for which you are loading the specified model and data. Since we are training a model, it is set to "train." Later, when we test our model's performance, we will set it to "predict." +- **epochs**: This delimits the number of times YOLOv8 will pass through our entire data set. +- **batch**: The numerical value stipulates the training batch sizes. Batches are the number of images a model processes before it updates its parameters. +- **lr0**: Specifies the model's initial learning rate. +- **plots**: Directs YOLO to generate and save plots of our model's training and evaluation metrics. + +For a detailed understanding of the model training process and best practices, refer to the [YOLOv8 Model Training guide](../modes/train.md). This guide will help you get the most out of your experiments and ensure you're using YOLOv8 effectively. + +### Step 6: Test the Model + +We can now run inference to test the performance of our fine-tuned model: + +!!! Example "Test the YOLOv8 model" + + === "CLI" + + ```bash + !yolo task=detect mode=predict source={work_dir}/trash_ICRA19/dataset/test/images model={work_dir}/runs/detect/train/weights/best.pt conf=0.5 iou=.5 save=True save_txt=True + ``` + +This brief script generates predicted labels for each image in our test set, as well as new output image files that overlay the predicted bounding box atop the original image. + +Predicted .txt labels for each image are saved via the `save_txt=True` argument and the output images with bounding box overlays are generated through the `save=True` argument. +The parameter `conf=0.5` informs the model to ignore all predictions with a confidence level of less than 50%. + +Lastly, `iou=.5` directs the model to ignore boxes in the same class with an overlap of 50% or greater. It helps to reduce potential duplicate boxes generated for the same object. +we can load the images with predicted bounding box overlays to view how our model performs on a handful of images. + +!!! Example "Display Predictions" + + === "Python" + + ```python + # Show the first ten images from the preceding prediction task + for pred_dir in glob.glob(f"{work_dir}/runs/detect/predict/*.jpg")[:10]: + img = Image.open(pred_dir) + display(img) + ``` + +The code above displays ten images from the test set with their predicted bounding boxes, accompanied by class name labels and confidence levels. + +### Step 7: Evaluate the Model + +We can produce visualizations of the model's precision and recall for each class. These visualizations are saved in the home directory, under the train folder. The precision score is displayed in the P_curve.png: + ++ +
+ +The graph shows an exponential increase in precision as the model's confidence level for predictions increases. However, the model precision has not yet leveled out at a certain confidence level after two epochs. + +The recall graph (R_curve.png) displays an inverse trend: + ++ +
+ +Unlike precision, recall moves in the opposite direction, showing greater recall with lower confidence instances and lower recall with higher confidence instances. This is an apt example of the trade-off in precision and recall for classification models. + +### Step 8: Calculating Intersection Over Union + +You can measure the prediction accuracy by calculating the IoU between a predicted bounding box and a ground truth bounding box for the same object. Check out [IBM's tutorial on training YOLOv8](https://developer.ibm.com/tutorials/awb-train-yolo-object-detection-model-in-python/) for more details. + +## Summary + +We explored IBM Watsonx key features, and how to train a YOLOv8 model using IBM Watsonx. We also saw how IBM Watsonx can enhance your AI workflows with advanced tools for model building, data management, and compliance. + +For further details on usage, visit [IBM Watsonx official documentation](https://www.ibm.com/watsonx). + +Also, be sure to check out the [Ultralytics integration guide page](./index.md), to learn more about different exciting integrations. + +## FAQ + +### How do I train a YOLOv8 model using IBM Watsonx? + +To train a YOLOv8 model using IBM Watsonx, follow these steps: + +1. **Set Up Your Environment**: Create an IBM Cloud account and set up a Watsonx.ai project. Use a Jupyter Notebook for your coding environment. +2. **Install Libraries**: Install necessary libraries like `torch`, `opencv`, and `ultralytics`. +3. **Load Data**: Use the Kaggle API to load your dataset into Watsonx. +4. **Preprocess Data**: Organize your dataset into the required directory structure and update the `.yaml` configuration file. +5. **Train the Model**: Use the YOLO command-line interface to train your model with specific parameters like `epochs`, `batch size`, and `learning rate`. +6. **Test and Evaluate**: Run inference to test the model and evaluate its performance using metrics like precision and recall. + +For detailed instructions, refer to our [YOLOv8 Model Training guide](../modes/train.md). + +### What are the key features of IBM Watsonx for AI model training? + +IBM Watsonx offers several key features for AI model training: + +- **Watsonx.ai**: Provides tools for AI development, including access to IBM-supported custom models and third-party models like Llama 3. It includes the Prompt Lab, Tuning Studio, and Flows Engine for comprehensive AI lifecycle management. +- **Watsonx.data**: Supports cloud and on-premises deployments, offering centralized data access, efficient query engines like Presto and Spark, and an AI-powered semantic layer. +- **Watsonx.governance**: Automates compliance, manages risk with alerts, and provides tools for detecting issues like bias and drift. It also includes dashboards and reporting tools for collaboration. + +For more information, visit the [IBM Watsonx official documentation](https://www.ibm.com/watsonx). + +### Why should I use IBM Watsonx for training Ultralytics YOLOv8 models? + +IBM Watsonx is an excellent choice for training Ultralytics YOLOv8 models due to its comprehensive suite of tools that streamline the AI lifecycle. Key benefits include: + +- **Scalability**: Easily scale your model training with IBM Cloud services. +- **Integration**: Seamlessly integrate with various data sources and APIs. +- **User-Friendly Interface**: Simplifies the development process with a collaborative and intuitive interface. +- **Advanced Tools**: Access to powerful tools like the Prompt Lab, Tuning Studio, and Flows Engine for enhancing model performance. + +Learn more about [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) and how to train models using IBM Watsonx in our [integration guide](./index.md). + +### How can I preprocess my dataset for YOLOv8 training on IBM Watsonx? + +To preprocess your dataset for YOLOv8 training on IBM Watsonx: + +1. **Organize Directories**: Ensure your dataset follows the YOLO directory structure with separate subdirectories for images and labels within the train/val/test split. +2. **Update .yaml File**: Modify the `.yaml` configuration file to reflect the new directory structure and class names. +3. **Run Preprocessing Script**: Use a Python script to reorganize your dataset and update the `.yaml` file accordingly. + +Here's a sample script to organize your dataset: + +```python +import os +import shutil + + +def organize_files(directory): + for subdir in ["train", "test", "val"]: + subdir_path = os.path.join(directory, subdir) + if not os.path.exists(subdir_path): + continue + + images_dir = os.path.join(subdir_path, "images") + labels_dir = os.path.join(subdir_path, "labels") + + os.makedirs(images_dir, exist_ok=True) + os.makedirs(labels_dir, exist_ok=True) + + for filename in os.listdir(subdir_path): + if filename.endswith(".txt"): + shutil.move(os.path.join(subdir_path, filename), os.path.join(labels_dir, filename)) + elif filename.endswith(".jpg") or filename.endswith(".png") or filename.endswith(".jpeg"): + shutil.move(os.path.join(subdir_path, filename), os.path.join(images_dir, filename)) + + +if __name__ == "__main__": + directory = f"{work_dir}/trash_ICRA19/dataset" + organize_files(directory) +``` + +For more details, refer to our [data preprocessing guide](../guides/preprocessing_annotated_data.md). + +### What are the prerequisites for training a YOLOv8 model on IBM Watsonx? + +Before you start training a YOLOv8 model on IBM Watsonx, ensure you have the following prerequisites: + +- **IBM Cloud Account**: Create an account on IBM Cloud to access Watsonx.ai. +- **Kaggle Account**: For loading datasets, you'll need a Kaggle account and an API key. +- **Jupyter Notebook**: Set up a Jupyter Notebook environment within Watsonx.ai for coding and model training. + +For more information on setting up your environment, visit our [Ultralytics Installation guide](../quickstart.md). diff --git a/ultralytics/docs/en/integrations/index.md b/ultralytics/docs/en/integrations/index.md new file mode 100644 index 0000000000000000000000000000000000000000..b3f6cb3655025d5aa965a7818b68ddf685b57764 --- /dev/null +++ b/ultralytics/docs/en/integrations/index.md @@ -0,0 +1,130 @@ +--- +comments: true +description: Discover Ultralytics integrations for streamlined ML workflows, dataset management, optimized model training, and robust deployment solutions. +keywords: Ultralytics, machine learning, ML workflows, dataset management, model training, model deployment, Roboflow, ClearML, Comet ML, DVC, MLFlow, Ultralytics HUB, Neptune, Ray Tune, TensorBoard, Weights & Biases, Amazon SageMaker, Paperspace Gradient, Google Colab, Neural Magic, Gradio, TorchScript, ONNX, OpenVINO, TensorRT, CoreML, TF SavedModel, TF GraphDef, TFLite, TFLite Edge TPU, TF.js, PaddlePaddle, NCNN +--- + +# Ultralytics Integrations + +Welcome to the Ultralytics Integrations page! This page provides an overview of our partnerships with various tools and platforms, designed to streamline your machine learning workflows, enhance dataset management, simplify model training, and facilitate efficient deployment. + + + +
+
+
+
+ Watch: Ultralytics YOLOv8 Deployment and Integrations
+
+ +
+ +The [NCNN](https://github.com/Tencent/ncnn) framework, developed by Tencent, is a high-performance neural network inference computing framework optimized specifically for mobile platforms, including mobile phones, embedded devices, and IoT devices. NCNN is compatible with a wide range of platforms, including Linux, Android, iOS, and macOS. + +NCNN is known for its fast processing speed on mobile CPUs and enables rapid deployment of deep learning models to mobile platforms. This makes it easier to build smart apps, putting the power of AI right at your fingertips. + +## Key Features of NCNN Models + +NCNN models offer a wide range of key features that enable on-device machine learning by helping developers run their models on mobile, embedded, and edge devices: + +- **Efficient and High-Performance**: NCNN models are made to be efficient and lightweight, optimized for running on mobile and embedded devices like Raspberry Pi with limited resources. They can also achieve high performance with high accuracy on various computer vision-based tasks. + +- **Quantization**: NCNN models often support quantization which is a technique that reduces the precision of the model's weights and activations. This leads to further improvements in performance and reduces memory footprint. + +- **Compatibility**: NCNN models are compatible with popular deep learning frameworks like [TensorFlow](https://www.tensorflow.org/), [Caffe](https://caffe.berkeleyvision.org/), and [ONNX](https://onnx.ai/). This compatibility allows developers to use existing models and workflows easily. + +- **Easy to Use**: NCNN models are designed for easy integration into various applications, thanks to their compatibility with popular deep learning frameworks. Additionally, NCNN offers user-friendly tools for converting models between different formats, ensuring smooth interoperability across the development landscape. + +## Deployment Options with NCNN + +Before we look at the code for exporting YOLOv8 models to the NCNN format, let's understand how NCNN models are normally used. + +NCNN models, designed for efficiency and performance, are compatible with a variety of deployment platforms: + +- **Mobile Deployment**: Specifically optimized for Android and iOS, allowing for seamless integration into mobile applications for efficient on-device inference. + +- **Embedded Systems and IoT Devices**: If you find that running inference on a Raspberry Pi with the [Ultralytics Guide](../guides/raspberry-pi.md) isn't fast enough, switching to an NCNN exported model could help speed things up. NCNN is great for devices like Raspberry Pi and NVIDIA Jetson, especially in situations where you need quick processing right on the device. + +- **Desktop and Server Deployment**: Capable of being deployed in desktop and server environments across Linux, Windows, and macOS, supporting development, training, and evaluation with higher computational capacities. + +## Export to NCNN: Converting Your YOLOv8 Model + +You can expand model compatibility and deployment flexibility by converting YOLOv8 models to NCNN format. + +### Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to NCNN format + model.export(format="ncnn") # creates '/yolov8n_ncnn_model' + + # Load the exported NCNN model + ncnn_model = YOLO("./yolov8n_ncnn_model") + + # Run inference + results = ncnn_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to NCNN format + yolo export model=yolov8n.pt format=ncnn # creates '/yolov8n_ncnn_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_ncnn_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 NCNN Models + +After successfully exporting your Ultralytics YOLOv8 models to NCNN format, you can now deploy them. The primary and recommended first step for running a NCNN model is to utilize the YOLO("./model_ncnn_model") method, as outlined in the previous usage code snippet. However, for in-depth instructions on deploying your NCNN models in various other settings, take a look at the following resources: + +- **[Android](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-android)**: This blog explains how to use NCNN models for performing tasks like object detection through Android applications. + +- **[macOS](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-macos)**: Understand how to use NCNN models for performing tasks through macOS. + +- **[Linux](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-linux)**: Explore this page to learn how to deploy NCNN models on limited resource devices like Raspberry Pi and other similar devices. + +- **[Windows x64 using VS2017](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-windows-x64-using-visual-studio-community-2017)**: Explore this blog to learn how to deploy NCNN models on windows x64 using Visual Studio Community 2017. + +## Summary + +In this guide, we've gone over exporting Ultralytics YOLOv8 models to the NCNN format. This conversion step is crucial for improving the efficiency and speed of YOLOv8 models, making them more effective and suitable for limited-resource computing environments. + +For detailed instructions on usage, please refer to the [official NCNN documentation](https://ncnn.readthedocs.io/en/latest/index.html). + +Also, if you're interested in exploring other integration options for Ultralytics YOLOv8, be sure to visit our [integration guide page](index.md) for further insights and information. + +## FAQ + +### How do I export Ultralytics YOLOv8 models to NCNN format? + +To export your Ultralytics YOLOv8 model to NCNN format, follow these steps: + +- **Python**: Use the `export` function from the YOLO class. + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export to NCNN format + model.export(format="ncnn") # creates '/yolov8n_ncnn_model' + ``` + +- **CLI**: Use the `yolo` command with the `export` argument. + ```bash + yolo export model=yolov8n.pt format=ncnn # creates '/yolov8n_ncnn_model' + ``` + +For detailed export options, check the [Export](../modes/export.md) page in the documentation. + +### What are the advantages of exporting YOLOv8 models to NCNN? + +Exporting your Ultralytics YOLOv8 models to NCNN offers several benefits: + +- **Efficiency**: NCNN models are optimized for mobile and embedded devices, ensuring high performance even with limited computational resources. +- **Quantization**: NCNN supports techniques like quantization that improve model speed and reduce memory usage. +- **Broad Compatibility**: You can deploy NCNN models on multiple platforms, including Android, iOS, Linux, and macOS. + +For more details, see the [Export to NCNN](#why-should-you-export-to-ncnn) section in the documentation. + +### Why should I use NCNN for my mobile AI applications? + +NCNN, developed by Tencent, is specifically optimized for mobile platforms. Key reasons to use NCNN include: + +- **High Performance**: Designed for efficient and fast processing on mobile CPUs. +- **Cross-Platform**: Compatible with popular frameworks such as TensorFlow and ONNX, making it easier to convert and deploy models across different platforms. +- **Community Support**: Active community support ensures continual improvements and updates. + +To understand more, visit the [NCNN overview](#key-features-of-ncnn-models) in the documentation. + +### What platforms are supported for NCNN model deployment? + +NCNN is versatile and supports various platforms: + +- **Mobile**: Android, iOS. +- **Embedded Systems and IoT Devices**: Devices like Raspberry Pi and NVIDIA Jetson. +- **Desktop and Servers**: Linux, Windows, and macOS. + +If running models on a Raspberry Pi isn't fast enough, converting to the NCNN format could speed things up as detailed in our [Raspberry Pi Guide](../guides/raspberry-pi.md). + +### How can I deploy Ultralytics YOLOv8 NCNN models on Android? + +To deploy your YOLOv8 models on Android: + +1. **Build for Android**: Follow the [NCNN Build for Android](https://github.com/Tencent/ncnn/wiki/how-to-build#build-for-android) guide. +2. **Integrate with Your App**: Use the NCNN Android SDK to integrate the exported model into your application for efficient on-device inference. + +For step-by-step instructions, refer to our guide on [Deploying YOLOv8 NCNN Models](#deploying-exported-yolov8-ncnn-models). + +For more advanced guides and use cases, visit the [Ultralytics documentation page](../guides/model-deployment-options.md). diff --git a/ultralytics/docs/en/integrations/neural-magic.md b/ultralytics/docs/en/integrations/neural-magic.md new file mode 100644 index 0000000000000000000000000000000000000000..1ae52e89974f3cb69c66b5d843aecae98494879a --- /dev/null +++ b/ultralytics/docs/en/integrations/neural-magic.md @@ -0,0 +1,211 @@ +--- +comments: true +description: Enhance YOLOv8 performance using Neural Magic's DeepSparse Engine. Learn how to deploy and benchmark YOLOv8 models on CPUs for efficient object detection. +keywords: YOLOv8, DeepSparse, Neural Magic, model optimization, object detection, inference speed, CPU performance, sparsity, pruning, quantization +--- + +# Optimizing YOLOv8 Inferences with Neural Magic's DeepSparse Engine + +When deploying object detection models like [Ultralytics YOLOv8](https://ultralytics.com) on various hardware, you can bump into unique issues like optimization. This is where YOLOv8's integration with Neural Magic's DeepSparse Engine steps in. It transforms the way YOLOv8 models are executed and enables GPU-level performance directly on CPUs. + +This guide shows you how to deploy YOLOv8 using Neural Magic's DeepSparse, how to run inferences, and also how to benchmark performance to ensure it is optimized. + +## Neural Magic's DeepSparse + ++ +
+ +[Neural Magic's DeepSparse](https://neuralmagic.com/deepsparse/) is an inference run-time designed to optimize the execution of neural networks on CPUs. It applies advanced techniques like sparsity, pruning, and quantization to dramatically reduce computational demands while maintaining accuracy. DeepSparse offers an agile solution for efficient and scalable neural network execution across various devices. + +## Benefits of Integrating Neural Magic's DeepSparse with YOLOv8 + +Before diving into how to deploy YOLOV8 using DeepSparse, let's understand the benefits of using DeepSparse. Some key advantages include: + +- **Enhanced Inference Speed**: Achieves up to 525 FPS (on YOLOv8n), significantly speeding up YOLOv8's inference capabilities compared to traditional methods. + ++ +
+ +- **Optimized Model Efficiency**: Uses pruning and quantization to enhance YOLOv8's efficiency, reducing model size and computational requirements while maintaining accuracy. + ++ +
+ +- **High Performance on Standard CPUs**: Delivers GPU-like performance on CPUs, providing a more accessible and cost-effective option for various applications. + +- **Streamlined Integration and Deployment**: Offers user-friendly tools for easy integration of YOLOv8 into applications, including image and video annotation features. + +- **Support for Various Model Types**: Compatible with both standard and sparsity-optimized YOLOv8 models, adding deployment flexibility. + +- **Cost-Effective and Scalable Solution**: Reduces operational expenses and offers scalable deployment of advanced object detection models. + +## How Does Neural Magic's DeepSparse Technology Works? + +Neural Magic's Deep Sparse technology is inspired by the human brain's efficiency in neural network computation. It adopts two key principles from the brain as follows: + +- **Sparsity**: The process of sparsification involves pruning redundant information from deep learning networks, leading to smaller and faster models without compromising accuracy. This technique reduces the network's size and computational needs significantly. + +- **Locality of Reference**: DeepSparse uses a unique execution method, breaking the network into Tensor Columns. These columns are executed depth-wise, fitting entirely within the CPU's cache. This approach mimics the brain's efficiency, minimizing data movement and maximizing the CPU's cache use. + ++ +
+ +For more details on how Neural Magic's DeepSparse technology work, check out [their blog post](https://neuralmagic.com/blog/how-neural-magics-deep-sparse-technology-works/). + +## Creating A Sparse Version of YOLOv8 Trained on a Custom Dataset + +SparseZoo, an open-source model repository by Neural Magic, offers [a collection of pre-sparsified YOLOv8 model checkpoints](https://sparsezoo.neuralmagic.com/?modelSet=computer_vision&searchModels=yolo). With SparseML, seamlessly integrated with Ultralytics, users can effortlessly fine-tune these sparse checkpoints on their specific datasets using a straightforward command-line interface. + +Checkout [Neural Magic's SparseML YOLOv8 documentation](https://github.com/neuralmagic/sparseml/tree/main/integrations/ultralytics-yolov8) for more details. + +## Usage: Deploying YOLOV8 using DeepSparse + +Deploying YOLOv8 with Neural Magic's DeepSparse involves a few straightforward steps. Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. Here's how you can get started. + +### Step 1: Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages + pip install deepsparse[yolov8] + ``` + +### Step 2: Exporting YOLOv8 to ONNX Format + +DeepSparse Engine requires YOLOv8 models in ONNX format. Exporting your model to this format is essential for compatibility with DeepSparse. Use the following command to export YOLOv8 models: + +!!! Tip "Model Export" + + === "CLI" + + ```bash + # Export YOLOv8 model to ONNX format + yolo task=detect mode=export model=yolov8n.pt format=onnx opset=13 + ``` + +This command will save the `yolov8n.onnx` model to your disk. + +### Step 3: Deploying and Running Inferences + +With your YOLOv8 model in ONNX format, you can deploy and run inferences using DeepSparse. This can be done easily with their intuitive Python API: + +!!! Tip "Deploying and Running Inferences" + + === "Python" + + ```python + from deepsparse import Pipeline + + # Specify the path to your YOLOv8 ONNX model + model_path = "path/to/yolov8n.onnx" + + # Set up the DeepSparse Pipeline + yolo_pipeline = Pipeline.create(task="yolov8", model_path=model_path) + + # Run the model on your images + images = ["path/to/image.jpg"] + pipeline_outputs = yolo_pipeline(images=images) + ``` + +### Step 4: Benchmarking Performance + +It's important to check that your YOLOv8 model is performing optimally on DeepSparse. You can benchmark your model's performance to analyze throughput and latency: + +!!! Tip "Benchmarking" + + === "CLI" + + ```bash + # Benchmark performance + deepsparse.benchmark model_path="path/to/yolov8n.onnx" --scenario=sync --input_shapes="[1,3,640,640]" + ``` + +### Step 5: Additional Features + +DeepSparse provides additional features for practical integration of YOLOv8 in applications, such as image annotation and dataset evaluation. + +!!! Tip "Additional Features" + + === "CLI" + + ```bash + # For image annotation + deepsparse.yolov8.annotate --source "path/to/image.jpg" --model_filepath "path/to/yolov8n.onnx" + + # For evaluating model performance on a dataset + deepsparse.yolov8.eval --model_path "path/to/yolov8n.onnx" + ``` + +Running the annotate command processes your specified image, detecting objects, and saving the annotated image with bounding boxes and classifications. The annotated image will be stored in an annotation-results folder. This helps provide a visual representation of the model's detection capabilities. + ++ +
+ +After running the eval command, you will receive detailed output metrics such as precision, recall, and mAP (mean Average Precision). This provides a comprehensive view of your model's performance on the dataset. This functionality is particularly useful for fine-tuning and optimizing your YOLOv8 models for specific use cases, ensuring high accuracy and efficiency. + +## Summary + +This guide explored integrating Ultralytics' YOLOv8 with Neural Magic's DeepSparse Engine. It highlighted how this integration enhances YOLOv8's performance on CPU platforms, offering GPU-level efficiency and advanced neural network sparsity techniques. + +For more detailed information and advanced usage, visit [Neural Magic's DeepSparse documentation](https://docs.neuralmagic.com/products/deepsparse/). Also, check out Neural Magic's documentation on the integration with YOLOv8 [here](https://github.com/neuralmagic/deepsparse/tree/main/src/deepsparse/yolov8#yolov8-inference-pipelines) and watch a great session on it [here](https://www.youtube.com/watch?v=qtJ7bdt52x8). + +Additionally, for a broader understanding of various YOLOv8 integrations, visit the [Ultralytics integration guide page](../integrations/index.md), where you can discover a range of other exciting integration possibilities. + +## FAQ + +### What is Neural Magic's DeepSparse Engine and how does it optimize YOLOv8 performance? + +Neural Magic's DeepSparse Engine is an inference runtime designed to optimize the execution of neural networks on CPUs through advanced techniques such as sparsity, pruning, and quantization. By integrating DeepSparse with YOLOv8, you can achieve GPU-like performance on standard CPUs, significantly enhancing inference speed, model efficiency, and overall performance while maintaining accuracy. For more details, check out the [Neural Magic's DeepSparse section](#neural-magics-deepsparse). + +### How can I install the needed packages to deploy YOLOv8 using Neural Magic's DeepSparse? + +Installing the required packages for deploying YOLOv8 with Neural Magic's DeepSparse is straightforward. You can easily install them using the CLI. Here's the command you need to run: + +```bash +pip install deepsparse[yolov8] +``` + +Once installed, follow the steps provided in the [Installation section](#step-1-installation) to set up your environment and start using DeepSparse with YOLOv8. + +### How do I convert YOLOv8 models to ONNX format for use with DeepSparse? + +To convert YOLOv8 models to the ONNX format, which is required for compatibility with DeepSparse, you can use the following CLI command: + +```bash +yolo task=detect mode=export model=yolov8n.pt format=onnx opset=13 +``` + +This command will export your YOLOv8 model (`yolov8n.pt`) to a format (`yolov8n.onnx`) that can be utilized by the DeepSparse Engine. More information about model export can be found in the [Model Export section](#step-2-exporting-yolov8-to-onnx-format). + +### How do I benchmark YOLOv8 performance on the DeepSparse Engine? + +Benchmarking YOLOv8 performance on DeepSparse helps you analyze throughput and latency to ensure your model is optimized. You can use the following CLI command to run a benchmark: + +```bash +deepsparse.benchmark model_path="path/to/yolov8n.onnx" --scenario=sync --input_shapes="[1,3,640,640]" +``` + +This command will provide you with vital performance metrics. For more details, see the [Benchmarking Performance section](#step-4-benchmarking-performance). + +### Why should I use Neural Magic's DeepSparse with YOLOv8 for object detection tasks? + +Integrating Neural Magic's DeepSparse with YOLOv8 offers several benefits: + +- **Enhanced Inference Speed:** Achieves up to 525 FPS, significantly speeding up YOLOv8's capabilities. +- **Optimized Model Efficiency:** Uses sparsity, pruning, and quantization techniques to reduce model size and computational needs while maintaining accuracy. +- **High Performance on Standard CPUs:** Offers GPU-like performance on cost-effective CPU hardware. +- **Streamlined Integration:** User-friendly tools for easy deployment and integration. +- **Flexibility:** Supports both standard and sparsity-optimized YOLOv8 models. +- **Cost-Effective:** Reduces operational expenses through efficient resource utilization. + +For a deeper dive into these advantages, visit the [Benefits of Integrating Neural Magic's DeepSparse with YOLOv8 section](#benefits-of-integrating-neural-magics-deepsparse-with-yolov8). diff --git a/ultralytics/docs/en/integrations/onnx.md b/ultralytics/docs/en/integrations/onnx.md new file mode 100644 index 0000000000000000000000000000000000000000..94865cba12bf6ee49ef33bbe104e55409d533442 --- /dev/null +++ b/ultralytics/docs/en/integrations/onnx.md @@ -0,0 +1,213 @@ +--- +comments: true +description: Learn how to export YOLOv8 models to ONNX format for flexible deployment across various platforms with enhanced performance. +keywords: YOLOv8, ONNX, model export, Ultralytics, ONNX Runtime, machine learning, model deployment, computer vision, deep learning +--- + +# ONNX Export for YOLOv8 Models + +Often, when deploying computer vision models, you'll need a model format that's both flexible and compatible with multiple platforms. + +Exporting [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models to ONNX format streamlines deployment and ensures optimal performance across various environments. This guide will show you how to easily convert your YOLOv8 models to ONNX and enhance their scalability and effectiveness in real-world applications. + +## ONNX and ONNX Runtime + +[ONNX](https://onnx.ai/), which stands for Open Neural Network Exchange, is a community project that Facebook and Microsoft initially developed. The ongoing development of ONNX is a collaborative effort supported by various organizations like IBM, Amazon (through AWS), and Google. The project aims to create an open file format designed to represent machine learning models in a way that allows them to be used across different AI frameworks and hardware. + +ONNX models can be used to transition between different frameworks seamlessly. For instance, a deep learning model trained in PyTorch can be exported to ONNX format and then easily imported into TensorFlow. + ++ +
+ +Alternatively, ONNX models can be used with ONNX Runtime. [ONNX Runtime](https://onnxruntime.ai/) is a versatile cross-platform accelerator for machine learning models that is compatible with frameworks like PyTorch, TensorFlow, TFLite, scikit-learn, etc. + +ONNX Runtime optimizes the execution of ONNX models by leveraging hardware-specific capabilities. This optimization allows the models to run efficiently and with high performance on various hardware platforms, including CPUs, GPUs, and specialized accelerators. + ++ +
+ +Whether used independently or in tandem with ONNX Runtime, ONNX provides a flexible solution for machine learning model deployment and compatibility. + +## Key Features of ONNX Models + +The ability of ONNX to handle various formats can be attributed to the following key features: + +- **Common Model Representation**: ONNX defines a common set of operators (like convolutions, layers, etc.) and a standard data format. When a model is converted to ONNX format, its architecture and weights are translated into this common representation. This uniformity ensures that the model can be understood by any framework that supports ONNX. + +- **Versioning and Backward Compatibility**: ONNX maintains a versioning system for its operators. This ensures that even as the standard evolves, models created in older versions remain usable. Backward compatibility is a crucial feature that prevents models from becoming obsolete quickly. + +- **Graph-based Model Representation**: ONNX represents models as computational graphs. This graph-based structure is a universal way of representing machine learning models, where nodes represent operations or computations, and edges represent the tensors flowing between them. This format is easily adaptable to various frameworks which also represent models as graphs. + +- **Tools and Ecosystem**: There is a rich ecosystem of tools around ONNX that assist in model conversion, visualization, and optimization. These tools make it easier for developers to work with ONNX models and to convert models between different frameworks seamlessly. + +## Common Usage of ONNX + +Before we jump into how to export YOLOv8 models to the ONNX format, let's take a look at where ONNX models are usually used. + +### CPU Deployment + +ONNX models are often deployed on CPUs due to their compatibility with ONNX Runtime. This runtime is optimized for CPU execution. It significantly improves inference speed and makes real-time CPU deployments feasible. + +### Supported Deployment Options + +While ONNX models are commonly used on CPUs, they can also be deployed on the following platforms: + +- **GPU Acceleration**: ONNX fully supports GPU acceleration, particularly NVIDIA CUDA. This enables efficient execution on NVIDIA GPUs for tasks that demand high computational power. + +- **Edge and Mobile Devices**: ONNX extends to edge and mobile devices, perfect for on-device and real-time inference scenarios. It's lightweight and compatible with edge hardware. + +- **Web Browsers**: ONNX can run directly in web browsers, powering interactive and dynamic web-based AI applications. + +## Exporting YOLOv8 Models to ONNX + +You can expand model compatibility and deployment flexibility by converting YOLOv8 models to ONNX format. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to ONNX format + model.export(format="onnx") # creates 'yolov8n.onnx' + + # Load the exported ONNX model + onnx_model = YOLO("yolov8n.onnx") + + # Run inference + results = onnx_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to ONNX format + yolo export model=yolov8n.pt format=onnx # creates 'yolov8n.onnx' + + # Run inference with the exported model + yolo predict model=yolov8n.onnx source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +## Deploying Exported YOLOv8 ONNX Models + +Once you've successfully exported your Ultralytics YOLOv8 models to ONNX format, the next step is deploying these models in various environments. For detailed instructions on deploying your ONNX models, take a look at the following resources: + +- **[ONNX Runtime Python API Documentation](https://onnxruntime.ai/docs/api/python/api_summary.html)**: This guide provides essential information for loading and running ONNX models using ONNX Runtime. + +- **[Deploying on Edge Devices](https://onnxruntime.ai/docs/tutorials/iot-edge/)**: Check out this docs page for different examples of deploying ONNX models on edge. + +- **[ONNX Tutorials on GitHub](https://github.com/onnx/tutorials)**: A collection of comprehensive tutorials that cover various aspects of using and implementing ONNX models in different scenarios. + +## Summary + +In this guide, you've learned how to export Ultralytics YOLOv8 models to ONNX format to increase their interoperability and performance across various platforms. You were also introduced to the ONNX Runtime and ONNX deployment options. + +For further details on usage, visit the [ONNX official documentation](https://onnx.ai/onnx/intro/). + +Also, if you'd like to know more about other Ultralytics YOLOv8 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of useful resources and insights there. + +## FAQ + +### How do I export YOLOv8 models to ONNX format using Ultralytics? + +To export your YOLOv8 models to ONNX format using Ultralytics, follow these steps: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to ONNX format + model.export(format="onnx") # creates 'yolov8n.onnx' + + # Load the exported ONNX model + onnx_model = YOLO("yolov8n.onnx") + + # Run inference + results = onnx_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to ONNX format + yolo export model=yolov8n.pt format=onnx # creates 'yolov8n.onnx' + + # Run inference with the exported model + yolo predict model=yolov8n.onnx source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details, visit the [export documentation](../modes/export.md). + +### What are the advantages of using ONNX Runtime for deploying YOLOv8 models? + +Using ONNX Runtime for deploying YOLOv8 models offers several advantages: + +- **Cross-platform compatibility**: ONNX Runtime supports various platforms, such as Windows, macOS, and Linux, ensuring your models run smoothly across different environments. +- **Hardware acceleration**: ONNX Runtime can leverage hardware-specific optimizations for CPUs, GPUs, and dedicated accelerators, providing high-performance inference. +- **Framework interoperability**: Models trained in popular frameworks like PyTorch or TensorFlow can be easily converted to ONNX format and run using ONNX Runtime. + +Learn more by checking the [ONNX Runtime documentation](https://onnxruntime.ai/docs/api/python/api_summary.html). + +### What deployment options are available for YOLOv8 models exported to ONNX? + +YOLOv8 models exported to ONNX can be deployed on various platforms including: + +- **CPUs**: Utilizing ONNX Runtime for optimized CPU inference. +- **GPUs**: Leveraging NVIDIA CUDA for high-performance GPU acceleration. +- **Edge devices**: Running lightweight models on edge and mobile devices for real-time, on-device inference. +- **Web browsers**: Executing models directly within web browsers for interactive web-based applications. + +For more information, explore our guide on [model deployment options](../guides/model-deployment-options.md). + +### Why should I use ONNX format for Ultralytics YOLOv8 models? + +Using ONNX format for Ultralytics YOLOv8 models provides numerous benefits: + +- **Interoperability**: ONNX allows models to be transferred between different machine learning frameworks seamlessly. +- **Performance Optimization**: ONNX Runtime can enhance model performance by utilizing hardware-specific optimizations. +- **Flexibility**: ONNX supports various deployment environments, enabling you to use the same model on different platforms without modification. + +Refer to the comprehensive guide on [exporting YOLOv8 models to ONNX](https://www.ultralytics.com/blog/export-and-optimize-a-yolov8-model-for-inference-on-openvino). + +### How can I troubleshoot issues when exporting YOLOv8 models to ONNX? + +When exporting YOLOv8 models to ONNX, you might encounter common issues such as mismatched dependencies or unsupported operations. To troubleshoot these problems: + +1. Verify that you have the correct version of required dependencies installed. +2. Check the official [ONNX documentation](https://onnx.ai/onnx/intro/) for supported operators and features. +3. Review the error messages for clues and consult the [Ultralytics Common Issues guide](../guides/yolo-common-issues.md). + +If issues persist, contact Ultralytics support for further assistance. diff --git a/ultralytics/docs/en/integrations/openvino.md b/ultralytics/docs/en/integrations/openvino.md new file mode 100644 index 0000000000000000000000000000000000000000..0ab0aad7d3e4a540670aecb4616570d40c25bd7d --- /dev/null +++ b/ultralytics/docs/en/integrations/openvino.md @@ -0,0 +1,393 @@ +--- +comments: true +description: Learn to export YOLOv8 models to OpenVINO format for up to 3x CPU speedup and hardware acceleration on Intel GPU and NPU. +keywords: YOLOv8, OpenVINO, model export, Intel, AI inference, CPU speedup, GPU acceleration, NPU, deep learning +--- + +# Intel OpenVINO Export + + + +In this guide, we cover exporting YOLOv8 models to the [OpenVINO](https://docs.openvino.ai/) format, which can provide up to 3x [CPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/cpu-device.html) speedup, as well as accelerating YOLO inference on Intel [GPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/gpu-device.html) and [NPU](https://docs.openvino.ai/2024/openvino-workflow/running-inference/inference-devices-and-modes/npu-device.html) hardware. + +OpenVINO, short for Open Visual Inference & Neural Network Optimization toolkit, is a comprehensive toolkit for optimizing and deploying AI inference models. Even though the name contains Visual, OpenVINO also supports various additional tasks including language, audio, time series, etc. + +
+
+
+
+ Watch: How To Export and Optimize an Ultralytics YOLOv8 Model for Inference with OpenVINO.
+
+ +
+ +Developed by Baidu, [PaddlePaddle](https://www.paddlepaddle.org.cn/en) (**PA**rallel **D**istributed **D**eep **LE**arning) is China's first open-source deep learning platform. Unlike some frameworks built mainly for research, PaddlePaddle prioritizes ease of use and smooth integration across industries. + +It offers tools and resources similar to popular frameworks like TensorFlow and PyTorch, making it accessible for developers of all experience levels. From farming and factories to service businesses, PaddlePaddle's large developer community of over 4.77 million is helping create and deploy AI applications. + +By exporting your Ultralytics YOLOv8 models to PaddlePaddle format, you can tap into PaddlePaddle's strengths in performance optimization. PaddlePaddle prioritizes efficient model execution and reduced memory usage. As a result, your YOLOv8 models can potentially achieve even better performance, delivering top-notch results in practical scenarios. + +## Key Features of PaddlePaddle Models + +PaddlePaddle models offer a range of key features that contribute to their flexibility, performance, and scalability across diverse deployment scenarios: + +- **Dynamic-to-Static Graph**: PaddlePaddle supports [dynamic-to-static compilation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/jit/index_en.html), where models can be translated into a static computational graph. This enables optimizations that reduce runtime overhead and boost inference performance. + +- **Operator Fusion**: PaddlePaddle, like TensorRT, uses [operator fusion](https://developer.nvidia.com/gtc/2020/video/s21436-vid) to streamline computation and reduce overhead. The framework minimizes memory transfers and computational steps by merging compatible operations, resulting in faster inference. + +- **Quantization**: PaddlePaddle supports [quantization techniques](https://www.paddlepaddle.org.cn/documentation/docs/en/api/paddle/quantization/PTQ_en.html), including post-training quantization and quantization-aware training. These techniques allow for the use of lower-precision data representations, effectively boosting performance and reducing model size. + +## Deployment Options in PaddlePaddle + +Before diving into the code for exporting YOLOv8 models to PaddlePaddle, let's take a look at the different deployment scenarios in which PaddlePaddle models excel. + +PaddlePaddle provides a range of options, each offering a distinct balance of ease of use, flexibility, and performance: + +- **Paddle Serving**: This framework simplifies the deployment of PaddlePaddle models as high-performance RESTful APIs. Paddle Serving is ideal for production environments, providing features like model versioning, online A/B testing, and scalability for handling large volumes of requests. + +- **Paddle Inference API**: The Paddle Inference API gives you low-level control over model execution. This option is well-suited for scenarios where you need to integrate the model tightly within a custom application or optimize performance for specific hardware. + +- **Paddle Lite**: Paddle Lite is designed for deployment on mobile and embedded devices where resources are limited. It optimizes models for smaller sizes and faster inference on ARM CPUs, GPUs, and other specialized hardware. + +- **Paddle.js**: Paddle.js enables you to deploy PaddlePaddle models directly within web browsers. Paddle.js can either load a pre-trained model or transform a model from [paddle-hub](https://github.com/PaddlePaddle/PaddleHub) with model transforming tools provided by Paddle.js. It can run in browsers that support WebGL/WebGPU/WebAssembly. + +## Export to PaddlePaddle: Converting Your YOLOv8 Model + +Converting YOLOv8 models to the PaddlePaddle format can improve execution flexibility and optimize performance for various deployment scenarios. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to PaddlePaddle format + model.export(format="paddle") # creates '/yolov8n_paddle_model' + + # Load the exported PaddlePaddle model + paddle_model = YOLO("./yolov8n_paddle_model") + + # Run inference + results = paddle_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to PaddlePaddle format + yolo export model=yolov8n.pt format=paddle # creates '/yolov8n_paddle_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_paddle_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 PaddlePaddle Models + +After successfully exporting your Ultralytics YOLOv8 models to PaddlePaddle format, you can now deploy them. The primary and recommended first step for running a PaddlePaddle model is to use the YOLO("./model_paddle_model") method, as outlined in the previous usage code snippet. + +However, for in-depth instructions on deploying your PaddlePaddle models in various other settings, take a look at the following resources: + +- **[Paddle Serving](https://github.com/PaddlePaddle/Serving/blob/v0.9.0/README_CN.md)**: Learn how to deploy your PaddlePaddle models as performant services using Paddle Serving. + +- **[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite/blob/develop/README_en.md)**: Explore how to optimize and deploy models on mobile and embedded devices using Paddle Lite. + +- **[Paddle.js](https://github.com/PaddlePaddle/Paddle.js)**: Discover how to run PaddlePaddle models in web browsers for client-side AI using Paddle.js. + +## Summary + +In this guide, we explored the process of exporting Ultralytics YOLOv8 models to the PaddlePaddle format. By following these steps, you can leverage PaddlePaddle's strengths in diverse deployment scenarios, optimizing your models for different hardware and software environments. + +For further details on usage, visit the [PaddlePaddle official documentation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/index_en.html) + +Want to explore more ways to integrate your Ultralytics YOLOv8 models? Our [integration guide page](index.md) explores various options, equipping you with valuable resources and insights. + +## FAQ + +### How do I export Ultralytics YOLOv8 models to PaddlePaddle format? + +Exporting Ultralytics YOLOv8 models to PaddlePaddle format is straightforward. You can use the `export` method of the YOLO class to perform this exportation. Here is an example using Python: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to PaddlePaddle format + model.export(format="paddle") # creates '/yolov8n_paddle_model' + + # Load the exported PaddlePaddle model + paddle_model = YOLO("./yolov8n_paddle_model") + + # Run inference + results = paddle_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to PaddlePaddle format + yolo export model=yolov8n.pt format=paddle # creates '/yolov8n_paddle_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_paddle_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more detailed setup and troubleshooting, check the [Ultralytics Installation Guide](../quickstart.md) and [Common Issues Guide](../guides/yolo-common-issues.md). + +### What are the advantages of using PaddlePaddle for model deployment? + +PaddlePaddle offers several key advantages for model deployment: + +- **Performance Optimization**: PaddlePaddle excels in efficient model execution and reduced memory usage. +- **Dynamic-to-Static Graph Compilation**: It supports dynamic-to-static compilation, allowing for runtime optimizations. +- **Operator Fusion**: By merging compatible operations, it reduces computational overhead. +- **Quantization Techniques**: Supports both post-training and quantization-aware training, enabling lower-precision data representations for improved performance. + +You can achieve enhanced results by exporting your Ultralytics YOLOv8 models to PaddlePaddle, ensuring flexibility and high performance across various applications and hardware platforms. Learn more about PaddlePaddle's features [here](https://www.paddlepaddle.org.cn/en). + +### Why should I choose PaddlePaddle for deploying my YOLOv8 models? + +PaddlePaddle, developed by Baidu, is optimized for industrial and commercial AI deployments. Its large developer community and robust framework provide extensive tools similar to TensorFlow and PyTorch. By exporting your YOLOv8 models to PaddlePaddle, you leverage: + +- **Enhanced Performance**: Optimal execution speed and reduced memory footprint. +- **Flexibility**: Wide compatibility with various devices from smartphones to cloud servers. +- **Scalability**: Efficient parallel processing capabilities for distributed environments. + +These features make PaddlePaddle a compelling choice for deploying YOLOv8 models in production settings. + +### How does PaddlePaddle improve model performance over other frameworks? + +PaddlePaddle employs several advanced techniques to optimize model performance: + +- **Dynamic-to-Static Graph**: Converts models into a static computational graph for runtime optimizations. +- **Operator Fusion**: Combines compatible operations to minimize memory transfer and increase inference speed. +- **Quantization**: Reduces model size and increases efficiency using lower-precision data while maintaining accuracy. + +These techniques prioritize efficient model execution, making PaddlePaddle an excellent option for deploying high-performance YOLOv8 models. For more on optimization, see the [PaddlePaddle official documentation](https://www.paddlepaddle.org.cn/documentation/docs/en/guides/index_en.html). + +### What deployment options does PaddlePaddle offer for YOLOv8 models? + +PaddlePaddle provides flexible deployment options: + +- **Paddle Serving**: Deploys models as RESTful APIs, ideal for production with features like model versioning and online A/B testing. +- **Paddle Inference API**: Gives low-level control over model execution for custom applications. +- **Paddle Lite**: Optimizes models for mobile and embedded devices' limited resources. +- **Paddle.js**: Enables deploying models directly within web browsers. + +These options cover a broad range of deployment scenarios, from on-device inference to scalable cloud services. Explore more deployment strategies on the [Ultralytics Model Deployment Options page](../guides/model-deployment-options.md). diff --git a/ultralytics/docs/en/integrations/paperspace.md b/ultralytics/docs/en/integrations/paperspace.md new file mode 100644 index 0000000000000000000000000000000000000000..346fb06c6f529190ce8defbade0c8d4fb56f868c --- /dev/null +++ b/ultralytics/docs/en/integrations/paperspace.md @@ -0,0 +1,115 @@ +--- +comments: true +description: Simplify YOLOv8 training with Paperspace Gradient's all-in-one MLOps platform. Access GPUs, automate workflows, and deploy with ease. +keywords: YOLOv8, Paperspace Gradient, MLOps, machine learning, training, GPUs, Jupyter notebooks, model deployment, AI, cloud platform +--- + +# YOLOv8 Model Training Made Simple with Paperspace Gradient + +Training computer vision models like [YOLOv8](https://github.com/ultralytics/ultralytics) can be complicated. It involves managing large datasets, using different types of computer hardware like GPUs, TPUs, and CPUs, and making sure data flows smoothly during the training process. Typically, developers end up spending a lot of time managing their computer systems and environments. It can be frustrating when you just want to focus on building the best model. + +This is where a platform like Paperspace Gradient can make things simpler. Paperspace Gradient is a MLOps platform that lets you build, train, and deploy machine learning models all in one place. With Gradient, developers can focus on training their YOLOv8 models without the hassle of managing infrastructure and environments. + +## Paperspace + ++ +
+ +[Paperspace](https://www.paperspace.com/), launched in 2014 by University of Michigan graduates and acquired by DigitalOcean in 2023, is a cloud platform specifically designed for machine learning. It provides users with powerful GPUs, collaborative Jupyter notebooks, a container service for deployments, automated workflows for machine learning tasks, and high-performance virtual machines. These features aim to streamline the entire machine learning development process, from coding to deployment. + +## Paperspace Gradient + ++ +
+ +Paperspace Gradient is a suite of tools designed to make working with AI and machine learning in the cloud much faster and easier. Gradient addresses the entire machine learning development process, from building and training models to deploying them. + +Within its toolkit, it includes support for Google's TPUs via a job runner, comprehensive support for Jupyter notebooks and containers, and new programming language integrations. Its focus on language integration particularly stands out, allowing users to easily adapt their existing Python projects to use the most advanced GPU infrastructure available. + +## Training YOLOv8 Using Paperspace Gradient + +Paperspace Gradient makes training a YOLOv8 model possible with a few clicks. Thanks to the integration, you can access the [Paperspace console](https://console.paperspace.com/github/ultralytics/ultralytics) and start training your model immediately. For a detailed understanding of the model training process and best practices, refer to our [YOLOv8 Model Training guide](../modes/train.md). + +Sign in and then click on the “Start Machine” button shown in the image below. In a few seconds, a managed GPU environment will start up, and then you can run the notebook's cells. + +![Training YOLOv8 Using Paperspace Gradient](https://github.com/ultralytics/ultralytics/assets/25847604/d6bd083a-cde3-427f-8ac3-b80fa49f6ba1) + +Explore more capabilities of YOLOv8 and Paperspace Gradient in a discussion with Glenn Jocher, Ultralytics founder, and James Skelton from Paperspace. Watch the discussion below. + +
+
+
+
+ Watch: Ultralytics Live Session 7: It's All About the Environment: Optimizing YOLOv8 Training With Gradient
+
+ +
+ +[Ray Tune](https://docs.ray.io/en/latest/tune/index.html) is a hyperparameter tuning library designed for efficiency and flexibility. It supports various search strategies, parallelism, and early stopping strategies, and seamlessly integrates with popular machine learning frameworks, including Ultralytics YOLOv8. + +### Integration with Weights & Biases + +YOLOv8 also allows optional integration with [Weights & Biases](https://wandb.ai/site) for monitoring the tuning process. + +## Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install and update Ultralytics and Ray Tune packages + pip install -U ultralytics "ray[tune]" + + # Optionally install W&B for logging + pip install wandb + ``` + +## Usage + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a YOLOv8n model + model = YOLO("yolov8n.pt") + + # Start tuning hyperparameters for YOLOv8n training on the COCO8 dataset + result_grid = model.tune(data="coco8.yaml", use_ray=True) + ``` + +## `tune()` Method Parameters + +The `tune()` method in YOLOv8 provides an easy-to-use interface for hyperparameter tuning with Ray Tune. It accepts several arguments that allow you to customize the tuning process. Below is a detailed explanation of each parameter: + +| Parameter | Type | Description | Default Value | +| --------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | +| `data` | `str` | The dataset configuration file (in YAML format) to run the tuner on. This file should specify the training and validation data paths, as well as other dataset-specific settings. | | +| `space` | `dict, optional` | A dictionary defining the hyperparameter search space for Ray Tune. Each key corresponds to a hyperparameter name, and the value specifies the range of values to explore during tuning. If not provided, YOLOv8 uses a default search space with various hyperparameters. | | +| `grace_period` | `int, optional` | The grace period in epochs for the [ASHA scheduler](https://docs.ray.io/en/latest/tune/api/schedulers.html) in Ray Tune. The scheduler will not terminate any trial before this number of epochs, allowing the model to have some minimum training before making a decision on early stopping. | 10 | +| `gpu_per_trial` | `int, optional` | The number of GPUs to allocate per trial during tuning. This helps manage GPU usage, particularly in multi-GPU environments. If not provided, the tuner will use all available GPUs. | None | +| `iterations` | `int, optional` | The maximum number of trials to run during tuning. This parameter helps control the total number of hyperparameter combinations tested, ensuring the tuning process does not run indefinitely. | 10 | +| `**train_args` | `dict, optional` | Additional arguments to pass to the `train()` method during tuning. These arguments can include settings like the number of training epochs, batch size, and other training-specific configurations. | {} | + +By customizing these parameters, you can fine-tune the hyperparameter optimization process to suit your specific needs and available computational resources. + +## Default Search Space Description + +The following table lists the default search space parameters for hyperparameter tuning in YOLOv8 with Ray Tune. Each parameter has a specific value range defined by `tune.uniform()`. + +| Parameter | Value Range | Description | +| ----------------- | -------------------------- | ---------------------------------------- | +| `lr0` | `tune.uniform(1e-5, 1e-1)` | Initial learning rate | +| `lrf` | `tune.uniform(0.01, 1.0)` | Final learning rate factor | +| `momentum` | `tune.uniform(0.6, 0.98)` | Momentum | +| `weight_decay` | `tune.uniform(0.0, 0.001)` | Weight decay | +| `warmup_epochs` | `tune.uniform(0.0, 5.0)` | Warmup epochs | +| `warmup_momentum` | `tune.uniform(0.0, 0.95)` | Warmup momentum | +| `box` | `tune.uniform(0.02, 0.2)` | Box loss weight | +| `cls` | `tune.uniform(0.2, 4.0)` | Class loss weight | +| `hsv_h` | `tune.uniform(0.0, 0.1)` | Hue augmentation range | +| `hsv_s` | `tune.uniform(0.0, 0.9)` | Saturation augmentation range | +| `hsv_v` | `tune.uniform(0.0, 0.9)` | Value (brightness) augmentation range | +| `degrees` | `tune.uniform(0.0, 45.0)` | Rotation augmentation range (degrees) | +| `translate` | `tune.uniform(0.0, 0.9)` | Translation augmentation range | +| `scale` | `tune.uniform(0.0, 0.9)` | Scaling augmentation range | +| `shear` | `tune.uniform(0.0, 10.0)` | Shear augmentation range (degrees) | +| `perspective` | `tune.uniform(0.0, 0.001)` | Perspective augmentation range | +| `flipud` | `tune.uniform(0.0, 1.0)` | Vertical flip augmentation probability | +| `fliplr` | `tune.uniform(0.0, 1.0)` | Horizontal flip augmentation probability | +| `mosaic` | `tune.uniform(0.0, 1.0)` | Mosaic augmentation probability | +| `mixup` | `tune.uniform(0.0, 1.0)` | Mixup augmentation probability | +| `copy_paste` | `tune.uniform(0.0, 1.0)` | Copy-paste augmentation probability | + +## Custom Search Space Example + +In this example, we demonstrate how to use a custom search space for hyperparameter tuning with Ray Tune and YOLOv8. By providing a custom search space, you can focus the tuning process on specific hyperparameters of interest. + +!!! Example "Usage" + + ```python + from ultralytics import YOLO + + # Define a YOLO model + model = YOLO("yolov8n.pt") + + # Run Ray Tune on the model + result_grid = model.tune( + data="coco8.yaml", + space={"lr0": tune.uniform(1e-5, 1e-1)}, + epochs=50, + use_ray=True, + ) + ``` + +In the code snippet above, we create a YOLO model with the "yolov8n.pt" pretrained weights. Then, we call the `tune()` method, specifying the dataset configuration with "coco8.yaml". We provide a custom search space for the initial learning rate `lr0` using a dictionary with the key "lr0" and the value `tune.uniform(1e-5, 1e-1)`. Finally, we pass additional training arguments, such as the number of epochs directly to the tune method as `epochs=50`. + +## Processing Ray Tune Results + +After running a hyperparameter tuning experiment with Ray Tune, you might want to perform various analyses on the obtained results. This guide will take you through common workflows for processing and analyzing these results. + +### Loading Tune Experiment Results from a Directory + +After running the tuning experiment with `tuner.fit()`, you can load the results from a directory. This is useful, especially if you're performing the analysis after the initial training script has exited. + +```python +experiment_path = f"{storage_path}/{exp_name}" +print(f"Loading results from {experiment_path}...") + +restored_tuner = tune.Tuner.restore(experiment_path, trainable=train_mnist) +result_grid = restored_tuner.get_results() +``` + +### Basic Experiment-Level Analysis + +Get an overview of how trials performed. You can quickly check if there were any errors during the trials. + +```python +if result_grid.errors: + print("One or more trials failed!") +else: + print("No errors!") +``` + +### Basic Trial-Level Analysis + +Access individual trial hyperparameter configurations and the last reported metrics. + +```python +for i, result in enumerate(result_grid): + print(f"Trial #{i}: Configuration: {result.config}, Last Reported Metrics: {result.metrics}") +``` + +### Plotting the Entire History of Reported Metrics for a Trial + +You can plot the history of reported metrics for each trial to see how the metrics evolved over time. + +```python +import matplotlib.pyplot as plt + +for i, result in enumerate(result_grid): + plt.plot( + result.metrics_dataframe["training_iteration"], + result.metrics_dataframe["mean_accuracy"], + label=f"Trial {i}", + ) + +plt.xlabel("Training Iterations") +plt.ylabel("Mean Accuracy") +plt.legend() +plt.show() +``` + +## Summary + +In this documentation, we covered common workflows to analyze the results of experiments run with Ray Tune using Ultralytics. The key steps include loading the experiment results from a directory, performing basic experiment-level and trial-level analysis and plotting metrics. + +Explore further by looking into Ray Tune's [Analyze Results](https://docs.ray.io/en/latest/tune/examples/tune_analyze_results.html) docs page to get the most out of your hyperparameter tuning experiments. + +## FAQ + +### How do I tune the hyperparameters of my YOLOv8 model using Ray Tune? + +To tune the hyperparameters of your Ultralytics YOLOv8 model using Ray Tune, follow these steps: + +1. **Install the required packages:** + + ```bash + pip install -U ultralytics "ray[tune]" + pip install wandb # optional for logging + ``` + +2. **Load your YOLOv8 model and start tuning:** + + ```python + from ultralytics import YOLO + + # Load a YOLOv8 model + model = YOLO("yolov8n.pt") + + # Start tuning with the COCO8 dataset + result_grid = model.tune(data="coco8.yaml", use_ray=True) + ``` + +This utilizes Ray Tune's advanced search strategies and parallelism to efficiently optimize your model's hyperparameters. For more information, check out the [Ray Tune documentation](https://docs.ray.io/en/latest/tune/index.html). + +### What are the default hyperparameters for YOLOv8 tuning with Ray Tune? + +Ultralytics YOLOv8 uses the following default hyperparameters for tuning with Ray Tune: + +| Parameter | Value Range | Description | +| --------------- | -------------------------- | ------------------------------ | +| `lr0` | `tune.uniform(1e-5, 1e-1)` | Initial learning rate | +| `lrf` | `tune.uniform(0.01, 1.0)` | Final learning rate factor | +| `momentum` | `tune.uniform(0.6, 0.98)` | Momentum | +| `weight_decay` | `tune.uniform(0.0, 0.001)` | Weight decay | +| `warmup_epochs` | `tune.uniform(0.0, 5.0)` | Warmup epochs | +| `box` | `tune.uniform(0.02, 0.2)` | Box loss weight | +| `cls` | `tune.uniform(0.2, 4.0)` | Class loss weight | +| `hsv_h` | `tune.uniform(0.0, 0.1)` | Hue augmentation range | +| `translate` | `tune.uniform(0.0, 0.9)` | Translation augmentation range | + +These hyperparameters can be customized to suit your specific needs. For a complete list and more details, refer to the [Hyperparameter Tuning](../guides/hyperparameter-tuning.md) guide. + +### How can I integrate Weights & Biases with my YOLOv8 model tuning? + +To integrate Weights & Biases (W&B) with your Ultralytics YOLOv8 tuning process: + +1. **Install W&B:** + + ```bash + pip install wandb + ``` + +2. **Modify your tuning script:** + + ```python + import wandb + + from ultralytics import YOLO + + wandb.init(project="YOLO-Tuning", entity="your-entity") + + # Load YOLO model + model = YOLO("yolov8n.pt") + + # Tune hyperparameters + result_grid = model.tune(data="coco8.yaml", use_ray=True) + ``` + +This setup will allow you to monitor the tuning process, track hyperparameter configurations, and visualize results in W&B. + +### Why should I use Ray Tune for hyperparameter optimization with YOLOv8? + +Ray Tune offers numerous advantages for hyperparameter optimization: + +- **Advanced Search Strategies:** Utilizes algorithms like Bayesian Optimization and HyperOpt for efficient parameter search. +- **Parallelism:** Supports parallel execution of multiple trials, significantly speeding up the tuning process. +- **Early Stopping:** Employs strategies like ASHA to terminate under-performing trials early, saving computational resources. + +Ray Tune seamlessly integrates with Ultralytics YOLOv8, providing an easy-to-use interface for tuning hyperparameters effectively. To get started, check out the [Efficient Hyperparameter Tuning with Ray Tune and YOLOv8](../guides/hyperparameter-tuning.md) guide. + +### How can I define a custom search space for YOLOv8 hyperparameter tuning? + +To define a custom search space for your YOLOv8 hyperparameter tuning with Ray Tune: + +```python +from ray import tune + +from ultralytics import YOLO + +model = YOLO("yolov8n.pt") +search_space = {"lr0": tune.uniform(1e-5, 1e-1), "momentum": tune.uniform(0.6, 0.98)} +result_grid = model.tune(data="coco8.yaml", space=search_space, use_ray=True) +``` + +This customizes the range of hyperparameters like initial learning rate and momentum to be explored during the tuning process. For advanced configurations, refer to the [Custom Search Space Example](#custom-search-space-example) section. diff --git a/ultralytics/docs/en/integrations/roboflow.md b/ultralytics/docs/en/integrations/roboflow.md new file mode 100644 index 0000000000000000000000000000000000000000..af3a0015a6df0fea82eff819d9960d2bfb1330c9 --- /dev/null +++ b/ultralytics/docs/en/integrations/roboflow.md @@ -0,0 +1,269 @@ +--- +comments: true +description: Learn how to gather, label, and deploy data for custom YOLOv8 models using Roboflow's powerful tools. Optimize your computer vision pipeline effortlessly. +keywords: Roboflow, YOLOv8, data labeling, computer vision, model training, model deployment, dataset management, automated image annotation, AI tools +--- + +# Roboflow + +[Roboflow](https://roboflow.com/?ref=ultralytics) has everything you need to build and deploy computer vision models. Connect Roboflow at any step in your pipeline with APIs and SDKs, or use the end-to-end interface to automate the entire process from image to inference. Whether you're in need of [data labeling](https://roboflow.com/annotate?ref=ultralytics), [model training](https://roboflow.com/train?ref=ultralytics), or [model deployment](https://roboflow.com/deploy?ref=ultralytics), Roboflow gives you building blocks to bring custom computer vision solutions to your project. + +!!! Question "Licensing" + + Ultralytics offers two licensing options: + + - The [AGPL-3.0 License](https://github.com/ultralytics/ultralytics/blob/main/LICENSE), an [OSI-approved](https://opensource.org/licenses/) open-source license ideal for students and enthusiasts. + - The [Enterprise License](https://ultralytics.com/license) for businesses seeking to incorporate our AI models into their products and services. + + For more details see [Ultralytics Licensing](https://ultralytics.com/license). + +In this guide, we are going to showcase how to find, label, and organize data for use in training a custom Ultralytics YOLOv8 model. Use the table of contents below to jump directly to a specific section: + +- Gather data for training a custom YOLOv8 model +- Upload, convert and label data for YOLOv8 format +- Pre-process and augment data for model robustness +- Dataset management for [YOLOv8](../models/yolov8.md) +- Export data in 40+ formats for model training +- Upload custom YOLOv8 model weights for testing and deployment +- Gather Data for Training a Custom YOLOv8 Model + +Roboflow provides two services that can help you collect data for YOLOv8 models: [Universe](https://universe.roboflow.com/?ref=ultralytics) and [Collect](https://roboflow.com/collect?ref=ultralytics). + +Universe is an online repository with over 250,000 vision datasets totalling over 100 million images. + ++ +
+ +With a [free Roboflow account](https://app.roboflow.com/?ref=ultralytics), you can export any dataset available on Universe. To export a dataset, click the "Download this Dataset" button on any dataset. + ++ +
+ +For YOLOv8, select "YOLOv8" as the export format: + ++ +
+ +Universe also has a page that aggregates all [public fine-tuned YOLOv8 models uploaded to Roboflow](https://universe.roboflow.com/search?q=model:yolov8). You can use this page to explore pre-trained models you can use for testing or [for automated data labeling](https://docs.roboflow.com/annotate/use-roboflow-annotate/model-assisted-labeling) or to prototype with [Roboflow inference](https://roboflow.com/inference?ref=ultralytics). + +If you want to gather images yourself, try [Collect](https://github.com/roboflow/roboflow-collect), an open source project that allows you to automatically gather images using a webcam on the edge. You can use text or image prompts with Collect to instruct what data should be collected, allowing you to capture only the useful data you need to build your vision model. + +## Upload, Convert and Label Data for YOLOv8 Format + +[Roboflow Annotate](https://docs.roboflow.com/annotate/use-roboflow-annotate) is an online annotation tool for use in labeling images for object detection, classification, and segmentation. + +To label data for a YOLOv8 object detection, instance segmentation, or classification model, first create a project in Roboflow. + ++ +
+ +Next, upload your images, and any pre-existing annotations you have from other tools ([using one of the 40+ supported import formats](https://roboflow.com/formats?ref=ultralytics)), into Roboflow. + ++ +
+ +Select the batch of images you have uploaded on the Annotate page to which you are taken after uploading images. Then, click "Start Annotating" to label images. + +To label with bounding boxes, press the `B` key on your keyboard or click the box icon in the sidebar. Click on a point where you want to start your bounding box, then drag to create the box: + ++ +
+ +A pop-up will appear asking you to select a class for your annotation once you have created an annotation. + +To label with polygons, press the `P` key on your keyboard, or the polygon icon in the sidebar. With the polygon annotation tool enabled, click on individual points in the image to draw a polygon. + +Roboflow offers a SAM-based label assistant with which you can label images faster than ever. SAM (Segment Anything Model) is a state-of-the-art computer vision model that can precisely label images. With SAM, you can significantly speed up the image labeling process. Annotating images with polygons becomes as simple as a few clicks, rather than the tedious process of precisely clicking points around an object. + +To use the label assistant, click the cursor icon in the sidebar, SAM will be loaded for use in your project. + ++ +
+ +Hover over any object in the image and SAM will recommend an annotation. You can hover to find the right place to annotate, then click to create your annotation. To amend your annotation to be more or less specific, you can click inside or outside the annotation SAM has created on the document. + +You can also add tags to images from the Tags panel in the sidebar. You can apply tags to data from a particular area, taken from a specific camera, and more. You can then use these tags to search through data for images matching a tag and generate versions of a dataset with images that contain a particular tag or set of tags. + ++ +
+ +Models hosted on Roboflow can be used with Label Assist, an automated annotation tool that uses your YOLOv8 model to recommend annotations. To use Label Assist, first upload a YOLOv8 model to Roboflow (see instructions later in the guide). Then, click the magic wand icon in the left sidebar and select your model for use in Label Assist. + +Choose a model, then click "Continue" to enable Label Assist: + ++ +
+ +When you open new images for annotation, Label Assist will trigger and recommend annotations. + ++ +
+ +## Dataset Management for YOLOv8 + +Roboflow provides a suite of tools for understanding computer vision datasets. + +First, you can use dataset search to find images that meet a semantic text description (i.e. find all images that contain people), or that meet a specified label (i.e. the image is associated with a specific tag). To use dataset search, click "Dataset" in the sidebar. Then, input a search query using the search bar and associated filters at the top of the page. + +For example, the following text query finds images that contain people in a dataset: + ++ +
+ +You can narrow your search to images with a particular tag using the "Tags" selector: + ++ +
+ +Before you start training a model with your dataset, we recommend using Roboflow [Health Check](https://docs.roboflow.com/datasets/dataset-health-check), a web tool that provides an insight into your dataset and how you can improve the dataset prior to training a vision model. + +To use Health Check, click the "Health Check" sidebar link. A list of statistics will appear that show the average size of images in your dataset, class balance, a heatmap of where annotations are in your images, and more. + ++ +
+ +Health Check may recommend changes to help enhance dataset performance. For example, the class balance feature may show that there is an imbalance in labels that, if solved, may boost performance or your model. + +## Export Data in 40+ Formats for Model Training + +To export your data, you will need a dataset version. A version is a state of your dataset frozen-in-time. To create a version, first click "Versions" in the sidebar. Then, click the "Create New Version" button. On this page, you will be able to choose augmentations and preprocessing steps to apply to your dataset: + ++ +
+ +For each augmentation you select, a pop-up will appear allowing you to tune the augmentation to your needs. Here is an example of tuning a brightness augmentation within specified parameters: + ++ +
+ +When your dataset version has been generated, you can export your data into a range of formats. Click the "Export Dataset" button on your dataset version page to export your data: + ++ +
+ +You are now ready to train YOLOv8 on a custom dataset. Follow this [written guide](https://blog.roboflow.com/how-to-train-yolov8-on-a-custom-dataset/) and [YouTube video](https://www.youtube.com/watch?v=wuZtUMEiKWY) for step-by-step instructions or refer to the [Ultralytics documentation](../modes/train.md). + +## Upload Custom YOLOv8 Model Weights for Testing and Deployment + +Roboflow offers an infinitely scalable API for deployed models and SDKs for use with NVIDIA Jetsons, Luxonis OAKs, Raspberry Pis, GPU-based devices, and more. + +You can deploy YOLOv8 models by uploading YOLOv8 weights to Roboflow. You can do this in a few lines of Python code. Create a new Python file and add the following code: + +```python +import roboflow # install with 'pip install roboflow' + +roboflow.login() + +rf = roboflow.Roboflow() + +project = rf.workspace(WORKSPACE_ID).project("football-players-detection-3zvbc") +dataset = project.version(VERSION).download("yolov8") + +project.version(dataset.version).deploy(model_type="yolov8", model_path=f"{HOME}/runs/detect/train/") +``` + +In this code, replace the project ID and version ID with the values for your account and project. [Learn how to retrieve your Roboflow API key](https://docs.roboflow.com/api-reference/authentication#retrieve-an-api-key). + +When you run the code above, you will be asked to authenticate. Then, your model will be uploaded and an API will be created for your project. This process can take up to 30 minutes to complete. + +To test your model and find deployment instructions for supported SDKs, go to the "Deploy" tab in the Roboflow sidebar. At the top of this page, a widget will appear with which you can test your model. You can use your webcam for live testing or upload images or videos. + ++ +
+ +You can also use your uploaded model as a [labeling assistant](https://docs.roboflow.com/annotate/use-roboflow-annotate/model-assisted-labeling). This feature uses your trained model to recommend annotations on images uploaded to Roboflow. + +## How to Evaluate YOLOv8 Models + +Roboflow provides a range of features for use in evaluating models. + +Once you have uploaded a model to Roboflow, you can access our model evaluation tool, which provides a confusion matrix showing the performance of your model as well as an interactive vector analysis plot. These features can help you find opportunities to improve your model. + +To access a confusion matrix, go to your model page on the Roboflow dashboard, then click "View Detailed Evaluation": + ++ +
+ +A pop-up will appear showing a confusion matrix: + ++ +
+ +Hover over a box on the confusion matrix to see the value associated with the box. Click on a box to see images in the respective category. Click on an image to view the model predictions and ground truth data associated with that image. + +For more insights, click Vector Analysis. This will show a scatter plot of the images in your dataset, calculated using CLIP. The closer images are in the plot, the more similar they are, semantically. Each image is represented as a dot with a color between white and red. The more red the dot, the worse the model performed. + ++ +
+ +You can use Vector Analysis to: + +- Find clusters of images; +- Identify clusters where the model performs poorly, and; +- Visualize commonalities between images on which the model performs poorly. + +## Learning Resources + +Want to learn more about using Roboflow for creating YOLOv8 models? The following resources may be helpful in your work. + +- [Train YOLOv8 on a Custom Dataset](https://github.com/roboflow/notebooks/blob/main/notebooks/train-yolov8-object-detection-on-custom-dataset.ipynb): Follow our interactive notebook that shows you how to train a YOLOv8 model on a custom dataset. +- [Autodistill](https://autodistill.github.io/autodistill/): Use large foundation vision models to label data for specific models. You can label images for use in training YOLOv8 classification, detection, and segmentation models with Autodistill. +- [Supervision](https://roboflow.github.io/supervision/): A Python package with helpful utilities for use in working with computer vision models. You can use supervision to filter detections, compute confusion matrices, and more, all in a few lines of Python code. +- [Roboflow Blog](https://blog.roboflow.com/): The Roboflow Blog features over 500 articles on computer vision, covering topics from how to train a YOLOv8 model to annotation best practices. +- [Roboflow YouTube channel](https://www.youtube.com/@Roboflow): Browse dozens of in-depth computer vision guides on our YouTube channel, covering topics from training YOLOv8 models to automated image labeling. + +## Project Showcase + +Below are a few of the many pieces of feedback we have received for using YOLOv8 and Roboflow together to create computer vision models. + ++ + + +
+ +## FAQ + +### How do I label data for YOLOv8 models using Roboflow? + +Labeling data for YOLOv8 models using Roboflow is straightforward with Roboflow Annotate. First, create a project on Roboflow and upload your images. After uploading, select the batch of images and click "Start Annotating." You can use the `B` key for bounding boxes or the `P` key for polygons. For faster annotation, use the SAM-based label assistant by clicking the cursor icon in the sidebar. Detailed steps can be found [here](#upload-convert-and-label-data-for-yolov8-format). + +### What services does Roboflow offer for collecting YOLOv8 training data? + +Roboflow provides two key services for collecting YOLOv8 training data: [Universe](https://universe.roboflow.com/?ref=ultralytics) and [Collect](https://roboflow.com/collect?ref=ultralytics). Universe offers access to over 250,000 vision datasets, while Collect helps you gather images using a webcam and automated prompts. + +### How can I manage and analyze my YOLOv8 dataset using Roboflow? + +Roboflow offers robust dataset management tools, including dataset search, tagging, and Health Check. Use the search feature to find images based on text descriptions or tags. Health Check provides insights into dataset quality, showing class balance, image sizes, and annotation heatmaps. This helps optimize dataset performance before training YOLOv8 models. Detailed information can be found [here](#dataset-management-for-yolov8). + +### How do I export my YOLOv8 dataset from Roboflow? + +To export your YOLOv8 dataset from Roboflow, you need to create a dataset version. Click "Versions" in the sidebar, then "Create New Version" and apply any desired augmentations. Once the version is generated, click "Export Dataset" and choose the YOLOv8 format. Follow this process [here](#export-data-in-40-formats-for-model-training). + +### How can I integrate and deploy YOLOv8 models with Roboflow? + +Integrate and deploy YOLOv8 models on Roboflow by uploading your YOLOv8 weights through a few lines of Python code. Use the provided script to authenticate and upload your model, which will create an API for deployment. For details on the script and further instructions, see [this section](#upload-custom-yolov8-model-weights-for-testing-and-deployment). + +### What tools does Roboflow provide for evaluating YOLOv8 models? + +Roboflow offers model evaluation tools, including a confusion matrix and vector analysis plots. Access these tools from the "View Detailed Evaluation" button on your model page. These features help identify model performance issues and find areas for improvement. For more information, refer to [this section](#how-to-evaluate-yolov8-models). diff --git a/ultralytics/docs/en/integrations/tensorboard.md b/ultralytics/docs/en/integrations/tensorboard.md new file mode 100644 index 0000000000000000000000000000000000000000..752ad4e7f7fa52c8d83c6571166a690f8be65d88 --- /dev/null +++ b/ultralytics/docs/en/integrations/tensorboard.md @@ -0,0 +1,213 @@ +--- +comments: true +description: Learn how to integrate YOLOv8 with TensorBoard for real-time visual insights into your model's training metrics, performance graphs, and debugging workflows. +keywords: YOLOv8, TensorBoard, model training, visualization, machine learning, deep learning, Ultralytics, training metrics, performance analysis +--- + +# Gain Visual Insights with YOLOv8's Integration with TensorBoard + +Understanding and fine-tuning computer vision models like [Ultralytics' YOLOv8](https://ultralytics.com) becomes more straightforward when you take a closer look at their training processes. Model training visualization helps with getting insights into the model's learning patterns, performance metrics, and overall behavior. YOLOv8's integration with TensorBoard makes this process of visualization and analysis easier and enables more efficient and informed adjustments to the model. + +This guide covers how to use TensorBoard with YOLOv8. You'll learn about various visualizations, from tracking metrics to analyzing model graphs. These tools will help you understand your YOLOv8 model's performance better. + +## TensorBoard + ++ +
+ +[TensorBoard](https://www.tensorflow.org/tensorboard), TensorFlow's visualization toolkit, is essential for machine learning experimentation. TensorBoard features a range of visualization tools, crucial for monitoring machine learning models. These tools include tracking key metrics like loss and accuracy, visualizing model graphs, and viewing histograms of weights and biases over time. It also provides capabilities for projecting embeddings to lower-dimensional spaces and displaying multimedia data. + +## YOLOv8 Training with TensorBoard + +Using TensorBoard while training YOLOv8 models is straightforward and offers significant benefits. + +## Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 and Tensorboard + pip install ultralytics + ``` + +TensorBoard is conveniently pre-installed with YOLOv8, eliminating the need for additional setup for visualization purposes. + +For detailed instructions and best practices related to the installation process, be sure to check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +## Configuring TensorBoard for Google Colab + +When using Google Colab, it's important to set up TensorBoard before starting your training code: + +!!! Example "Configure TensorBoard for Google Colab" + + === "Python" + + ```ipython + %load_ext tensorboard + %tensorboard --logdir path/to/runs + ``` + +## Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load a pre-trained model + model = YOLO("yolov8n.pt") + + # Train the model + results = model.train(data="coco8.yaml", epochs=100, imgsz=640) + ``` + +Upon running the usage code snippet above, you can expect the following output: + +```bash +TensorBoard: Start with 'tensorboard --logdir path_to_your_tensorboard_logs', view at http://localhost:6006/ +``` + +This output indicates that TensorBoard is now actively monitoring your YOLOv8 training session. You can access the TensorBoard dashboard by visiting the provided URL (http://localhost:6006/) to view real-time training metrics and model performance. For users working in Google Colab, the TensorBoard will be displayed in the same cell where you executed the TensorBoard configuration commands. + +For more information related to the model training process, be sure to check our [YOLOv8 Model Training guide](../modes/train.md). If you are interested in learning more about logging, checkpoints, plotting, and file management, read our [usage guide on configuration](../usage/cfg.md). + +## Understanding Your TensorBoard for YOLOv8 Training + +Now, let's focus on understanding the various features and components of TensorBoard in the context of YOLOv8 training. The three key sections of the TensorBoard are Time Series, Scalars, and Graphs. + +### Time Series + +The Time Series feature in the TensorBoard offers a dynamic and detailed perspective of various training metrics over time for YOLOv8 models. It focuses on the progression and trends of metrics across training epochs. Here's an example of what you can expect to see. + +![image](https://github.com/ultralytics/ultralytics/assets/25847604/20b3e038-0356-465e-a37e-1ea232c68354) + +#### Key Features of Time Series in TensorBoard + +- **Filter Tags and Pinned Cards**: This functionality allows users to filter specific metrics and pin cards for quick comparison and access. It's particularly useful for focusing on specific aspects of the training process. + +- **Detailed Metric Cards**: Time Series divides metrics into different categories like learning rate (lr), training (train), and validation (val) metrics, each represented by individual cards. + +- **Graphical Display**: Each card in the Time Series section shows a detailed graph of a specific metric over the course of training. This visual representation aids in identifying trends, patterns, or anomalies in the training process. + +- **In-Depth Analysis**: Time Series provides an in-depth analysis of each metric. For instance, different learning rate segments are shown, offering insights into how adjustments in learning rate impact the model's learning curve. + +#### Importance of Time Series in YOLOv8 Training + +The Time Series section is essential for a thorough analysis of the YOLOv8 model's training progress. It lets you track the metrics in real time to promptly identify and solve issues. It also offers a detailed view of each metrics progression, which is crucial for fine-tuning the model and enhancing its performance. + +### Scalars + +Scalars in the TensorBoard are crucial for plotting and analyzing simple metrics like loss and accuracy during the training of YOLOv8 models. They offer a clear and concise view of how these metrics evolve with each training epoch, providing insights into the model's learning effectiveness and stability. Here's an example of what you can expect to see. + +![image](https://github.com/ultralytics/ultralytics/assets/25847604/f9228193-13e9-4768-9edf-8fa15ecd24fa) + +#### Key Features of Scalars in TensorBoard + +- **Learning Rate (lr) Tags**: These tags show the variations in the learning rate across different segments (e.g., `pg0`, `pg1`, `pg2`). This helps us understand the impact of learning rate adjustments on the training process. + +- **Metrics Tags**: Scalars include performance indicators such as: + + - `mAP50 (B)`: Mean Average Precision at 50% Intersection over Union (IoU), crucial for assessing object detection accuracy. + + - `mAP50-95 (B)`: Mean Average Precision calculated over a range of IoU thresholds, offering a more comprehensive evaluation of accuracy. + + - `Precision (B)`: Indicates the ratio of correctly predicted positive observations, key to understanding prediction accuracy. + + - `Recall (B)`: Important for models where missing a detection is significant, this metric measures the ability to detect all relevant instances. + + - To learn more about the different metrics, read our guide on [performance metrics](../guides/yolo-performance-metrics.md). + +- **Training and Validation Tags (`train`, `val`)**: These tags display metrics specifically for the training and validation datasets, allowing for a comparative analysis of model performance across different data sets. + +#### Importance of Monitoring Scalars + +Observing scalar metrics is crucial for fine-tuning the YOLOv8 model. Variations in these metrics, such as spikes or irregular patterns in loss graphs, can highlight potential issues such as overfitting, underfitting, or inappropriate learning rate settings. By closely monitoring these scalars, you can make informed decisions to optimize the training process, ensuring that the model learns effectively and achieves the desired performance. + +### Difference Between Scalars and Time Series + +While both Scalars and Time Series in TensorBoard are used for tracking metrics, they serve slightly different purposes. Scalars focus on plotting simple metrics such as loss and accuracy as scalar values. They provide a high-level overview of how these metrics change with each training epoch. While, the time-series section of the TensorBoard offers a more detailed timeline view of various metrics. It is particularly useful for monitoring the progression and trends of metrics over time, providing a deeper dive into the specifics of the training process. + +### Graphs + +The Graphs section of the TensorBoard visualizes the computational graph of the YOLOv8 model, showing how operations and data flow within the model. It's a powerful tool for understanding the model's structure, ensuring that all layers are connected correctly, and for identifying any potential bottlenecks in data flow. Here's an example of what you can expect to see. + +![image](https://github.com/ultralytics/ultralytics/assets/25847604/039028e0-4ab3-4170-bfa8-f93ce483f615) + +Graphs are particularly useful for debugging the model, especially in complex architectures typical in deep learning models like YOLOv8. They help in verifying layer connections and the overall design of the model. + +## Summary + +This guide aims to help you use TensorBoard with YOLOv8 for visualization and analysis of machine learning model training. It focuses on explaining how key TensorBoard features can provide insights into training metrics and model performance during YOLOv8 training sessions. + +For a more detailed exploration of these features and effective utilization strategies, you can refer to TensorFlow's official [TensorBoard documentation](https://www.tensorflow.org/tensorboard/get_started) and their [GitHub repository](https://github.com/tensorflow/tensorboard). + +Want to learn more about the various integrations of Ultralytics? Check out the [Ultralytics integrations guide page](../integrations/index.md) to see what other exciting capabilities are waiting to be discovered! + +## FAQ + +### What benefits does using TensorBoard with YOLOv8 offer? + +Using TensorBoard with YOLOv8 provides several visualization tools essential for efficient model training: + +- **Real-Time Metrics Tracking:** Track key metrics such as loss, accuracy, precision, and recall live. +- **Model Graph Visualization:** Understand and debug the model architecture by visualizing computational graphs. +- **Embedding Visualization:** Project embeddings to lower-dimensional spaces for better insight. + +These tools enable you to make informed adjustments to enhance your YOLOv8 model's performance. For more details on TensorBoard features, check out the TensorFlow [TensorBoard guide](https://www.tensorflow.org/tensorboard/get_started). + +### How can I monitor training metrics using TensorBoard when training a YOLOv8 model? + +To monitor training metrics while training a YOLOv8 model with TensorBoard, follow these steps: + +1. **Install TensorBoard and YOLOv8:** Run `pip install ultralytics` which includes TensorBoard. +2. **Configure TensorBoard Logging:** During the training process, YOLOv8 logs metrics to a specified log directory. +3. **Start TensorBoard:** Launch TensorBoard using the command `tensorboard --logdir path/to/your/tensorboard/logs`. + +The TensorBoard dashboard, accessible via [http://localhost:6006/](http://localhost:6006/), provides real-time insights into various training metrics. For a deeper dive into training configurations, visit our [YOLOv8 Configuration guide](../usage/cfg.md). + +### What kind of metrics can I visualize with TensorBoard when training YOLOv8 models? + +When training YOLOv8 models, TensorBoard allows you to visualize an array of important metrics including: + +- **Loss (Training and Validation):** Indicates how well the model is performing during training and validation. +- **Accuracy/Precision/Recall:** Key performance metrics to evaluate detection accuracy. +- **Learning Rate:** Track learning rate changes to understand its impact on training dynamics. +- **mAP (mean Average Precision):** For a comprehensive evaluation of object detection accuracy at various IoU thresholds. + +These visualizations are essential for tracking model performance and making necessary optimizations. For more information on these metrics, refer to our [Performance Metrics guide](../guides/yolo-performance-metrics.md). + +### Can I use TensorBoard in a Google Colab environment for training YOLOv8? + +Yes, you can use TensorBoard in a Google Colab environment to train YOLOv8 models. Here's a quick setup: + +!!! Example "Configure TensorBoard for Google Colab" + + === "Python" + + ```ipython + %load_ext tensorboard + %tensorboard --logdir path/to/runs + ``` + + Then, run the YOLOv8 training script: + + ```python + from ultralytics import YOLO + + # Load a pre-trained model + model = YOLO("yolov8n.pt") + + # Train the model + results = model.train(data="coco8.yaml", epochs=100, imgsz=640) + ``` + +TensorBoard will visualize the training progress within Colab, providing real-time insights into metrics like loss and accuracy. For additional details on configuring YOLOv8 training, see our detailed [YOLOv8 Installation guide](../quickstart.md). diff --git a/ultralytics/docs/en/integrations/tensorrt.md b/ultralytics/docs/en/integrations/tensorrt.md new file mode 100644 index 0000000000000000000000000000000000000000..d10ff69682e256842945854f5287612f3af601fa --- /dev/null +++ b/ultralytics/docs/en/integrations/tensorrt.md @@ -0,0 +1,546 @@ +--- +comments: true +description: Learn to convert YOLOv8 models to TensorRT for high-speed NVIDIA GPU inference. Boost efficiency and deploy optimized models with our step-by-step guide. +keywords: YOLOv8, TensorRT, NVIDIA, GPU, deep learning, model optimization, high-speed inference, model export +--- + +# TensorRT Export for YOLOv8 Models + +Deploying computer vision models in high-performance environments can require a format that maximizes speed and efficiency. This is especially true when you are deploying your model on NVIDIA GPUs. + +By using the TensorRT export format, you can enhance your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for swift and efficient inference on NVIDIA hardware. This guide will give you easy-to-follow steps for the conversion process and help you make the most of NVIDIA's advanced technology in your deep learning projects. + +## TensorRT + ++ +
+ +[TensorRT](https://developer.nvidia.com/tensorrt), developed by NVIDIA, is an advanced software development kit (SDK) designed for high-speed deep learning inference. It's well-suited for real-time applications like object detection. + +This toolkit optimizes deep learning models for NVIDIA GPUs and results in faster and more efficient operations. TensorRT models undergo TensorRT optimization, which includes techniques like layer fusion, precision calibration (INT8 and FP16), dynamic tensor memory management, and kernel auto-tuning. Converting deep learning models into the TensorRT format allows developers to realize the potential of NVIDIA GPUs fully. + +TensorRT is known for its compatibility with various model formats, including TensorFlow, PyTorch, and ONNX, providing developers with a flexible solution for integrating and optimizing models from different frameworks. This versatility enables efficient model deployment across diverse hardware and software environments. + +## Key Features of TensorRT Models + +TensorRT models offer a range of key features that contribute to their efficiency and effectiveness in high-speed deep learning inference: + +- **Precision Calibration**: TensorRT supports precision calibration, allowing models to be fine-tuned for specific accuracy requirements. This includes support for reduced precision formats like INT8 and FP16, which can further boost inference speed while maintaining acceptable accuracy levels. + +- **Layer Fusion**: The TensorRT optimization process includes layer fusion, where multiple layers of a neural network are combined into a single operation. This reduces computational overhead and improves inference speed by minimizing memory access and computation. + ++ +
+ +- **Dynamic Tensor Memory Management**: TensorRT efficiently manages tensor memory usage during inference, reducing memory overhead and optimizing memory allocation. This results in more efficient GPU memory utilization. + +- **Automatic Kernel Tuning**: TensorRT applies automatic kernel tuning to select the most optimized GPU kernel for each layer of the model. This adaptive approach ensures that the model takes full advantage of the GPU's computational power. + +## Deployment Options in TensorRT + +Before we look at the code for exporting YOLOv8 models to the TensorRT format, let's understand where TensorRT models are normally used. + +TensorRT offers several deployment options, and each option balances ease of integration, performance optimization, and flexibility differently: + +- **Deploying within TensorFlow**: This method integrates TensorRT into TensorFlow, allowing optimized models to run in a familiar TensorFlow environment. It's useful for models with a mix of supported and unsupported layers, as TF-TRT can handle these efficiently. + ++ +
+ +- **Standalone TensorRT Runtime API**: Offers granular control, ideal for performance-critical applications. It's more complex but allows for custom implementation of unsupported operators. + +- **NVIDIA Triton Inference Server**: An option that supports models from various frameworks. Particularly suited for cloud or edge inference, it provides features like concurrent model execution and model analysis. + +## Exporting YOLOv8 Models to TensorRT + +You can improve execution efficiency and optimize performance by converting YOLOv8 models to TensorRT format. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TensorRT format + model.export(format="engine") # creates 'yolov8n.engine' + + # Load the exported TensorRT model + tensorrt_model = YOLO("yolov8n.engine") + + # Run inference + results = tensorrt_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TensorRT format + yolo export model=yolov8n.pt format=engine # creates 'yolov8n.engine'' + + # Run inference with the exported model + yolo predict model=yolov8n.engine source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +### Exporting TensorRT with INT8 Quantization + +Exporting Ultralytics YOLO models using TensorRT with INT8 precision executes post-training quantization (PTQ). TensorRT uses calibration for PTQ, which measures the distribution of activations within each activation tensor as the YOLO model processes inference on representative input data, and then uses that distribution to estimate scale values for each tensor. Each activation tensor that is a candidate for quantization has an associated scale that is deduced by a calibration process. + +When processing implicitly quantized networks TensorRT uses INT8 opportunistically to optimize layer execution time. If a layer runs faster in INT8 and has assigned quantization scales on its data inputs and outputs, then a kernel with INT8 precision is assigned to that layer, otherwise TensorRT selects a precision of either FP32 or FP16 for the kernel based on whichever results in faster execution time for that layer. + +!!! tip + + It is **critical** to ensure that the same device that will use the TensorRT model weights for deployment is used for exporting with INT8 precision, as the calibration results can vary across devices. + +#### Configuring INT8 Export + +The arguments provided when using [export](../modes/export.md) for an Ultralytics YOLO model will **greatly** influence the performance of the exported model. They will also need to be selected based on the device resources available, however the default arguments _should_ work for most [Ampere (or newer) NVIDIA discrete GPUs](https://developer.nvidia.com/blog/nvidia-ampere-architecture-in-depth/). The calibration algorithm used is `"ENTROPY_CALIBRATION_2"` and you can read more details about the options available [in the TensorRT Developer Guide](https://docs.nvidia.com/deeplearning/tensorrt/developer-guide/index.html#enable_int8_c). Ultralytics tests found that `"ENTROPY_CALIBRATION_2"` was the best choice and exports are fixed to using this algorithm. + +- `workspace` : Controls the size (in GiB) of the device memory allocation while converting the model weights. + + - Adjust the `workspace` value according to your calibration needs and resource availability. While a larger `workspace` may increase calibration time, it allows TensorRT to explore a wider range of optimization tactics, potentially enhancing model performance and accuracy. Conversely, a smaller `workspace` can reduce calibration time but may limit the optimization strategies, affecting the quality of the quantized model. + + - Default is `workspace=4` (GiB), this value may need to be increased if calibration crashes (exits without warning). + + - TensorRT will report `UNSUPPORTED_STATE` during export if the value for `workspace` is larger than the memory available to the device, which means the value for `workspace` should be lowered. + + - If `workspace` is set to max value and calibration fails/crashes, consider reducing the values for `imgsz` and `batch` to reduce memory requirements. + + - Remember calibration for INT8 is specific to each device, borrowing a "high-end" GPU for calibration, might result in poor performance when inference is run on another device. + +- `batch` : The maximum batch-size that will be used for inference. During inference smaller batches can be used, but inference will not accept batches any larger than what is specified. + +!!! note + + During calibration, twice the `batch` size provided will be used. Using small batches can lead to inaccurate scaling during calibration. This is because the process adjusts based on the data it sees. Small batches might not capture the full range of values, leading to issues with the final calibration, so the `batch` size is doubled automatically. If no batch size is specified `batch=1`, calibration will be run at `batch=1 * 2` to reduce calibration scaling errors. + +Experimentation by NVIDIA led them to recommend using at least 500 calibration images that are representative of the data for your model, with INT8 quantization calibration. This is a guideline and not a _hard_ requirement, and **you will need to experiment with what is required to perform well for your dataset**. Since the calibration data is required for INT8 calibration with TensorRT, make certain to use the `data` argument when `int8=True` for TensorRT and use `data="my_dataset.yaml"`, which will use the images from [validation](../modes/val.md) to calibrate with. When no value is passed for `data` with export to TensorRT with INT8 quantization, the default will be to use one of the ["small" example datasets based on the model task](../datasets/index.md) instead of throwing an error. + +!!! example + + === "Python" + + ```{ .py .annotate } + from ultralytics import YOLO + + model = YOLO("yolov8n.pt") + model.export( + format="engine", + dynamic=True, # (1)! + batch=8, # (2)! + workspace=4, # (3)! + int8=True, + data="coco.yaml", # (4)! + ) + + # Load the exported TensorRT INT8 model + model = YOLO("yolov8n.engine", task="detect") + + # Run inference + result = model.predict("https://ultralytics.com/images/bus.jpg") + ``` + + 1. Exports with dynamic axes, this will be enabled by default when exporting with `int8=True` even when not explicitly set. See [export arguments](../modes/export.md#arguments) for additional information. + 2. Sets max batch size of 8 for exported model, which calibrates with `batch = 2 * 8` to avoid scaling errors during calibration. + 3. Allocates 4 GiB of memory instead of allocating the entire device for conversion process. + 4. Uses [COCO dataset](../datasets/detect/coco.md) for calibration, specifically the images used for [validation](../modes/val.md) (5,000 total). + + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TensorRT format with INT8 quantization + yolo export model=yolov8n.pt format=engine batch=8 workspace=4 int8=True data=coco.yaml # creates 'yolov8n.engine'' + + # Run inference with the exported TensorRT quantized model + yolo predict model=yolov8n.engine source='https://ultralytics.com/images/bus.jpg' + ``` + +???+ warning "Calibration Cache" + + TensorRT will generate a calibration `.cache` which can be re-used to speed up export of future model weights using the same data, but this may result in poor calibration when the data is vastly different or if the `batch` value is changed drastically. In these circumstances, the existing `.cache` should be renamed and moved to a different directory or deleted entirely. + +#### Advantages of using YOLO with TensorRT INT8 + +- **Reduced model size:** Quantization from FP32 to INT8 can reduce the model size by 4x (on disk or in memory), leading to faster download times. lower storage requirements, and reduced memory footprint when deploying a model. + +- **Lower power consumption:** Reduced precision operations for INT8 exported YOLO models can consume less power compared to FP32 models, especially for battery-powered devices. + +- **Improved inference speeds:** TensorRT optimizes the model for the target hardware, potentially leading to faster inference speeds on GPUs, embedded devices, and accelerators. + +??? note "Note on Inference Speeds" + + The first few inference calls with a model exported to TensorRT INT8 can be expected to have longer than usual preprocessing, inference, and/or postprocessing times. This may also occur when changing `imgsz` during inference, especially when `imgsz` is not the same as what was specified during export (export `imgsz` is set as TensorRT "optimal" profile). + +#### Drawbacks of using YOLO with TensorRT INT8 + +- **Decreases in evaluation metrics:** Using a lower precision will mean that `mAP`, `Precision`, `Recall` or any [other metric used to evaluate model performance](../guides/yolo-performance-metrics.md) is likely to be somewhat worse. See the [Performance results section](#ultralytics-yolo-tensorrt-export-performance) to compare the differences in `mAP50` and `mAP50-95` when exporting with INT8 on small sample of various devices. + +- **Increased development times:** Finding the "optimal" settings for INT8 calibration for dataset and device can take a significant amount of testing. + +- **Hardware dependency:** Calibration and performance gains could be highly hardware dependent and model weights are less transferable. + +## Ultralytics YOLO TensorRT Export Performance + +### NVIDIA A100 + +!!! tip "Performance" + + Tested with Ubuntu 22.04.3 LTS, `python 3.10.12`, `ultralytics==8.2.4`, `tensorrt==8.6.1.post1` + + === "Detection (COCO)" + + See [Detection Docs](../tasks/detect.md) for usage examples with these models trained on [COCO](../datasets/detect/coco.md), which include 80 pre-trained classes. + + !!! note + Inference times shown for `mean`, `min` (fastest), and `max` (slowest) for each test using pre-trained weights `yolov8n.engine` + + | Precision | Eval test | mean+ +
+ +## Why Should You Export to TF GraphDef? + +TF GraphDef is a powerful component of the TensorFlow ecosystem that was developed by Google. It can be used to optimize and deploy models like YOLOv8. Exporting to TF GraphDef lets us move models from research to real-world applications. It allows models to run in environments without the full TensorFlow framework. + +The GraphDef format represents the model as a serialized computation graph. This enables various optimization techniques like constant folding, quantization, and graph transformations. These optimizations ensure efficient execution, reduced memory usage, and faster inference speeds. + +GraphDef models can use hardware accelerators such as GPUs, TPUs, and AI chips, unlocking significant performance gains for the YOLOv8 inference pipeline. The TF GraphDef format creates a self-contained package with the model and its dependencies, simplifying deployment and integration into diverse systems. + +## Key Features of TF GraphDef Models + +TF GraphDef offers distinct features for streamlining model deployment and optimization. + +Here's a look at its key characteristics: + +- **Model Serialization**: TF GraphDef provides a way to serialize and store TensorFlow models in a platform-independent format. This serialized representation allows you to load and execute your models without the original Python codebase, making deployment easier. + +- **Graph Optimization**: TF GraphDef enables the optimization of computational graphs. These optimizations can boost performance by streamlining execution flow, reducing redundancies, and tailoring operations to suit specific hardware. + +- **Deployment Flexibility**: Models exported to the GraphDef format can be used in various environments, including resource-constrained devices, web browsers, and systems with specialized hardware. This opens up possibilities for wider deployment of your TensorFlow models. + +- **Production Focus**: GraphDef is designed for production deployment. It supports efficient execution, serialization features, and optimizations that align with real-world use cases. + +## Deployment Options with TF GraphDef + +Before we dive into the process of exporting YOLOv8 models to TF GraphDef, let's take a look at some typical deployment situations where this format is used. + +Here's how you can deploy with TF GraphDef efficiently across various platforms. + +- **TensorFlow Serving:** This framework is designed to deploy TensorFlow models in production environments. TensorFlow Serving offers model management, versioning, and the infrastructure for efficient model serving at scale. It's a seamless way to integrate your GraphDef-based models into production web services or APIs. + +- **Mobile and Embedded Devices:** With tools like TensorFlow Lite, you can convert TF GraphDef models into formats optimized for smartphones, tablets, and various embedded devices. Your models can then be used for on-device inference, where execution is done locally, often providing performance gains and offline capabilities. + +- **Web Browsers:** TensorFlow.js enables the deployment of TF GraphDef models directly within web browsers. It paves the way for real-time object detection applications running on the client side, using the capabilities of YOLOv8 through JavaScript. + +- **Specialized Hardware:** TF GraphDef's platform-agnostic nature allows it to target custom hardware, such as accelerators and TPUs (Tensor Processing Units). These devices can provide performance advantages for computationally intensive models. + +## Exporting YOLOv8 Models to TF GraphDef + +You can convert your YOLOv8 object detection model to the TF GraphDef format, which is compatible with various systems, to improve its performance across platforms. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF GraphDef format + model.export(format="pb") # creates 'yolov8n.pb' + + # Load the exported TF GraphDef model + tf_graphdef_model = YOLO("yolov8n.pb") + + # Run inference + results = tf_graphdef_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TF GraphDef format + yolo export model=yolov8n.pt format=pb # creates 'yolov8n.pb' + + # Run inference with the exported model + yolo predict model='yolov8n.pb' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 TF GraphDef Models + +Once you've exported your YOLOv8 model to the TF GraphDef format, the next step is deployment. The primary and recommended first step for running a TF GraphDef model is to use the YOLO("model.pb") method, as previously shown in the usage code snippet. + +However, for more information on deploying your TF GraphDef models, take a look at the following resources: + +- **[TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)**: A guide on TensorFlow Serving that teaches how to deploy and serve machine learning models efficiently in production environments. + +- **[TensorFlow Lite](https://www.tensorflow.org/api_docs/python/tf/lite/TFLiteConverter)**: This page describes how to convert machine learning models into a format optimized for on-device inference with TensorFlow Lite. + +- **[TensorFlow.js](https://www.tensorflow.org/js/guide/conversion)**: A guide on model conversion that teaches how to convert TensorFlow or Keras models into TensorFlow.js format for use in web applications. + +## Summary + +In this guide, we explored how to export Ultralytics YOLOv8 models to the TF GraphDef format. By doing this, you can flexibly deploy your optimized YOLOv8 models in different environments. + +For further details on usage, visit the [TF GraphDef official documentation](https://www.tensorflow.org/api_docs/python/tf/Graph). + +For more information on integrating Ultralytics YOLOv8 with other platforms and frameworks, don't forget to check out our [integration guide page](index.md). It has great resources and insights to help you make the most of YOLOv8 in your projects. + +## FAQ + +### How do I export a YOLOv8 model to TF GraphDef format? + +Ultralytics YOLOv8 models can be exported to TensorFlow GraphDef (TF GraphDef) format seamlessly. This format provides a serialized, platform-independent representation of the model, ideal for deploying in varied environments like mobile and web. To export a YOLOv8 model to TF GraphDef, follow these steps: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF GraphDef format + model.export(format="pb") # creates 'yolov8n.pb' + + # Load the exported TF GraphDef model + tf_graphdef_model = YOLO("yolov8n.pb") + + # Run inference + results = tf_graphdef_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TF GraphDef format + yolo export model="yolov8n.pt" format="pb" # creates 'yolov8n.pb' + + # Run inference with the exported model + yolo predict model="yolov8n.pb" source="https://ultralytics.com/images/bus.jpg" + ``` + +For more information on different export options, visit the [Ultralytics documentation on model export](../modes/export.md). + +### What are the benefits of using TF GraphDef for YOLOv8 model deployment? + +Exporting YOLOv8 models to the TF GraphDef format offers multiple advantages, including: + +1. **Platform Independence**: TF GraphDef provides a platform-independent format, allowing models to be deployed across various environments including mobile and web browsers. +2. **Optimizations**: The format enables several optimizations, such as constant folding, quantization, and graph transformations, which enhance execution efficiency and reduce memory usage. +3. **Hardware Acceleration**: Models in TF GraphDef format can leverage hardware accelerators like GPUs, TPUs, and AI chips for performance gains. + +Read more about the benefits in the [TF GraphDef section](#why-should-you-export-to-tf-graphdef) of our documentation. + +### Why should I use Ultralytics YOLOv8 over other object detection models? + +Ultralytics YOLOv8 offers numerous advantages compared to other models like YOLOv5 and YOLOv7. Some key benefits include: + +1. **State-of-the-Art Performance**: YOLOv8 provides exceptional speed and accuracy for real-time object detection, segmentation, and classification. +2. **Ease of Use**: Features a user-friendly API for model training, validation, prediction, and export, making it accessible for both beginners and experts. +3. **Broad Compatibility**: Supports multiple export formats including ONNX, TensorRT, CoreML, and TensorFlow, for versatile deployment options. + +Explore further details in our [introduction to YOLOv8](https://docs.ultralytics.com/models/yolov8/). + +### How can I deploy a YOLOv8 model on specialized hardware using TF GraphDef? + +Once a YOLOv8 model is exported to TF GraphDef format, you can deploy it across various specialized hardware platforms. Typical deployment scenarios include: + +- **TensorFlow Serving**: Use TensorFlow Serving for scalable model deployment in production environments. It supports model management and efficient serving. +- **Mobile Devices**: Convert TF GraphDef models to TensorFlow Lite, optimized for mobile and embedded devices, enabling on-device inference. +- **Web Browsers**: Deploy models using TensorFlow.js for client-side inference in web applications. +- **AI Accelerators**: Leverage TPUs and custom AI chips for accelerated inference. + +Check the [deployment options](#deployment-options-with-tf-graphdef) section for detailed information. + +### Where can I find solutions for common issues while exporting YOLOv8 models? + +For troubleshooting common issues with exporting YOLOv8 models, Ultralytics provides comprehensive guides and resources. If you encounter problems during installation or model export, refer to: + +- **[Common Issues Guide](../guides/yolo-common-issues.md)**: Offers solutions to frequently faced problems. +- **[Installation Guide](../quickstart.md)**: Step-by-step instructions for setting up the required packages. + +These resources should help you resolve most issues related to YOLOv8 model export and deployment. diff --git a/ultralytics/docs/en/integrations/tf-savedmodel.md b/ultralytics/docs/en/integrations/tf-savedmodel.md new file mode 100644 index 0000000000000000000000000000000000000000..223e713be0042971db2aa68ad8ecbdf09be28c23 --- /dev/null +++ b/ultralytics/docs/en/integrations/tf-savedmodel.md @@ -0,0 +1,197 @@ +--- +comments: true +description: Learn how to export Ultralytics YOLOv8 models to TensorFlow SavedModel format for easy deployment across various platforms and environments. +keywords: YOLOv8, TF SavedModel, Ultralytics, TensorFlow, model export, model deployment, machine learning, AI +--- + +# Understand How to Export to TF SavedModel Format From YOLOv8 + +Deploying machine learning models can be challenging. However, using an efficient and flexible model format can make your job easier. TF SavedModel is an open-source machine-learning framework used by TensorFlow to load machine-learning models in a consistent way. It is like a suitcase for TensorFlow models, making them easy to carry and use on different devices and systems. + +Learning how to export to TF SavedModel from [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models can help you deploy models easily across different platforms and environments. In this guide, we'll walk through how to convert your models to the TF SavedModel format, simplifying the process of running inferences with your models on different devices. + +## Why Should You Export to TF SavedModel? + +The TensorFlow SavedModel format is a part of the TensorFlow ecosystem developed by Google as shown below. It is designed to save and serialize TensorFlow models seamlessly. It encapsulates the complete details of models like the architecture, weights, and even compilation information. This makes it straightforward to share, deploy, and continue training across different environments. + ++ +
+ +The TF SavedModel has a key advantage: its compatibility. It works well with TensorFlow Serving, TensorFlow Lite, and TensorFlow.js. This compatibility makes it easier to share and deploy models across various platforms, including web and mobile applications. The TF SavedModel format is useful both for research and production. It provides a unified way to manage your models, ensuring they are ready for any application. + +## Key Features of TF SavedModels + +Here are the key features that make TF SavedModel a great option for AI developers: + +- **Portability**: TF SavedModel provides a language-neutral, recoverable, hermetic serialization format. They enable higher-level systems and tools to produce, consume, and transform TensorFlow models. SavedModels can be easily shared and deployed across different platforms and environments. + +- **Ease of Deployment**: TF SavedModel bundles the computational graph, trained parameters, and necessary metadata into a single package. They can be easily loaded and used for inference without requiring the original code that built the model. This makes the deployment of TensorFlow models straightforward and efficient in various production environments. + +- **Asset Management**: TF SavedModel supports the inclusion of external assets such as vocabularies, embeddings, or lookup tables. These assets are stored alongside the graph definition and variables, ensuring they are available when the model is loaded. This feature simplifies the management and distribution of models that rely on external resources. + +## Deployment Options with TF SavedModel + +Before we dive into the process of exporting YOLOv8 models to the TF SavedModel format, let's explore some typical deployment scenarios where this format is used. + +TF SavedModel provides a range of options to deploy your machine learning models: + +- **TensorFlow Serving:** TensorFlow Serving is a flexible, high-performance serving system designed for production environments. It natively supports TF SavedModels, making it easy to deploy and serve your models on cloud platforms, on-premises servers, or edge devices. + +- **Cloud Platforms:** Major cloud providers like Google Cloud Platform (GCP), Amazon Web Services (AWS), and Microsoft Azure offer services for deploying and running TensorFlow models, including TF SavedModels. These services provide scalable and managed infrastructure, allowing you to deploy and scale your models easily. + +- **Mobile and Embedded Devices:** TensorFlow Lite, a lightweight solution for running machine learning models on mobile, embedded, and IoT devices, supports converting TF SavedModels to the TensorFlow Lite format. This allows you to deploy your models on a wide range of devices, from smartphones and tablets to microcontrollers and edge devices. + +- **TensorFlow Runtime:** TensorFlow Runtime (`tfrt`) is a high-performance runtime for executing TensorFlow graphs. It provides lower-level APIs for loading and running TF SavedModels in C++ environments. TensorFlow Runtime offers better performance compared to the standard TensorFlow runtime. It is suitable for deployment scenarios that require low-latency inference and tight integration with existing C++ codebases. + +## Exporting YOLOv8 Models to TF SavedModel + +By exporting YOLOv8 models to the TF SavedModel format, you enhance their adaptability and ease of deployment across various platforms. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF SavedModel format + model.export(format="saved_model") # creates '/yolov8n_saved_model' + + # Load the exported TF SavedModel model + tf_savedmodel_model = YOLO("./yolov8n_saved_model") + + # Run inference + results = tf_savedmodel_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TF SavedModel format + yolo export model=yolov8n.pt format=saved_model # creates '/yolov8n_saved_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_saved_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 TF SavedModel Models + +Now that you have exported your YOLOv8 model to the TF SavedModel format, the next step is to deploy it. The primary and recommended first step for running a TF GraphDef model is to use the YOLO("./yolov8n_saved_model") method, as previously shown in the usage code snippet. + +However, for in-depth instructions on deploying your TF SavedModel models, take a look at the following resources: + +- **[TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)**: Here's the developer documentation for how to deploy your TF SavedModel models using TensorFlow Serving. + +- **[Run a TensorFlow SavedModel in Node.js](https://blog.tensorflow.org/2020/01/run-tensorflow-savedmodel-in-nodejs-directly-without-conversion.html)**: A TensorFlow blog post on running a TensorFlow SavedModel in Node.js directly without conversion. + +- **[Deploying on Cloud](https://blog.tensorflow.org/2020/04/how-to-deploy-tensorflow-2-models-on-cloud-ai-platform.html)**: A TensorFlow blog post on deploying a TensorFlow SavedModel model on the Cloud AI Platform. + +## Summary + +In this guide, we explored how to export Ultralytics YOLOv8 models to the TF SavedModel format. By exporting to TF SavedModel, you gain the flexibility to optimize, deploy, and scale your YOLOv8 models on a wide range of platforms. + +For further details on usage, visit the [TF SavedModel official documentation](https://www.tensorflow.org/guide/saved_model). + +For more information on integrating Ultralytics YOLOv8 with other platforms and frameworks, don't forget to check out our [integration guide page](index.md). It's packed with great resources to help you make the most of YOLOv8 in your projects. + +## FAQ + +### How do I export an Ultralytics YOLO model to TensorFlow SavedModel format? + +Exporting an Ultralytics YOLO model to the TensorFlow SavedModel format is straightforward. You can use either Python or CLI to achieve this: + +!!! Example "Exporting YOLOv8 to TF SavedModel" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF SavedModel format + model.export(format="saved_model") # creates '/yolov8n_saved_model' + + # Load the exported TF SavedModel for inference + tf_savedmodel_model = YOLO("./yolov8n_saved_model") + results = tf_savedmodel_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export the YOLOv8 model to TF SavedModel format + yolo export model=yolov8n.pt format=saved_model # creates '/yolov8n_saved_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_saved_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +Refer to the [Ultralytics Export documentation](../modes/export.md) for more details. + +### Why should I use the TensorFlow SavedModel format? + +The TensorFlow SavedModel format offers several advantages for model deployment: + +- **Portability:** It provides a language-neutral format, making it easy to share and deploy models across different environments. +- **Compatibility:** Integrates seamlessly with tools like TensorFlow Serving, TensorFlow Lite, and TensorFlow.js, which are essential for deploying models on various platforms, including web and mobile applications. +- **Complete encapsulation:** Encodes the model architecture, weights, and compilation information, allowing for straightforward sharing and training continuation. + +For more benefits and deployment options, check out the [Ultralytics YOLO model deployment options](../guides/model-deployment-options.md). + +### What are the typical deployment scenarios for TF SavedModel? + +TF SavedModel can be deployed in various environments, including: + +- **TensorFlow Serving:** Ideal for production environments requiring scalable and high-performance model serving. +- **Cloud Platforms:** Supports major cloud services like Google Cloud Platform (GCP), Amazon Web Services (AWS), and Microsoft Azure for scalable model deployment. +- **Mobile and Embedded Devices:** Using TensorFlow Lite to convert TF SavedModels allows for deployment on mobile devices, IoT devices, and microcontrollers. +- **TensorFlow Runtime:** For C++ environments needing low-latency inference with better performance. + +For detailed deployment options, visit the official guides on [deploying TensorFlow models](https://www.tensorflow.org/tfx/guide/serving). + +### How can I install the necessary packages to export YOLOv8 models? + +To export YOLOv8 models, you need to install the `ultralytics` package. Run the following command in your terminal: + +```bash +pip install ultralytics +``` + +For more detailed installation instructions and best practices, refer to our [Ultralytics Installation guide](../quickstart.md). If you encounter any issues, consult our [Common Issues guide](../guides/yolo-common-issues.md). + +### What are the key features of the TensorFlow SavedModel format? + +TF SavedModel format is beneficial for AI developers due to the following features: + +- **Portability:** Allows sharing and deployment across various environments effortlessly. +- **Ease of Deployment:** Encapsulates the computational graph, trained parameters, and metadata into a single package, which simplifies loading and inference. +- **Asset Management:** Supports external assets like vocabularies, ensuring they are available when the model loads. + +For further details, explore the [official TensorFlow documentation](https://www.tensorflow.org/guide/saved_model). diff --git a/ultralytics/docs/en/integrations/tfjs.md b/ultralytics/docs/en/integrations/tfjs.md new file mode 100644 index 0000000000000000000000000000000000000000..47504e4f3e7060a10cc335f3fee4d8bff712d423 --- /dev/null +++ b/ultralytics/docs/en/integrations/tfjs.md @@ -0,0 +1,194 @@ +--- +comments: true +description: Convert your Ultralytics YOLOv8 models to TensorFlow.js for high-speed, local object detection. Learn how to optimize ML models for browser and Node.js apps. +keywords: YOLOv8, TensorFlow.js, TF.js, model export, machine learning, object detection, browser ML, Node.js, Ultralytics, YOLO, export models +--- + +# Export to TF.js Model Format From a YOLOv8 Model Format + +Deploying machine learning models directly in the browser or on Node.js can be tricky. You'll need to make sure your model format is optimized for faster performance so that the model can be used to run interactive applications locally on the user's device. The TensorFlow.js, or TF.js, model format is designed to use minimal power while delivering fast performance. + +The 'export to TF.js model format' feature allows you to optimize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for high-speed and locally-run object detection inference. In this guide, we'll walk you through converting your models to the TF.js format, making it easier for your models to perform well on various local browsers and Node.js applications. + +## Why Should You Export to TF.js? + +Exporting your machine learning models to TensorFlow.js, developed by the TensorFlow team as part of the broader TensorFlow ecosystem, offers numerous advantages for deploying machine learning applications. It helps enhance user privacy and security by keeping sensitive data on the device. The image below shows the TensorFlow.js architecture, and how machine learning models are converted and deployed on both web browsers and Node.js. + ++ +
+ +Running models locally also reduces latency and provides a more responsive user experience. TensorFlow.js also comes with offline capabilities, allowing users to use your application even without an internet connection. TF.js is designed for efficient execution of complex models on devices with limited resources as it is engineered for scalability, with GPU acceleration support. + +## Key Features of TF.js + +Here are the key features that make TF.js a powerful tool for developers: + +- **Cross-Platform Support:** TensorFlow.js can be used in both browser and Node.js environments, providing flexibility in deployment across different platforms. It lets developers build and deploy applications more easily. + +- **Support for Multiple Backends:** TensorFlow.js supports various backends for computation including CPU, WebGL for GPU acceleration, WebAssembly (WASM) for near-native execution speed, and WebGPU for advanced browser-based machine learning capabilities. + +- **Offline Capabilities:** With TensorFlow.js, models can run in the browser without the need for an internet connection, making it possible to develop applications that are functional offline. + +## Deployment Options with TensorFlow.js + +Before we dive into the process of exporting YOLOv8 models to the TF.js format, let's explore some typical deployment scenarios where this format is used. + +TF.js provides a range of options to deploy your machine learning models: + +- **In-Browser ML Applications:** You can build web applications that run machine learning models directly in the browser. The need for server-side computation is eliminated and the server load is reduced. + +- **Node.js Applications::** TensorFlow.js also supports deployment in Node.js environments, enabling the development of server-side machine learning applications. It is particularly useful for applications that require the processing power of a server or access to server-side data. + +- **Chrome Extensions:** An interesting deployment scenario is the creation of Chrome extensions with TensorFlow.js. For instance, you can develop an extension that allows users to right-click on an image within any webpage to classify it using a pre-trained ML model. TensorFlow.js can be integrated into everyday web browsing experiences to provide immediate insights or augmentations based on machine learning. + +## Exporting YOLOv8 Models to TensorFlow.js + +You can expand model compatibility and deployment flexibility by converting YOLOv8 models to TF.js. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF.js format + model.export(format="tfjs") # creates '/yolov8n_web_model' + + # Load the exported TF.js model + tfjs_model = YOLO("./yolov8n_web_model") + + # Run inference + results = tfjs_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TF.js format + yolo export model=yolov8n.pt format=tfjs # creates '/yolov8n_web_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_web_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +## Deploying Exported YOLOv8 TensorFlow.js Models + +Now that you have exported your YOLOv8 model to the TF.js format, the next step is to deploy it. The primary and recommended first step for running a TF.js is to use the YOLO("./yolov8n_web_model") method, as previously shown in the usage code snippet. + +However, for in-depth instructions on deploying your TF.js models, take a look at the following resources: + +- **[Chrome Extension](https://www.tensorflow.org/js/tutorials/deployment/web_ml_in_chrome)**: Here's the developer documentation for how to deploy your TF.js models to a Chrome extension. + +- **[Run TensorFlow.js in Node.js](https://www.tensorflow.org/js/guide/nodejs)**: A TensorFlow blog post on running TensorFlow.js in Node.js directly. + +- **[Deploying TensorFlow.js - Node Project on Cloud Platform](https://www.tensorflow.org/js/guide/node_in_cloud)**: A TensorFlow blog post on deploying a TensorFlow.js model on a Cloud Platform. + +## Summary + +In this guide, we learned how to export Ultralytics YOLOv8 models to the TensorFlow.js format. By exporting to TF.js, you gain the flexibility to optimize, deploy, and scale your YOLOv8 models on a wide range of platforms. + +For further details on usage, visit the [TensorFlow.js official documentation](https://www.tensorflow.org/js/guide). + +For more information on integrating Ultralytics YOLOv8 with other platforms and frameworks, don't forget to check out our [integration guide page](index.md). It's packed with great resources to help you make the most of YOLOv8 in your projects. + +## FAQ + +### How do I export Ultralytics YOLOv8 models to TensorFlow.js format? + +Exporting Ultralytics YOLOv8 models to TensorFlow.js (TF.js) format is straightforward. You can follow these steps: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TF.js format + model.export(format="tfjs") # creates '/yolov8n_web_model' + + # Load the exported TF.js model + tfjs_model = YOLO("./yolov8n_web_model") + + # Run inference + results = tfjs_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TF.js format + yolo export model=yolov8n.pt format=tfjs # creates '/yolov8n_web_model' + + # Run inference with the exported model + yolo predict model='./yolov8n_web_model' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about supported export options, visit the [Ultralytics documentation page on deployment options](../guides/model-deployment-options.md). + +### Why should I export my YOLOv8 models to TensorFlow.js? + +Exporting YOLOv8 models to TensorFlow.js offers several advantages, including: + +1. **Local Execution:** Models can run directly in the browser or Node.js, reducing latency and enhancing user experience. +2. **Cross-Platform Support:** TF.js supports multiple environments, allowing flexibility in deployment. +3. **Offline Capabilities:** Enables applications to function without an internet connection, ensuring reliability and privacy. +4. **GPU Acceleration:** Leverages WebGL for GPU acceleration, optimizing performance on devices with limited resources. + +For a comprehensive overview, see our [Integrations with TensorFlow.js](../integrations/tf-graphdef.md). + +### How does TensorFlow.js benefit browser-based machine learning applications? + +TensorFlow.js is specifically designed for efficient execution of ML models in browsers and Node.js environments. Here's how it benefits browser-based applications: + +- **Reduces Latency:** Runs machine learning models locally, providing immediate results without relying on server-side computations. +- **Improves Privacy:** Keeps sensitive data on the user's device, minimizing security risks. +- **Enables Offline Use:** Models can operate without an internet connection, ensuring consistent functionality. +- **Supports Multiple Backends:** Offers flexibility with backends like CPU, WebGL, WebAssembly (WASM), and WebGPU for varying computational needs. + +Interested in learning more about TF.js? Check out the [official TensorFlow.js guide](https://www.tensorflow.org/js/guide). + +### What are the key features of TensorFlow.js for deploying YOLOv8 models? + +Key features of TensorFlow.js include: + +- **Cross-Platform Support:** TF.js can be used in both web browsers and Node.js, providing extensive deployment flexibility. +- **Multiple Backends:** Supports CPU, WebGL for GPU acceleration, WebAssembly (WASM), and WebGPU for advanced operations. +- **Offline Capabilities:** Models can run directly in the browser without internet connectivity, making it ideal for developing responsive web applications. + +For deployment scenarios and more in-depth information, see our section on [Deployment Options with TensorFlow.js](#deploying-exported-yolov8-tensorflowjs-models). + +### Can I deploy a YOLOv8 model on server-side Node.js applications using TensorFlow.js? + +Yes, TensorFlow.js allows the deployment of YOLOv8 models on Node.js environments. This enables server-side machine learning applications that benefit from the processing power of a server and access to server-side data. Typical use cases include real-time data processing and machine learning pipelines on backend servers. + +To get started with Node.js deployment, refer to the [Run TensorFlow.js in Node.js](https://www.tensorflow.org/js/guide/nodejs) guide from TensorFlow. diff --git a/ultralytics/docs/en/integrations/tflite.md b/ultralytics/docs/en/integrations/tflite.md new file mode 100644 index 0000000000000000000000000000000000000000..332018ecc3fbf8e7b2b5c3cc2aa244e8eb06d3a1 --- /dev/null +++ b/ultralytics/docs/en/integrations/tflite.md @@ -0,0 +1,193 @@ +--- +comments: true +description: Learn how to convert YOLOv8 models to TFLite for edge device deployment. Optimize performance and ensure seamless execution on various platforms. +keywords: YOLOv8, TFLite, model export, TensorFlow Lite, edge devices, deployment, Ultralytics, machine learning, on-device inference, model optimization +--- + +# A Guide on YOLOv8 Model Export to TFLite for Deployment + ++ +
+ +Deploying computer vision models on edge devices or embedded devices requires a format that can ensure seamless performance. + +The TensorFlow Lite or TFLite export format allows you to optimize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for tasks like object detection and image classification in edge device-based applications. In this guide, we'll walk through the steps for converting your models to the TFLite format, making it easier for your models to perform well on various edge devices. + +## Why should you export to TFLite? + +Introduced by Google in May 2017 as part of their TensorFlow framework, [TensorFlow Lite](https://www.tensorflow.org/lite/guide), or TFLite for short, is an open-source deep learning framework designed for on-device inference, also known as edge computing. It gives developers the necessary tools to execute their trained models on mobile, embedded, and IoT devices, as well as traditional computers. + +TensorFlow Lite is compatible with a wide range of platforms, including embedded Linux, Android, iOS, and MCU. Exporting your model to TFLite makes your applications faster, more reliable, and capable of running offline. + +## Key Features of TFLite Models + +TFLite models offer a wide range of key features that enable on-device machine learning by helping developers run their models on mobile, embedded, and edge devices: + +- **On-device Optimization**: TFLite optimizes for on-device ML, reducing latency by processing data locally, enhancing privacy by not transmitting personal data, and minimizing model size to save space. + +- **Multiple Platform Support**: TFLite offers extensive platform compatibility, supporting Android, iOS, embedded Linux, and microcontrollers. + +- **Diverse Language Support**: TFLite is compatible with various programming languages, including Java, Swift, Objective-C, C++, and Python. + +- **High Performance**: Achieves superior performance through hardware acceleration and model optimization. + +## Deployment Options in TFLite + +Before we look at the code for exporting YOLOv8 models to the TFLite format, let's understand how TFLite models are normally used. + +TFLite offers various on-device deployment options for machine learning models, including: + +- **Deploying with Android and iOS**: Both Android and iOS applications with TFLite can analyze edge-based camera feeds and sensors to detect and identify objects. TFLite also offers native iOS libraries written in [Swift](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/swift) and [Objective-C](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/objc). The architecture diagram below shows the process of deploying a trained model onto Android and iOS platforms using TensorFlow Lite. + ++ +
+ +- **Implementing with Embedded Linux**: If running inferences on a [Raspberry Pi](https://www.raspberrypi.org/) using the [Ultralytics Guide](../guides/raspberry-pi.md) does not meet the speed requirements for your use case, you can use an exported TFLite model to accelerate inference times. Additionally, it's possible to further improve performance by utilizing a [Coral Edge TPU device](https://coral.withgoogle.com/). + +- **Deploying with Microcontrollers**: TFLite models can also be deployed on microcontrollers and other devices with only a few kilobytes of memory. The core runtime just fits in 16 KB on an Arm Cortex M3 and can run many basic models. It doesn't require operating system support, any standard C or C++ libraries, or dynamic memory allocation. + +## Export to TFLite: Converting Your YOLOv8 Model + +You can improve on-device model execution efficiency and optimize performance by converting them to TFLite format. + +### Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TFLite format + model.export(format="tflite") # creates 'yolov8n_float32.tflite' + + # Load the exported TFLite model + tflite_model = YOLO("yolov8n_float32.tflite") + + # Run inference + results = tflite_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TFLite format + yolo export model=yolov8n.pt format=tflite # creates 'yolov8n_float32.tflite' + + # Run inference with the exported model + yolo predict model='yolov8n_float32.tflite' source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +## Deploying Exported YOLOv8 TFLite Models + +After successfully exporting your Ultralytics YOLOv8 models to TFLite format, you can now deploy them. The primary and recommended first step for running a TFLite model is to utilize the YOLO("model.tflite") method, as outlined in the previous usage code snippet. However, for in-depth instructions on deploying your TFLite models in various other settings, take a look at the following resources: + +- **[Android](https://www.tensorflow.org/lite/android/quickstart)**: A quick start guide for integrating TensorFlow Lite into Android applications, providing easy-to-follow steps for setting up and running machine learning models. + +- **[iOS](https://www.tensorflow.org/lite/guide/ios)**: Check out this detailed guide for developers on integrating and deploying TensorFlow Lite models in iOS applications, offering step-by-step instructions and resources. + +- **[End-To-End Examples](https://www.tensorflow.org/lite/examples)**: This page provides an overview of various TensorFlow Lite examples, showcasing practical applications and tutorials designed to help developers implement TensorFlow Lite in their machine learning projects on mobile and edge devices. + +## Summary + +In this guide, we focused on how to export to TFLite format. By converting your Ultralytics YOLOv8 models to TFLite model format, you can improve the efficiency and speed of YOLOv8 models, making them more effective and suitable for edge computing environments. + +For further details on usage, visit the [TFLite official documentation](https://www.tensorflow.org/lite/guide). + +Also, if you're curious about other Ultralytics YOLOv8 integrations, make sure to check out our [integration guide page](../integrations/index.md). You'll find tons of helpful info and insights waiting for you there. + +## FAQ + +### How do I export a YOLOv8 model to TFLite format? + +To export a YOLOv8 model to TFLite format, you can use the Ultralytics library. First, install the required package using: + +```bash +pip install ultralytics +``` + +Then, use the following code snippet to export your model: + +```python +from ultralytics import YOLO + +# Load the YOLOv8 model +model = YOLO("yolov8n.pt") + +# Export the model to TFLite format +model.export(format="tflite") # creates 'yolov8n_float32.tflite' +``` + +For CLI users, you can achieve this with: + +```bash +yolo export model=yolov8n.pt format=tflite # creates 'yolov8n_float32.tflite' +``` + +For more details, visit the [Ultralytics export guide](../modes/export.md). + +### What are the benefits of using TensorFlow Lite for YOLOv8 model deployment? + +TensorFlow Lite (TFLite) is an open-source deep learning framework designed for on-device inference, making it ideal for deploying YOLOv8 models on mobile, embedded, and IoT devices. Key benefits include: + +- **On-device optimization**: Minimize latency and enhance privacy by processing data locally. +- **Platform compatibility**: Supports Android, iOS, embedded Linux, and MCU. +- **Performance**: Utilizes hardware acceleration to optimize model speed and efficiency. + +To learn more, check out the [TFLite guide](https://www.tensorflow.org/lite/guide). + +### Is it possible to run YOLOv8 TFLite models on Raspberry Pi? + +Yes, you can run YOLOv8 TFLite models on Raspberry Pi to improve inference speeds. First, export your model to TFLite format as explained [here](#how-do-i-export-a-yolov8-model-to-tflite-format). Then, use a tool like TensorFlow Lite Interpreter to execute the model on your Raspberry Pi. + +For further optimizations, you might consider using [Coral Edge TPU](https://coral.withgoogle.com/). For detailed steps, refer to our [Raspberry Pi deployment guide](../guides/raspberry-pi.md). + +### Can I use TFLite models on microcontrollers for YOLOv8 predictions? + +Yes, TFLite supports deployment on microcontrollers with limited resources. TFLite's core runtime requires only 16 KB of memory on an Arm Cortex M3 and can run basic YOLOv8 models. This makes it suitable for deployment on devices with minimal computational power and memory. + +To get started, visit the [TFLite Micro for Microcontrollers guide](https://www.tensorflow.org/lite/microcontrollers). + +### What platforms are compatible with TFLite exported YOLOv8 models? + +TensorFlow Lite provides extensive platform compatibility, allowing you to deploy YOLOv8 models on a wide range of devices, including: + +- **Android and iOS**: Native support through TFLite Android and iOS libraries. +- **Embedded Linux**: Ideal for single-board computers such as Raspberry Pi. +- **Microcontrollers**: Suitable for MCUs with constrained resources. + +For more information on deployment options, see our detailed [deployment guide](#deploying-exported-yolov8-tflite-models). + +### How do I troubleshoot common issues during YOLOv8 model export to TFLite? + +If you encounter errors while exporting YOLOv8 models to TFLite, common solutions include: + +- **Check package compatibility**: Ensure you're using compatible versions of Ultralytics and TensorFlow. Refer to our [installation guide](../quickstart.md). +- **Model support**: Verify that the specific YOLOv8 model supports TFLite export by checking [here](../modes/export.md). + +For additional troubleshooting tips, visit our [Common Issues guide](../guides/yolo-common-issues.md). diff --git a/ultralytics/docs/en/integrations/torchscript.md b/ultralytics/docs/en/integrations/torchscript.md new file mode 100644 index 0000000000000000000000000000000000000000..ff7650f387172069a77252d54ac8e989bb0173d1 --- /dev/null +++ b/ultralytics/docs/en/integrations/torchscript.md @@ -0,0 +1,204 @@ +--- +comments: true +description: Learn how to export Ultralytics YOLOv8 models to TorchScript for flexible, cross-platform deployment. Boost performance and utilize in various environments. +keywords: YOLOv8, TorchScript, model export, Ultralytics, PyTorch, deep learning, AI deployment, cross-platform, performance optimization +--- + +# YOLOv8 Model Export to TorchScript for Quick Deployment + +Deploying computer vision models across different environments, including embedded systems, web browsers, or platforms with limited Python support, requires a flexible and portable solution. TorchScript focuses on portability and the ability to run models in environments where the entire Python framework is unavailable. This makes it ideal for scenarios where you need to deploy your computer vision capabilities across various devices or platforms. + +Export to Torchscript to serialize your [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) models for cross-platform compatibility and streamlined deployment. In this guide, we'll show you how to export your YOLOv8 models to the TorchScript format, making it easier for you to use them across a wider range of applications. + +## Why should you export to TorchScript? + +![Torchscript Overview](https://github.com/ultralytics/ultralytics/assets/26833433/6873349d-c2f6-4620-b3cc-7b26b0698d0b) + +Developed by the creators of PyTorch, TorchScript is a powerful tool for optimizing and deploying PyTorch models across a variety of platforms. Exporting YOLOv8 models to [TorchScript](https://pytorch.org/docs/stable/jit.html) is crucial for moving from research to real-world applications. TorchScript, part of the PyTorch framework, helps make this transition smoother by allowing PyTorch models to be used in environments that don't support Python. + +The process involves two techniques: tracing and scripting. Tracing records operations during model execution, while scripting allows for the definition of models using a subset of Python. These techniques ensure that models like YOLOv8 can still work their magic even outside their usual Python environment. + +![TorchScript Script and Trace](https://github.com/ultralytics/ultralytics/assets/26833433/ea9ea24f-a3a9-44bb-aca7-9c358d7490d7) + +TorchScript models can also be optimized through techniques such as operator fusion and refinements in memory usage, ensuring efficient execution. Another advantage of exporting to TorchScript is its potential to accelerate model execution across various hardware platforms. It creates a standalone, production-ready representation of your PyTorch model that can be integrated into C++ environments, embedded systems, or deployed in web or mobile applications. + +## Key Features of TorchScript Models + +TorchScript, a key part of the PyTorch ecosystem, provides powerful features for optimizing and deploying deep learning models. + +![TorchScript Features](https://github.com/ultralytics/ultralytics/assets/26833433/44c7c5e3-1146-42db-952a-9060f070fead) + +Here are the key features that make TorchScript a valuable tool for developers: + +- **Static Graph Execution**: TorchScript uses a static graph representation of the model's computation, which is different from PyTorch's dynamic graph execution. In static graph execution, the computational graph is defined and compiled once before the actual execution, resulting in improved performance during inference. + +- **Model Serialization**: TorchScript allows you to serialize PyTorch models into a platform-independent format. Serialized models can be loaded without requiring the original Python code, enabling deployment in different runtime environments. + +- **JIT Compilation**: TorchScript uses Just-In-Time (JIT) compilation to convert PyTorch models into an optimized intermediate representation. JIT compiles the model's computational graph, enabling efficient execution on target devices. + +- **Cross-Language Integration**: With TorchScript, you can export PyTorch models to other languages such as C++, Java, and JavaScript. This makes it easier to integrate PyTorch models into existing software systems written in different languages. + +- **Gradual Conversion**: TorchScript provides a gradual conversion approach, allowing you to incrementally convert parts of your PyTorch model into TorchScript. This flexibility is particularly useful when dealing with complex models or when you want to optimize specific portions of the code. + +## Deployment Options in TorchScript + +Before we look at the code for exporting YOLOv8 models to the TorchScript format, let's understand where TorchScript models are normally used. + +TorchScript offers various deployment options for machine learning models, such as: + +- **C++ API**: The most common use case for TorchScript is its C++ API, which allows you to load and execute optimized TorchScript models directly within C++ applications. This is ideal for production environments where Python may not be suitable or available. The C++ API offers low-overhead and efficient execution of TorchScript models, maximizing performance potential. + +- **Mobile Deployment**: TorchScript offers tools for converting models into formats readily deployable on mobile devices. PyTorch Mobile provides a runtime for executing these models within iOS and Android apps. This enables low-latency, offline inference capabilities, enhancing user experience and data privacy. + +- **Cloud Deployment**: TorchScript models can be deployed to cloud-based servers using solutions like TorchServe. It provides features like model versioning, batching, and metrics monitoring for scalable deployment in production environments. Cloud deployment with TorchScript can make your models accessible via APIs or other web services. + +## Export to TorchScript: Converting Your YOLOv8 Model + +Exporting YOLOv8 models to TorchScript makes it easier to use them in different places and helps them run faster and more efficiently. This is great for anyone looking to use deep learning models more effectively in real-world applications. + +### Installation + +To install the required package, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions and best practices related to the installation process, check our [Ultralytics Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +### Usage + +Before diving into the usage instructions, it's important to note that while all [Ultralytics YOLOv8 models](../models/index.md) are available for exporting, you can ensure that the model you select supports export functionality [here](../modes/export.md). + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TorchScript format + model.export(format="torchscript") # creates 'yolov8n.torchscript' + + # Load the exported TorchScript model + torchscript_model = YOLO("yolov8n.torchscript") + + # Run inference + results = torchscript_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TorchScript format + yolo export model=yolov8n.pt format=torchscript # creates 'yolov8n.torchscript' + + # Run inference with the exported model + yolo predict model=yolov8n.torchscript source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, visit the [Ultralytics documentation page on exporting](../modes/export.md). + +## Deploying Exported YOLOv8 TorchScript Models + +After successfully exporting your Ultralytics YOLOv8 models to TorchScript format, you can now deploy them. The primary and recommended first step for running a TorchScript model is to utilize the YOLO("model.torchscript") method, as outlined in the previous usage code snippet. However, for in-depth instructions on deploying your TorchScript models in various other settings, take a look at the following resources: + +- **[Explore Mobile Deployment](https://pytorch.org/mobile/home/)**: The PyTorch Mobile Documentation provides comprehensive guidelines for deploying models on mobile devices, ensuring your applications are efficient and responsive. + +- **[Master Server-Side Deployment](https://pytorch.org/serve/getting_started.html)**: Learn how to deploy models server-side with TorchServe, offering a step-by-step tutorial for scalable, efficient model serving. + +- **[Implement C++ Deployment](https://pytorch.org/tutorials/advanced/cpp_export.html)**: Dive into the Tutorial on Loading a TorchScript Model in C++, facilitating the integration of your TorchScript models into C++ applications for enhanced performance and versatility. + +## Summary + +In this guide, we explored the process of exporting Ultralytics YOLOv8 models to the TorchScript format. By following the provided instructions, you can optimize YOLOv8 models for performance and gain the flexibility to deploy them across various platforms and environments. + +For further details on usage, visit [TorchScript's official documentation](https://pytorch.org/docs/stable/jit.html). + +Also, if you'd like to know more about other Ultralytics YOLOv8 integrations, visit our [integration guide page](../integrations/index.md). You'll find plenty of useful resources and insights there. + +## FAQ + +### What is Ultralytics YOLOv8 model export to TorchScript? + +Exporting an Ultralytics YOLOv8 model to TorchScript allows for flexible, cross-platform deployment. TorchScript, a part of the PyTorch ecosystem, facilitates the serialization of models, which can then be executed in environments that lack Python support. This makes it ideal for deploying models on embedded systems, C++ environments, mobile applications, and even web browsers. Exporting to TorchScript enables efficient performance and wider applicability of your YOLOv8 models across diverse platforms. + +### How can I export my YOLOv8 model to TorchScript using Ultralytics? + +To export a YOLOv8 model to TorchScript, you can use the following example code: + +!!! Example "Usage" + + === "Python" + + ```python + from ultralytics import YOLO + + # Load the YOLOv8 model + model = YOLO("yolov8n.pt") + + # Export the model to TorchScript format + model.export(format="torchscript") # creates 'yolov8n.torchscript' + + # Load the exported TorchScript model + torchscript_model = YOLO("yolov8n.torchscript") + + # Run inference + results = torchscript_model("https://ultralytics.com/images/bus.jpg") + ``` + + === "CLI" + + ```bash + # Export a YOLOv8n PyTorch model to TorchScript format + yolo export model=yolov8n.pt format=torchscript # creates 'yolov8n.torchscript' + + # Run inference with the exported model + yolo predict model=yolov8n.torchscript source='https://ultralytics.com/images/bus.jpg' + ``` + +For more details about the export process, refer to the [Ultralytics documentation on exporting](../modes/export.md). + +### Why should I use TorchScript for deploying YOLOv8 models? + +Using TorchScript for deploying YOLOv8 models offers several advantages: + +- **Portability**: Exported models can run in environments without the need for Python, such as C++ applications, embedded systems, or mobile devices. +- **Optimization**: TorchScript supports static graph execution and Just-In-Time (JIT) compilation, which can optimize model performance. +- **Cross-Language Integration**: TorchScript models can be integrated into other programming languages, enhancing flexibility and expandability. +- **Serialization**: Models can be serialized, allowing for platform-independent loading and inference. + +For more insights into deployment, visit the [PyTorch Mobile Documentation](https://pytorch.org/mobile/home/), [TorchServe Documentation](https://pytorch.org/serve/getting_started.html), and [C++ Deployment Guide](https://pytorch.org/tutorials/advanced/cpp_export.html). + +### What are the installation steps for exporting YOLOv8 models to TorchScript? + +To install the required package for exporting YOLOv8 models, use the following command: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required package for YOLOv8 + pip install ultralytics + ``` + +For detailed instructions, visit the [Ultralytics Installation guide](../quickstart.md). If any issues arise during installation, consult the [Common Issues guide](../guides/yolo-common-issues.md). + +### How do I deploy my exported TorchScript YOLOv8 models? + +After exporting YOLOv8 models to the TorchScript format, you can deploy them across a variety of platforms: + +- **C++ API**: Ideal for low-overhead, highly efficient production environments. +- **Mobile Deployment**: Use [PyTorch Mobile](https://pytorch.org/mobile/home/) for iOS and Android applications. +- **Cloud Deployment**: Utilize services like [TorchServe](https://pytorch.org/serve/getting_started.html) for scalable server-side deployment. + +Explore comprehensive guidelines for deploying models in these settings to take full advantage of TorchScript's capabilities. diff --git a/ultralytics/docs/en/integrations/weights-biases.md b/ultralytics/docs/en/integrations/weights-biases.md new file mode 100644 index 0000000000000000000000000000000000000000..a8d36c943b5bcdd9f2847e9a1a9ea12bc1d734d3 --- /dev/null +++ b/ultralytics/docs/en/integrations/weights-biases.md @@ -0,0 +1,244 @@ +--- +comments: true +description: Learn how to enhance YOLOv8 experiment tracking and visualization with Weights & Biases for better model performance and management. +keywords: YOLOv8, Weights & Biases, model training, experiment tracking, Ultralytics, machine learning, computer vision, model visualization +--- + +# Enhancing YOLOv8 Experiment Tracking and Visualization with Weights & Biases + +Object detection models like [Ultralytics YOLOv8](https://github.com/ultralytics/ultralytics) have become integral to many computer vision applications. However, training, evaluating, and deploying these complex models introduces several challenges. Tracking key training metrics, comparing model variants, analyzing model behavior, and detecting issues require substantial instrumentation and experiment management. + +
+
+
+
+ Watch: How to use Ultralytics YOLOv8 with Weights and Biases
+
+ +
+ +[Weights & Biases](https://wandb.ai/site) is a cutting-edge MLOps platform designed for tracking, visualizing, and managing machine learning experiments. It features automatic logging of training metrics for full experiment reproducibility, an interactive UI for streamlined data analysis, and efficient model management tools for deploying across various environments. + +## YOLOv8 Training with Weights & Biases + +You can use Weights & Biases to bring efficiency and automation to your YOLOv8 training process. + +## Installation + +To install the required packages, run: + +!!! Tip "Installation" + + === "CLI" + + ```bash + # Install the required packages for YOLOv8 and Weights & Biases + pip install --upgrade ultralytics==8.0.186 wandb + ``` + +For detailed instructions and best practices related to the installation process, be sure to check our [YOLOv8 Installation guide](../quickstart.md). While installing the required packages for YOLOv8, if you encounter any difficulties, consult our [Common Issues guide](../guides/yolo-common-issues.md) for solutions and tips. + +## Configuring Weights & Biases + +After installing the necessary packages, the next step is to set up your Weights & Biases environment. This includes creating a Weights & Biases account and obtaining the necessary API key for a smooth connection between your development environment and the W&B platform. + +Start by initializing the Weights & Biases environment in your workspace. You can do this by running the following command and following the prompted instructions. + +!!! Tip "Initial SDK Setup" + + === "CLI" + + ```bash + # Initialize your Weights & Biases environment + import wandb + wandb.login() + ``` + +Navigate to the Weights & Biases authorization page to create and retrieve your API key. Use this key to authenticate your environment with W&B. + +## Usage: Training YOLOv8 with Weights & Biases + +Before diving into the usage instructions for YOLOv8 model training with Weights & Biases, be sure to check out the range of [YOLOv8 models offered by Ultralytics](../models/index.md). This will help you choose the most appropriate model for your project requirements. + +!!! Example "Usage: Training YOLOv8 with Weights & Biases" + + === "Python" + + ```python + import wandb + from wandb.integration.ultralytics import add_wandb_callback + + from ultralytics import YOLO + + # Initialize a Weights & Biases run + wandb.init(project="ultralytics", job_type="training") + + # Load a YOLO model + model = YOLO("yolov8n.pt") + + # Add W&B Callback for Ultralytics + add_wandb_callback(model, enable_model_checkpointing=True) + + # Train and Fine-Tune the Model + model.train(project="ultralytics", data="coco8.yaml", epochs=5, imgsz=640) + + # Validate the Model + model.val() + + # Perform Inference and Log Results + model(["path/to/image1", "path/to/image2"]) + + # Finalize the W&B Run + wandb.finish() + ``` + +### Understanding the Code + +Let's understand the steps showcased in the usage code snippet above. + +- **Step 1: Initialize a Weights & Biases Run**: Start by initializing a Weights & Biases run, specifying the project name and the job type. This run will track and manage the training and validation processes of your model. + +- **Step 2: Define the YOLOv8 Model and Dataset**: Specify the model variant and the dataset you wish to use. The YOLO model is then initialized with the specified model file. + +- **Step 3: Add Weights & Biases Callback for Ultralytics**: This step is crucial as it enables the automatic logging of training metrics and validation results to Weights & Biases, providing a detailed view of the model's performance. + +- **Step 4: Train and Fine-Tune the Model**: Begin training the model with the specified dataset, number of epochs, and image size. The training process includes logging of metrics and predictions at the end of each epoch, offering a comprehensive view of the model's learning progress. + +- **Step 5: Validate the Model**: After training, the model is validated. This step is crucial for assessing the model's performance on unseen data and ensuring its generalizability. + +- **Step 6: Perform Inference and Log Results**: The model performs predictions on specified images. These predictions, along with visual overlays and insights, are automatically logged in a W&B Table for interactive exploration. + +- **Step 7: Finalize the W&B Run**: This step marks the end of data logging and saves the final state of your model's training and validation process in the W&B dashboard. + +### Understanding the Output + +Upon running the usage code snippet above, you can expect the following key outputs: + +- The setup of a new run with its unique ID, indicating the start of the training process. +- A concise summary of the model's structure, including the number of layers and parameters. +- Regular updates on important metrics such as box loss, cls loss, dfl loss, precision, recall, and mAP scores during each training epoch. +- At the end of training, detailed metrics including the model's inference speed, and overall accuracy metrics are displayed. +- Links to the Weights & Biases dashboard for in-depth analysis and visualization of the training process, along with information on local log file locations. + +### Viewing the Weights & Biases Dashboard + +After running the usage code snippet, you can access the Weights & Biases (W&B) dashboard through the provided link in the output. This dashboard offers a comprehensive view of your model's training process with YOLOv8. + +## Key Features of the Weights & Biases Dashboard + +- **Real-Time Metrics Tracking**: Observe metrics like loss, accuracy, and validation scores as they evolve during the training, offering immediate insights for model tuning. [See how experiments are tracked using Weights & Biases](https://imgur.com/D6NVnmN). + +- **Hyperparameter Optimization**: Weights & Biases aids in fine-tuning critical parameters such as learning rate, batch size, and more, enhancing the performance of YOLOv8. + +- **Comparative Analysis**: The platform allows side-by-side comparisons of different training runs, essential for assessing the impact of various model configurations. + +- **Visualization of Training Progress**: Graphical representations of key metrics provide an intuitive understanding of the model's performance across epochs. [See how Weights & Biases helps you visualize validation results](https://imgur.com/a/kU5h7W4). + +- **Resource Monitoring**: Keep track of CPU, GPU, and memory usage to optimize the efficiency of the training process. + +- **Model Artifacts Management**: Access and share model checkpoints, facilitating easy deployment and collaboration. + +- **Viewing Inference Results with Image Overlay**: Visualize the prediction results on images using interactive overlays in Weights & Biases, providing a clear and detailed view of model performance on real-world data. For more detailed information on Weights & Biases' image overlay capabilities, check out this [link](https://docs.wandb.ai/guides/track/log/media#image-overlays). [See how Weights & Biases' image overlays helps visualize model inferences](https://imgur.com/a/UTSiufs). + +By using these features, you can effectively track, analyze, and optimize your YOLOv8 model's training, ensuring the best possible performance and efficiency. + +## Summary + +This guide helped you explore Ultralytics' YOLOv8 integration with Weights & Biases. It illustrates the ability of this integration to efficiently track and visualize model training and prediction results. + +For further details on usage, visit [Weights & Biases' official documentation](https://docs.wandb.ai/guides/integrations/ultralytics). + +Also, be sure to check out the [Ultralytics integration guide page](../integrations/index.md), to learn more about different exciting integrations. + +## FAQ + +### How do I install the required packages for YOLOv8 and Weights & Biases? + +To install the required packages for YOLOv8 and Weights & Biases, open your command line interface and run: + +```bash +pip install --upgrade ultralytics==8.0.186 wandb +``` + +For further guidance on installation steps, refer to our [YOLOv8 Installation guide](../quickstart.md). If you encounter issues, consult the [Common Issues guide](../guides/yolo-common-issues.md) for troubleshooting tips. + +### What are the benefits of integrating Ultralytics YOLOv8 with Weights & Biases? + +Integrating Ultralytics YOLOv8 with Weights & Biases offers several benefits including: + +- **Real-Time Metrics Tracking:** Observe metric changes during training for immediate insights. +- **Hyperparameter Optimization:** Improve model performance by fine-tuning learning rate, batch size, etc. +- **Comparative Analysis:** Side-by-side comparison of different training runs. +- **Resource Monitoring:** Keep track of CPU, GPU, and memory usage. +- **Model Artifacts Management:** Easy access and sharing of model checkpoints. + +Explore these features in detail in the Weights & Biases Dashboard section above. + +### How can I configure Weights & Biases for YOLOv8 training? + +To configure Weights & Biases for YOLOv8 training, follow these steps: + +1. Run the command to initialize Weights & Biases: + ```bash + import wandb + wandb.login() + ``` +2. Retrieve your API key from the Weights & Biases website. +3. Use the API key to authenticate your development environment. + +Detailed setup instructions can be found in the Configuring Weights & Biases section above. + +### How do I train a YOLOv8 model using Weights & Biases? + +For training a YOLOv8 model using Weights & Biases, use the following steps in a Python script: + +```python +import wandb +from wandb.integration.ultralytics import add_wandb_callback + +from ultralytics import YOLO + +# Initialize a Weights & Biases run +wandb.init(project="ultralytics", job_type="training") + +# Load a YOLO model +model = YOLO("yolov8n.pt") + +# Add W&B Callback for Ultralytics +add_wandb_callback(model, enable_model_checkpointing=True) + +# Train and Fine-Tune the Model +model.train(project="ultralytics", data="coco8.yaml", epochs=5, imgsz=640) + +# Validate the Model +model.val() + +# Perform Inference and Log Results +model(["path/to/image1", "path/to/image2"]) + +# Finalize the W&B Run +wandb.finish() +``` + +This script initializes Weights & Biases, sets up the model, trains it, and logs results. For more details, visit the Usage section above. + +### Why should I use Ultralytics YOLOv8 with Weights & Biases over other platforms? + +Ultralytics YOLOv8 integrated with Weights & Biases offers several unique advantages: + +- **High Efficiency:** Real-time tracking of training metrics and performance optimization. +- **Scalability:** Easily manage large-scale training jobs with robust resource monitoring and utilization tools. +- **Interactivity:** A user-friendly interactive UI for data visualization and model management. +- **Community and Support:** Strong integration documentation and community support with flexible customization and enhancement options. + +For comparisons with other platforms like Comet and ClearML, refer to [Ultralytics integrations](../integrations/index.md). diff --git a/ultralytics/docs/en/macros/export-args.md b/ultralytics/docs/en/macros/export-args.md new file mode 100644 index 0000000000000000000000000000000000000000..43fc6ff6604a711b5b61d8477d17251bb2270bf3 --- /dev/null +++ b/ultralytics/docs/en/macros/export-args.md @@ -0,0 +1,14 @@ +| Argument | Type | Default | Description | +| ----------- | ---------------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `format` | `str` | `'torchscript'` | Target format for the exported model, such as `'onnx'`, `'torchscript'`, `'tensorflow'`, or others, defining compatibility with various deployment environments. | +| `imgsz` | `int` or `tuple` | `640` | Desired image size for the model input. Can be an integer for square images or a tuple `(height, width)` for specific dimensions. | +| `keras` | `bool` | `False` | Enables export to Keras format for TensorFlow SavedModel, providing compatibility with TensorFlow serving and APIs. | +| `optimize` | `bool` | `False` | Applies optimization for mobile devices when exporting to TorchScript, potentially reducing model size and improving performance. | +| `half` | `bool` | `False` | Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware. | +| `int8` | `bool` | `False` | Activates INT8 quantization, further compressing the model and speeding up inference with minimal accuracy loss, primarily for edge devices. | +| `dynamic` | `bool` | `False` | Allows dynamic input sizes for ONNX, TensorRT and OpenVINO exports, enhancing flexibility in handling varying image dimensions. | +| `simplify` | `bool` | `False` | Simplifies the model graph for ONNX exports with `onnxslim`, potentially improving performance and compatibility. | +| `opset` | `int` | `None` | Specifies the ONNX opset version for compatibility with different ONNX parsers and runtimes. If not set, uses the latest supported version. | +| `workspace` | `float` | `4.0` | Sets the maximum workspace size in GiB for TensorRT optimizations, balancing memory usage and performance. | +| `nms` | `bool` | `False` | Adds Non-Maximum Suppression (NMS) to the CoreML export, essential for accurate and efficient detection post-processing. | +| `batch` | `int` | `1` | Specifies export model batch inference size or the max number of images the exported model will process concurrently in `predict` mode. | diff --git a/ultralytics/docs/en/macros/export-table.md b/ultralytics/docs/en/macros/export-table.md new file mode 100644 index 0000000000000000000000000000000000000000..829f378fb679ec53b83dca291417e7bfb3e5f07f --- /dev/null +++ b/ultralytics/docs/en/macros/export-table.md @@ -0,0 +1,15 @@ +| Format | `format` Argument | Model | Metadata | Arguments | +| ------------------------------------------------- | ----------------- | ----------------------------------------------- | -------- | -------------------------------------------------------------------- | +| [PyTorch](https://pytorch.org/) | - | `{{ model_name or "yolov8n" }}.pt` | ✅ | - | +| [TorchScript](../integrations/torchscript.md) | `torchscript` | `{{ model_name or "yolov8n" }}.torchscript` | ✅ | `imgsz`, `optimize`, `batch` | +| [ONNX](../integrations/onnx.md) | `onnx` | `{{ model_name or "yolov8n" }}.onnx` | ✅ | `imgsz`, `half`, `dynamic`, `simplify`, `opset`, `batch` | +| [OpenVINO](../integrations/openvino.md) | `openvino` | `{{ model_name or "yolov8n" }}_openvino_model/` | ✅ | `imgsz`, `half`, `int8`, `batch` | +| [TensorRT](../integrations/tensorrt.md) | `engine` | `{{ model_name or "yolov8n" }}.engine` | ✅ | `imgsz`, `half`, `dynamic`, `simplify`, `workspace`, `int8`, `batch` | +| [CoreML](../integrations/coreml.md) | `coreml` | `{{ model_name or "yolov8n" }}.mlpackage` | ✅ | `imgsz`, `half`, `int8`, `nms`, `batch` | +| [TF SavedModel](../integrations/tf-savedmodel.md) | `saved_model` | `{{ model_name or "yolov8n" }}_saved_model/` | ✅ | `imgsz`, `keras`, `int8`, `batch` | +| [TF GraphDef](../integrations/tf-graphdef.md) | `pb` | `{{ model_name or "yolov8n" }}.pb` | ❌ | `imgsz`, `batch` | +| [TF Lite](../integrations/tflite.md) | `tflite` | `{{ model_name or "yolov8n" }}.tflite` | ✅ | `imgsz`, `half`, `int8`, `batch` | +| [TF Edge TPU](../integrations/edge-tpu.md) | `edgetpu` | `{{ model_name or "yolov8n" }}_edgetpu.tflite` | ✅ | `imgsz` | +| [TF.js](../integrations/tfjs.md) | `tfjs` | `{{ model_name or "yolov8n" }}_web_model/` | ✅ | `imgsz`, `half`, `int8`, `batch` | +| [PaddlePaddle](../integrations/paddlepaddle.md) | `paddle` | `{{ model_name or "yolov8n" }}_paddle_model/` | ✅ | `imgsz`, `batch` | +| [NCNN](../integrations/ncnn.md) | `ncnn` | `{{ model_name or "yolov8n" }}_ncnn_model/` | ✅ | `imgsz`, `half`, `batch` | diff --git a/ultralytics/docs/en/models/fast-sam.md b/ultralytics/docs/en/models/fast-sam.md new file mode 100644 index 0000000000000000000000000000000000000000..66cdb4d1bb29641f4e81619ef044a9da6fb223c2 --- /dev/null +++ b/ultralytics/docs/en/models/fast-sam.md @@ -0,0 +1,322 @@ +--- +comments: true +description: Discover FastSAM, a real-time CNN-based solution for segmenting any object in an image. Efficient, competitive, and ideal for various vision tasks. +keywords: FastSAM, Fast Segment Anything Model, Ultralytics, real-time segmentation, CNN, YOLOv8-seg, object segmentation, image processing, computer vision +--- + +# Fast Segment Anything Model (FastSAM) + +The Fast Segment Anything Model (FastSAM) is a novel, real-time CNN-based solution for the Segment Anything task. This task is designed to segment any object within an image based on various possible user interaction prompts. FastSAM significantly reduces computational demands while maintaining competitive performance, making it a practical choice for a variety of vision tasks. + +
+
+
+
+ Watch: Object Tracking using FastSAM with Ultralytics
+
+
+
+
+ Watch: Run Ultralytics YOLO models in just a few lines of code.
+
+
+
+
+ Watch: Real-Time Detection Transformer (RT-DETR)
+
+
+
+
+ Watch: YOLO World training workflow on custom dataset
+
+
+
+
+ Watch: How to Train YOLOv10 on SKU-110k Dataset using Ultralytics | Retail Dataset
+
+
+
+
+ Watch: Ultralytics YOLOv8 Model Overview
+
+
+
+
+ Watch: YOLOv9 Training on Custom Data using Ultralytics | Industrial Package Dataset
+
+
+
+
+ Watch: How to Benchmark the YOLOv9 Model Using the Ultralytics Python Package
+
+
+
+
+ Watch: Ultralytics Modes Tutorial: Benchmark
+
+
+
+
+ Watch: How To Export Custom Trained Ultralytics YOLOv8 Model and Run Live Inference on Webcam.
+
+
+
+
+ Watch: Ultralytics Modes Tutorial: Train, Validate, Predict, Export & Benchmark.
+
+
+
+
+ Watch: How to Extract the Outputs from Ultralytics YOLOv8 Model for Custom Projects.
+
+
+
+
+ Watch: Object Detection and Tracking with Ultralytics YOLOv8.
+
+
+
+
+ Watch: How to Train a YOLOv8 model on Your Custom Dataset in Google Colab.
+
+
+
+
+ Watch: Ultralytics Modes Tutorial: Validation
+
+
+
+
+ Watch: Ultralytics YOLO Quick Start Guide
+
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/ultralytics?logo=python&logoColor=gold)
+ + === "Pip install (recommended)" + + Install the `ultralytics` package using pip, or update an existing installation by running `pip install -U ultralytics`. Visit the Python Package Index (PyPI) for more details on the `ultralytics` package: [https://pypi.org/project/ultralytics/](https://pypi.org/project/ultralytics/). + + [![PyPI - Version](https://img.shields.io/pypi/v/ultralytics?logo=pypi&logoColor=white)](https://pypi.org/project/ultralytics/) + [![Downloads](https://static.pepy.tech/badge/ultralytics)](https://pepy.tech/project/ultralytics) + + ```bash + # Install the ultralytics package from PyPI + pip install ultralytics + ``` + + You can also install the `ultralytics` package directly from the GitHub [repository](https://github.com/ultralytics/ultralytics). This might be useful if you want the latest development version. Make sure to have the Git command-line tool installed on your system. The `@main` command installs the `main` branch and may be modified to another branch, i.e. `@my-branch`, or removed entirely to default to `main` branch. + + ```bash + # Install the ultralytics package from GitHub + pip install git+https://github.com/ultralytics/ultralytics.git@main + ``` + + === "Conda install" + + Conda is an alternative package manager to pip which may also be used for installation. Visit Anaconda for more details at [https://anaconda.org/conda-forge/ultralytics](https://anaconda.org/conda-forge/ultralytics). Ultralytics feedstock repository for updating the conda package is at [https://github.com/conda-forge/ultralytics-feedstock/](https://github.com/conda-forge/ultralytics-feedstock/). + + [![Conda Version](https://img.shields.io/conda/vn/conda-forge/ultralytics?logo=condaforge)](https://anaconda.org/conda-forge/ultralytics) + [![Conda Downloads](https://img.shields.io/conda/dn/conda-forge/ultralytics.svg)](https://anaconda.org/conda-forge/ultralytics) + [![Conda Recipe](https://img.shields.io/badge/recipe-ultralytics-green.svg)](https://anaconda.org/conda-forge/ultralytics) + [![Conda Platforms](https://img.shields.io/conda/pn/conda-forge/ultralytics.svg)](https://anaconda.org/conda-forge/ultralytics) + + ```bash + # Install the ultralytics package using conda + conda install -c conda-forge ultralytics + ``` + + !!! Note + + If you are installing in a CUDA environment best practice is to install `ultralytics`, `pytorch` and `pytorch-cuda` in the same command to allow the conda package manager to resolve any conflicts, or else to install `pytorch-cuda` last to allow it override the CPU-specific `pytorch` package if necessary. + ```bash + # Install all packages together using conda + conda install -c pytorch -c nvidia -c conda-forge pytorch torchvision pytorch-cuda=11.8 ultralytics + ``` + + ### Conda Docker Image + + Ultralytics Conda Docker images are also available from [DockerHub](https://hub.docker.com/r/ultralytics/ultralytics). These images are based on [Miniconda3](https://docs.conda.io/projects/miniconda/en/latest/) and are an simple way to start using `ultralytics` in a Conda environment. + + ```bash + # Set image name as a variable + t=ultralytics/ultralytics:latest-conda + + # Pull the latest ultralytics image from Docker Hub + sudo docker pull $t + + # Run the ultralytics image in a container with GPU support + sudo docker run -it --ipc=host --gpus all $t # all GPUs + sudo docker run -it --ipc=host --gpus '"device=2,3"' $t # specify GPUs + ``` + + === "Git clone" + + Clone the `ultralytics` repository if you are interested in contributing to the development or wish to experiment with the latest source code. After cloning, navigate into the directory and install the package in editable mode `-e` using pip. + + [![GitHub last commit](https://img.shields.io/github/last-commit/ultralytics/ultralytics?logo=github)](https://github.com/ultralytics/ultralytics) + [![GitHub commit activity](https://img.shields.io/github/commit-activity/t/ultralytics/ultralytics)](https://github.com/ultralytics/ultralytics) + + ```bash + # Clone the ultralytics repository + git clone https://github.com/ultralytics/ultralytics + + # Navigate to the cloned directory + cd ultralytics + + # Install the package in editable mode for development + pip install -e . + ``` + + === "Docker" + + Utilize Docker to effortlessly execute the `ultralytics` package in an isolated container, ensuring consistent and smooth performance across various environments. By choosing one of the official `ultralytics` images from [Docker Hub](https://hub.docker.com/r/ultralytics/ultralytics), you not only avoid the complexity of local installation but also benefit from access to a verified working environment. Ultralytics offers 5 main supported Docker images, each designed to provide high compatibility and efficiency for different platforms and use cases: + + [![Docker Image Version](https://img.shields.io/docker/v/ultralytics/ultralytics?sort=semver&logo=docker)](https://hub.docker.com/r/ultralytics/ultralytics) + [![Docker Pulls](https://img.shields.io/docker/pulls/ultralytics/ultralytics)](https://hub.docker.com/r/ultralytics/ultralytics) + + - **Dockerfile:** GPU image recommended for training. + - **Dockerfile-arm64:** Optimized for ARM64 architecture, allowing deployment on devices like Raspberry Pi and other ARM64-based platforms. + - **Dockerfile-cpu:** Ubuntu-based CPU-only version suitable for inference and environments without GPUs. + - **Dockerfile-jetson:** Tailored for NVIDIA Jetson devices, integrating GPU support optimized for these platforms. + - **Dockerfile-python:** Minimal image with just Python and necessary dependencies, ideal for lightweight applications and development. + - **Dockerfile-conda:** Based on Miniconda3 with conda installation of ultralytics package. + + Below are the commands to get the latest image and execute it: + + ```bash + # Set image name as a variable + t=ultralytics/ultralytics:latest + + # Pull the latest ultralytics image from Docker Hub + sudo docker pull $t + + # Run the ultralytics image in a container with GPU support + sudo docker run -it --ipc=host --gpus all $t # all GPUs + sudo docker run -it --ipc=host --gpus '"device=2,3"' $t # specify GPUs + ``` + + The above command initializes a Docker container with the latest `ultralytics` image. The `-it` flag assigns a pseudo-TTY and maintains stdin open, enabling you to interact with the container. The `--ipc=host` flag sets the IPC (Inter-Process Communication) namespace to the host, which is essential for sharing memory between processes. The `--gpus all` flag enables access to all available GPUs inside the container, which is crucial for tasks that require GPU computation. + + Note: To work with files on your local machine within the container, use Docker volumes for mounting a local directory into the container: + + ```bash + # Mount local directory to a directory inside the container + sudo docker run -it --ipc=host --gpus all -v /path/on/host:/path/in/container $t + ``` + + Alter `/path/on/host` with the directory path on your local machine, and `/path/in/container` with the desired path inside the Docker container for accessibility. + + For advanced Docker usage, feel free to explore the [Ultralytics Docker Guide](./guides/docker-quickstart.md). + +See the `ultralytics` [pyproject.toml](https://github.com/ultralytics/ultralytics/blob/main/pyproject.toml) file for a list of dependencies. Note that all examples above install all required dependencies. + +!!! Tip "Tip" + + PyTorch requirements vary by operating system and CUDA requirements, so it's recommended to install PyTorch first following instructions at [https://pytorch.org/get-started/locally](https://pytorch.org/get-started/locally). + + +