mteb/FreshStackRetrieval huggingface dataset:
I have a YOLOv7 model trained on my custom dataset. I exported the model to TensorFlow lite successfully and was able to use it for inference in Python. But when I try to use the same model in Android, using the object detection project with TensorFlow lite, it throws this error:
java.lang.IllegalArgumentException: Error occurred when initializing ObjectDetector: The input tensor should have dimensions 1 x height x width x 3. Got 1 x 3 x 640 x 640.
Is it possible to change the input shape for the ObjectDetector class, or export the YOLOv7 or YOLOv5 model with corresponding input shape?
I tried to tweak the export process to change the input shape of ONNX model which is the intermediate model in exporting from PyTorch to TensorFlow Lite but it throws this error:
ONNX export failure: Given groups=1, weight of size [32, 3, 3, 3], expected input[1, 640, 640, 3] to have 3 channels, but got 640 channels instead
update: I used onnx2tf to export .tflite model with NHWC input shape. Now the Android project throws this error:
java.lang.RuntimeException: Error occurred when initializing ObjectDetector: Input tensor has type kTfLiteFloat32: it requires specifying NormalizationOptions metadata to preprocess input images.
I couldn't find a way to add normalization options to metadata using this doc. Any solutions?
I have a trained model and I have detected my required object using following code
import cv2
from PIL import Image
from ultralytics import YOLO

image = cv2.imread("screenshot.png")
model = YOLO('runs/detect/train4/weights/best.pt')
results = model.predict(image, show=True, stream=True, classes=0, imgsz=512)
for result in results:
    for box in result.boxes:
        class_id = result.names[box.cls[0].item()]
        if (class_id == "myclassname"): 
            cords = box.xyxy[0].tolist()
            cords = [round(x) for x in cords]
            conf = round(box.conf[0].item(), 2)
            print("Object type:", class_id)
            print("Coordinates:", cords)
            print("Probability:", conf)
            print("---")

From this detected portion of image I need to detect an other class how I can do that?
I have searched enough but I could not see any post for this.
I trained a model to detect traffic lights and classify thier color, [red,green,yellow,off].
I want to only display the lights with a confidence above 50%, but I cant figure out how to do that with yolo v8. Ive tried using .conf and .prob as the documentation states but its all empty. Here is my current script. It streams webcam view and looks for stoplights.
Any help to get confidence values or even just the classification values from this would be amazing. I have uploaded the model to github here for people that want to test.
I reccomend using the best_traffic_nano_yolo.pt model as its the most lightweight.
Googling traffic light at intersection and holding an your phone up to the webcam should be enough for the model to detect and classify the light.
import cv2
from PIL import Image
from ultralytics import YOLO

# Load a pretrained YOLOv8n model
model = YOLO('/Models/best_traffic_nano_yolo.pt')

# open a video file or start a video stream
cap = cv2.VideoCapture(0)  # replace with 0 for webcam

while cap.isOpened():
    # Capture frame-by-frame
    ret, frame = cap.read()
    if not ret:
        break

    # flip the image
    # frame = cv2.flip(frame, -1) 

    # Run inference on the current frame
    results = model(frame)  # results list

    for r in results:
        frame = r.plot()

    # Display the resulting frame
    cv2.imshow('frame', frame)
    
    # Press 'q' on keyboard to  exit
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# After the loop release the cap object and destroy all windows
cap.release()
cv2.destroyAllWindows()
I am currently working with Ultralytics Yolov8 model for object detection.
I trained my model with a custom dataset. I annotated my dataset on Roboflow and exported in Yolov8 format.
The training result looks good. However there is a mismatch between the coordinates of bounding boxes of my manual annotation on Roboflow and that of prediction.
For example I have this image called image_1.jpg which is annotated on Roboflow and is part of training set. Roboflow defined the coordinates (as the part of the label):

[0 0.36953125 0.39609375 0.54765625 0.7875]

where the last 4 numbers showing the x1,x2,y1,y2 coordinates of the bounding boxes. However when I passed this image into predict method (just for checking) using the following piece of code
model = YOLO(‘best_weights.pt’)
result = model.predict(path_of_example_image, save = True)
boxes = result[0].boxes.xyxy

I got the coordinates:

tensor([[ 22.5558,   9.7429, 619.7601, 345.7614]])

When I draw the bounding box with the  predicted coordinates, the area covers the object completely (with .99% accuracy on the (correctly) predicted class). On the other hand, when I draw the bounding box with the coordinates from manual annotation, the object is not covered at all as, obviously, all coordinates are below 1. So I just wonder what kind of preprocessing is applied within the model so that the coordinates from Roboflow's labelling make sense and my training result is actually good?
So, I'm trying to train a YOLO classification model on a custom dataset that contains jpg-images of two different classes. But when I launch training, I get following error:
Traceback (most recent call last):
  File "C:\pythonProject\main.py", line 5, in <module>
    model.train(data='./training_dataset', epochs=1, imgsz=64)
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\engine\model.py", line 341, in train
    self.trainer.train()
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\engine\trainer.py", line 192, in train
    self._do_train(world_size)
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\engine\trainer.py", line 288, in _do_train
    self._setup_train(world_size)
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\engine\trainer.py", line 255, in _setup_train
    self.test_loader = self.get_dataloader(self.testset, batch_size=batch_size * 2, rank=-1, mode='val')
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\models\yolo\classify\train.py", line 88, in get_dataloader
    dataset = self.build_dataset(dataset_path, mode)
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\models\yolo\classify\train.py", line 83, in build_dataset
    return ClassificationDataset(root=img_path, args=self.args, augment=mode == 'train', prefix=mode)
  File "C:\pythonProject\venv\lib\site-packages\ultralytics\data\dataset.py", line 220, in __init__
    super().__init__(root=root)
  File "C:\pythonProject\venv\lib\site-packages\torchvision\datasets\folder.py", line 309, in __init__
    super().__init__(
  File "C:\pythonProject\venv\lib\site-packages\torchvision\datasets\folder.py", line 145, in __init__
    samples = self.make_dataset(self.root, class_to_idx, extensions, is_valid_file)
  File "C:\pythonProject\venv\lib\site-packages\torchvision\datasets\folder.py", line 189, in make_dataset
    return make_dataset(directory, class_to_idx, extensions=extensions, is_valid_file=is_valid_file)
  File "C:\pythonProject\venv\lib\site-packages\torchvision\datasets\folder.py", line 61, in make_dataset
    directory = os.path.expanduser(directory)
  File "C:\Users\User\AppData\Local\Programs\Python\Python310\lib\ntpath.py", line 293, in expanduser
    path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType


I've tried two different datasets and that problem occured on both of them.
My full code:
from ultralytics import YOLO

model = YOLO('yolov8n-cls.pt')

model.train(data='./training_dataset', epochs=1, imgsz=64)

Dataset structure:
training_dataset/train/cats;
training_dataset/train/dogs
I have trained my YoloV8 detection model.
After training the model there are plenty of files stored within in the train folder.
One of the files are the train_batch.jpg and the val_batch.jpg.
The images within the train_batch.jpg are cutted of and scrambled.

The images within the val_batch.jpg looks fine.

The imagesize of the images within train_batch and val_batch are the same.
Does someone know if this is a problem?
I have tried to change the image sizes and used various batch sizes.
What are the class IDs and their corresponding class names for YOLOv8 models? I understand there are approximately 80 classes in the object detection model of YOLOv8. However, I'm looking to specifically identify each class along with their respective class IDs. Additionally, do all YOLO models (yolov3, yolov5, yolov7, yolov8) have the same number of classes and corresponding class IDs?
I need some help as I will be needing this to work for my final thesis project for the Model has 2 classes inheat and non-inheat, I did the codes here below but when it tries to predict or detect by frames of the video it detects the non-inheat but it only stacks in the ininheat frames instead of the non_ inheat frames I might be doing a wrong process here for by frames in stacking them up even its detecting non-inheat it stacks to the in_heat frames.
codes:
def process_video_with_second_model(video_path):
    cap = cv2.VideoCapture(video_path)
    class_counts = {'inheat': 0, 'non-inheat': 0}

    in_heat_frames = []
    non_in_heat_frames = []

    while True:
        ret, frame = cap.read()
        if frame is None:
            break # Break the loop when no more frames are available

        # Resize the frame to a smaller size (e.g., 400x400)
        frame_small = cv2.resize(frame, (400, 400))

        # Use the second model to detect in-heat behavior
        results_in_heat = yolov8_model_in_heat.predict(source=frame_small, show=True, conf=0.8)

        # Print results to inspect structure
        for results_in_heat_instance in results_in_heat:
            # Access bounding box coordinates
            boxes = results_in_heat_instance.boxes

            # CONFIDENCE 0.5
            if len(boxes) > 0:
                class_name = results_in_heat_instance.names[0]

                # Use a dictionary to store the counts for each class
                class_counts[class_name] += 1

                # Add the frame to the corresponding list based on the class name
                if class_name == 'non-inheat':
                    non_in_heat_frames.append(frame)
                elif class_name == 'inheat':
                    in_heat_frames.append(frame)

        print(f"Class Counts: {class_counts}")

        # Check if either condition is met (50 frames for inheat and 50 frames for non-inheat)
        if class_counts['inheat'] >= 50 and class_counts['non-inheat'] >= 50:
            break

    # Release resources for the second model
    cap.release()
    cv2.destroyAllWindows()

    # Stack the in-heat and non-in-heat frames vertically
    stacked_in_heat_frames = np.vstack(in_heat_frames)
    stacked_non_in_heat_frames = np.vstack(non_in_heat_frames)

    # Display the stacked in-heat and non-in-heat frames
    cv2.imshow('Stacked In-Heat Frames', stacked_in_heat_frames)
    cv2.imshow('Stacked Non-In-Heat Frames', stacked_non_in_heat_frames)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

    # Compare the counts and return the label with the higher count
    if class_counts['inheat'] > class_counts['non-inheat']:
        return 'inheat'
    elif class_counts['non-inheat'] > class_counts['inheat']:
        return 'non-inheat'

I did read the YOLOv8 Documents for predicting but still cant do it. I did this in the Pycharm IDE
I am using Ultralytics YOLO for license plate detection, and I'm encountering an issue when trying to extract bounding box coordinates from the Results.boxes object. I have inspected the structure of the Results.boxes object, but I am having difficulty accessing the bounding box information correctly.
class ImageProcessing:
    def __init__(self, model_path: Path, input_image: Path, output_image: Path):
        if not isinstance(model_path, Path):
            raise TypeError("model_path must be a pathlib.Path instance")
        if not isinstance(input_image, Path) or not isinstance(output_image, Path):
            raise TypeError("input_image and output_image must be pathlib.Path instances")
        # Load the YOLO model from the provided path
        self.model = YOLO(str(model_path))
        self.input_image = input_image
        self.output_image = output_image

    def ascertain_license_plates_as_image(self, threshold: float = 0.5, fontscale: float = 1.3, color: tuple = (0, 255, 0), thickness: int = 3):
        image = opencv.imread(str(self.input_image))
        results = self.model(image)

        # Check if results is a list and get the first result
        if isinstance(results, list):
            results = results[0]

        # Iterate through each detected object
        for box in results.boxes:
            # Extract coordinates, confidence, and class ID
            x1, y1, x2, y2, conf, class_id = box.data[0][0], box.data[0][1], box.data[0][2], box.data[0][3], box.conf.item(), int(box.cls.item())
            if conf > threshold:
                opencv.rectangle(image, (int(x1), int(y1)), (int(x2), int(y2)), color, thickness)
                label = results.names[class_id].upper() if results.names else f'class {class_id}'
                opencv.putText(image, label, (int(x1), int(y1) - 10), opencv.FONT_HERSHEY_SIMPLEX, fontscale, color, thickness, opencv.LINE_AA)

        opencv.imwrite(str(self.output_image), image)
        return results

However, I'm getting an IndexError, and it seems that my indexing might be incorrect for this particular Boxes object. Or even worse, cv2 is not highlighting the license plate.
I’ve custom trained a yolov8n model and I would like to run it on a Jetson Nano. Has anyone managed to do this? If yes, would you be so kind to help me out?
I’ve got a .pt of the custom trained model. The OS image offered by NVidia on their website is an Ubuntu 18.04 and I have run into many compatibility issues. I’m interested in finding out if anyone has managed to get yolo running on the Jetson specifically the yolov8n model from ultralytics.
Thanks in advance!
I'm currently working on an object detection project using YOLOv8 and a customized dataset. In my dataset, each image is accompanied by its corresponding label file (Data_1.png -> Data_1.txt).
The label file Data_1.txt follows the format:
Class_type, x_1 min, y_1 min, ..., x_4 min, y_4 min.

I'm interested in applying a perspective augmentation to my dataset, and I've decided to use the YOLOv8 augmentation functionality. However, I am unsure whether YOLOv8 generates labels for the augmented data or not. If it does not, I would greatly appreciate any suggestions or alternative approaches to handle the generation of labels for the augmented data.
How do I get the class names of segmented instances when detecting multiple classes in YOLOv8? The detections do not have a .cls attribute like here YOLOv8 get predicted class name. Also the docs do not seem to mention anything e.g. here
When I use the show=true argument in the prediction function, the classes are distinguished in the resulting image, but I cannot get them programmatically. My code that gets me all detections I wanjt but does not let me know which one is which:
from ultralytics import YOLO
model = YOLO("path/to/best.pt")
result = model.predict(os.path.join(cut_dir, im_name), save_conf=True, show=True)
if result[0].masks is not None:
    for counter, detection in enumerate(result[0].masks.data):
         detected = np.asarray(detection.cpu())
I want to close the detection window on a key press and not have to stop the entire code. I am able to do it for my image detection but since video runs on a loop I can't find how.
            root.video = filedialog.askopenfilename(initialdir="/desktop", title="Select video to detect")
            cap = cv2.VideoCapture(root.video)

            model = YOLO('best.pt')

            classNames = ['Orange', 'Orange', 'Pomegranate', 'Pomegranate', 'apple', 'apple', 'banana', 'banana',
                          'fruits', 'guava', 'guava', 'guava', 'lime', 'lime']
            while True:
                success, vid = cap.read()
                results = model(img, stream=True)

                        cv2.imshow("DetectVideo", vid)
                        cv2.waitKey(1)

I have tried the following
k = cv2.waitKey(0) #also tried with (1)
print(k)
if k == 27:  # close on ESC key
    cv2.destroyAllWindows()

which basically makes it so it just closes each frame of the video then continues to next frame on key press.
def extract_and_process_tracks(self, tracks):
    boxes = tracks[0].boxes.xyxy.cpu()
    clss = tracks[0].boxes.cls.cpu().tolist()
    track_ids = tracks[0].boxes.id.int().cpu().tolist()

    self.annotator = Annotator(self.im0, self.tf, self.names)
    self.annotator.draw_region(reg_pts=self.reg_pts, color=(0, 255, 0))

    for box, track_id, cls in zip(boxes, track_ids, clss):
        self.annotator.box_label(box, label=self.names[cls], color=colors(int(cls), True))  

        # Draw Tracks
        track_line = self.track_history[track_id]
        track_line.append((float((box[0] + box[2]) / 2), float((box[0] + box[2]) / 2))
        track_line.pop(0) if len(track_line) > 30 else None

        if self.draw_tracks:
            self.annotator.draw_centroid_and_tracks(track_line,
                                                    color=(0, 255, 0),
                                                    track_thickness=self.track_thickness)

The object_counter.py provided by ultralytics can achieve the counting job, the track_line.append store the center of the
box(float((box[0] + box[2]) / 2), float((box[0] + box[2]) / 2),
but how to change the center into the keypoints coordinate, e.g. I want to count the head keypoints of animal cross a specified line.
How to get the keypoints coordinate in yolo_pose?
can you please help me.........
I have my custom trained model (best.pt), it detects two things person and headlight. Now I want the output according to these conditions: 1. If model detect only headlight return 0, 2. If model detect only person return 1, 3. If model detect headlight and person both return 0.
import cv2
from ultralytics import YOLO

video_path = 'data/video1.mp4'
video_out_path = 'out.mp4'

cap = cv2.VideoCapture(video_path)

# Check if the video file is opened successfully
if not cap.isOpened():
    print("Error: Could not open the video file.")
    exit()

ret, frame = cap.read()

# Check if the first frame is read successfully
if not ret:
    print("Error: Could not read the first frame from the video.")
    exit()

cap_out = cv2.VideoWriter(video_out_path, cv2.VideoWriter_fourcc(*'MP4V'), cap.get(cv2.CAP_PROP_FPS),
                          (int(cap.get(3)), int(cap.get(4))))  # Use cap.get(3) and cap.get(4) for width and height

model = YOLO("bestall5.pt")

detection_threshold = 0.5
while ret:
    results_list = model(frame)

    headlight_detected = False
    person_detected = False

    # Iterate through the list of results
    for results in results_list:
        # Check if the current result has the necessary attributes
        if hasattr(results, 'xyxy'):
            for result in results.xyxy:
                x1, y1, x2, y2, score, class_id = result.tolist()
                x1, x2, y1, y2 = int(x1), int(x2), int(y1), int(y2)

                # Assuming class_id is the index of the class in the model's class list
                class_name = model.names[class_id]

                if class_name == "headlight" and score > detection_threshold:
                    headlight_detected = True
                elif class_name == "person" and score > detection_threshold:
                    person_detected = True

    # Output based on the specified conditions
    if headlight_detected and person_detected:
        output = 0
    elif headlight_detected:
        output = 0
    elif person_detected:
        output = 1
    else:
        output = -1  # No person or headlight detected

    print("Output:", output)

    cap_out.write(frame)

    cv2.imshow('Object Detection', frame)
    
    # Break the loop if 'q' key is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

    ret, frame = cap.read()

cap.release()
cap_out.release()
cv2.destroyAllWindows()

I tried this but getting only -1 as output but my video has both headlight and person
I am currently working with YOLOv8 and I'm wondering if there is a method similar to results.pandas().xyxy available in YOLOv5 to obtain structured results in tabular form.
With YOLOv5, it's possible to get results easily using the following code:
results = model(img)
df = results.pandas().xyxy[0]  # Get results in tabular format
print(df)
#      xmin    ymin    xmax   ymax  confidence  class    name
# 0  749.50   43.50  1148.0  704.5    0.874023      0  person
# 1  433.50  433.50   517.5  714.5    0.687988     27     tie
# 2  114.75  195.75  1095.0  708.0    0.624512      0  person
# 3  986.00  304.00  1028.0  420.0    0.286865     27     tie
When my nn starts learning, the kernel crashes
The Kernel crashed while executing code in the current cell or a previous cell. 

Please review the code in the cell(s) to identify a possible cause of the failure. 

Click here for more info. 

View Jupyter log for further details.

Yolo doesn't use my GPU resources for learning instead it trying to use CPU and I think that it is a reason why kernel crashes.
How I can fix it?
screenshot
I tried to install CUDA 7.5 (for my RTX 2060). I recently started to study neural networks and I am not sure that I need CUDA.
I also checked similar topics here but I am not sure that there is my case.
Let's say I have a folder called 'test' with folders inside, 'images' and 'labels'. I also have a YOLOv8 model which I've trained called 'best.pt. My labels are polygons (yolo-obb .txt files).
I want to find the mean average precision (MAP) of my YOLOv8 model on this test set.
I've read both the documentation for predicting and benchmarking, however, I'm struggling to find an example of calculating map from some test images.
https://docs.ultralytics.com/modes/predict/
https://docs.ultralytics.com/modes/benchmark/
from ultralytics import YOLO

# Load a pretrained YOLOv8n model
model = YOLO('best.pt')

# Run inference on an image
results = model(['test/images/bus.jpg', 'test/images/zidane.jpg'])  # list of 2 Results objects

I imagine I have to put the list of images in the above, then write code to calculate map for everything in the test folder and average it. Are there packages that have already done this?
What's the code to achieve this task?