Repository: keras-team/autokeras

Issue:
ObjectDetection
<!---
Please label your issue with `new_task_module`.
-->

### Task Module: Object Detection

### Task Description
* Input: one image or more, each of them contains one or more objects need to be detected.
* Output: for each input image, this module output one or more bounding boxes of objects to be detected, and the class name they belong to.

### Evaluation Metrics: mAP (mean Average Precision)
1. For each detected object, the bounding box IoU (intersection over union) is calculated, which is between 0~1
2. The detection is accepted iff: 1) classification is correct, 2) the IoU is great or equal to the IoU threshold
3. If there exists multiple IoU thresholds, then the precision is averaged over all thresholds
4. For each class we calculate its average precision (AP), and then we average all classes to get the mAP


### Benchmark Datasets
1. [PASCAL Visual Object Classification (PASCAL VOC) dataset](http://host.robots.ox.ac.uk/pascal/VOC/)
2. [Common Objects in COntext (COCO) dataset](http://cocodataset.org/#home)

### Reason
Object detection is one of the most important research areas and applications in Computer Vision, such as face detection, vehicle detection, people counting. This module will provide easy access to people or group who needs to perform object detection in their work with limited Computer Vision or Machine Learning background.

### Solution
Some add-ons on current AutoKeras to be considered:
1. Add RoI layer into search space and IoU loss function.
2. Consider using some popular architectures as initializations (e.g. SSD, YoLo).
3. Consider adding complexity constraint (e.g. depth, width) in searching metrics. Currently the accuracy is the only metrics for the model selection.



Discussion:

Resolving patch:
diff --git a/autokeras/pretrained/object_detector.py b/autokeras/pretrained/object_detector.py
--- a/autokeras/pretrained/object_detector.py
+++ b/autokeras/pretrained/object_detector.py
@@ -539,13 +539,15 @@ def predict(self, img_path, output_file_path=None):
         """
         
         Returns:
-            List of tuples. Each tuple is like ((x1, y1), (h, w), category, confidence).
+            List of dictionaries. Each dictionary is like
+            {"left": int, "top": int, "width": int, "height": int: "category": str, "confidence": float}
         """
         from matplotlib.ticker import NullLocator
 
         dataset_mean = (104, 117, 123)
 
         image = cv2.imread(img_path, cv2.IMREAD_COLOR)
+        height, width, _ = image.shape
         rgb_image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
         x = base_transform(rgb_image, 300, dataset_mean)
         x = x.astype(np.float32)
@@ -566,8 +568,16 @@ def predict(self, img_path, output_file_path=None):
                 score = detections[0, i, j, 0].item()
                 label_name = VOC_CLASSES[i - 1]
                 pt = (detections[0, i, j, 1:] * scale).cpu().numpy()
-                result = ((pt[0], pt[1]), (pt[2] - pt[0] + 1, pt[3] - pt[1] + 1), label_name, score)
-                results.append(set(result))
+                # result = ((pt[0], pt[1]), (pt[2] - pt[0] + 1, pt[3] - pt[1] + 1), label_name, score)
+                result = {
+                          "left": max(int(np.round(pt[0])), 0),
+                          "top": max(int(np.round(pt[1])), 0),
+                          "width": min(int(np.round(pt[2] - pt[0] + 1)), width),
+                          "height": min(int(np.round(pt[3] - pt[1] + 1)), height),
+                          "category": label_name,
+                          "confidence": score
+                         }
+                results.append(result)
                 j += 1
 
         if output_file_path is not None:
diff --git a/examples/object_detection/object_detection_example.py b/examples/object_detection/object_detection_example.py
new file mode 100644
--- /dev/null
+++ b/examples/object_detection/object_detection_example.py
@@ -0,0 +1,8 @@
+from autokeras.pretrained.object_detector import ObjectDetector
+
+if __name__ == '__main__':
+    detector = ObjectDetector()
+    detector.load()
+    results = detector.predict("example.jpg", output_file_path="./")
+
+    print(results)


Test patch: