Kati8/ICLR_2023_2025 huggingface dataset:
authors: Vagner Takiya Bonadio
authors: Geovane Fedrecheski
authors: Laisa Costa
authors: Marcelo Knorich Zuffo
reviews: 
Comments: The proposed work focuses on a smart surveillance application deployed on resource constraint devices with limited compute and memory capabilities. To address that, a pipeline-based object detection application is proposed. Specifically, size and correlation between detected objects is considered for passing to next phases of the pipeline. The goal for such approach is to improve the inference speed and reduce the number of false positives and negatives.
Strengths: 1) The problem of running advanced computer vision applications on resource constraints devices is very important especially in latency critical applications, where going to the cloud is not an option. Thus, the fact that this problem is emphasized in this work is really good. 
2) Also, the method was tested in real application, operating in the field, showing its usefulness in the selected market vertical. 
3) Description of the existing methods is very detailed with clear motivation for the work.
Weaknesses: 1) The proposed approach doesn't introduce anything novel to the ML research. Such implementation approach is pretty standard when deploying ML solutions in practice, and is tackling the software development portion of the application stack, not ML. Also, from the software implementation perspective the contribution is not very innovative, as other deployments usually use pipelined approached with model ensembles, loops and gating mechanisms when deciding what models to invoke.
2) The evaluation section is not comprehensive enough. The method should be compared with other techniques and not pipelined systems to justify the claims. Also, the abstract specifies that the inference time and mAP were improved, but it's not presented anywhere in the results section.
Recommendation: 1: strong reject
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 1 poor
Clarity: 3 good
reviews: 
Comments: The research paper introduces a pipeline-based approach for object detection on resource-constrained IoT devices, addressing the challenges associated with limited power and processing capacity. The proposed approach leverages computer vision and AI methods, considering object size and correlation to improve object detection in edge computing environments. The authors trained YOLO CNNs on their dataset and evaluated the approach in a mobile surveillance system. Pilot testing was conducted in collaboration with the police department, with a focus on license plate reading performance.
Strengths: -	The paper addresses a significant issue concerning object detection on resource-constrained IoT devices in the context of smart surveillance systems. Ensuring safety in metropolitan areas is a pressing concern, and the proposed approach aligns with the needs of smart cities and law enforcement.
-	Collaborative testing with the police department in real-world scenarios adds credibility to the research and demonstrates the practical applicability of the proposed approach.
Weaknesses: -	The work seems to be an implementation of already existing and popular YOLO architecture on custom dataset.
-	The work does not compare the work with other works that have been developed for surveillance.
-	The pipeline-based approach uses multiple runs of the YOLO model to detect objects which adds more time delay to detection missing other important potential events while processing one event.
-	While the paper provides some quantitative data, a more detailed analysis of the results, especially in the pilot testing, would enhance the paper's completeness.
Recommendation: 1: strong reject
Reviewer confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
Soundness/correctness: 2 fair
Clarity: 2 fair
reviews: 
Comments: Object detection on resource-constrained devices using computer vision and convolutional neural networks presents challenges due to their limited power and processing capabilities. The study introduces a multi-step detection pipeline tailored to the object sizes and their inter-correlations. When tested in a smart surveillance system incorporating edge computing and the IoT framework, and under real-world scenarios, this pipeline-based method tripled the processing speed and significantly enhanced the mean average precision of detection.
Strengths: 1. Followed by a very simple writeup, which is better for readability, specially for newer researchers.
2. The implemented hardware prototype the authors propose in Section 4.1. 
3. This work has good real-life implementation.
Weaknesses: 1. The abstract is really small. It does not convey major information. For instance, “multi-step detection pipeline” - it does not have what this may do. 
2. No research questions defined.
3. Too elaborated literature review on individual papers, but not enough number of papers in the lit review. 
4. Too old models used for the AI pipeline. No proper explanation is also present why they choose older models instead of state-of-the-art models.  
5. Even though the authors mention that they worked with four models, they did not mention which one they worked with mainly
6. No sample of data provided in paper.
7. Since the approach is worked on resource-constrained IoT devices, to compare, the authors did not bring any such metric (For instance, FLOPs, Parameters, Energy Consumptions) to compute the computational power needed by the models.  
8. No result comparison is shown among the four models used.
9. Too less number of references used.
10. No limitation is present.
11. No discussion is present.
Recommendation: 3: reject, not good enough
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 1 poor
Clarity: 2 fair