dataesr/scientific-paragraphs-categorization huggingface dataset:
• Performance evaluation: In addition to analyzing the algorithm's complexity, we evaluate the performance of YOLOv4 on self-driving datasets. This evaluation encompasses accuracy assessment, where we quantify the detection precision and recall rates achieved by YOLOv4 on the autonomous driving datasets. • FPGA-based GEMM implementation: To optimize the computational efficiency of YOLOv4, we propose a novel FPGA architecture specifically designed for implementing the GEMM operation within the algorithm. This paper is organized in the following sections. Section 2 introduces the related work. Section 3 presents the autonomous driving datasets and provides an overview of the YOLOv4 algorithm and its architecture. Section 4 outlines the experiment setting and describes the evaluation metrics employed to assess the performance of the algorithm.
Experiments on the KAIST dataset demonstrate that YOLOv4 can achieve high accuracy while retaining low latency of 30 ms and low false negatives, making it suitable for real-time decision-making. The experiments also show that thermal and visible light images can be merged in the modern YOLOv4 architecture without additional computational cost, assuming the two images are aligned and synchronized. Another implementation on an embedded GPU was done by  [9] , which introduces a new object detection model, YOffleNet, based on YOLOv4, but with a much lighter network architecture that uses ShuffleNet modules instead of CSP DenseNet. Experiments with the KITTI dataset showed that YOffleNet is compressed by 4. 7 times, achieving a speed of 46 FPS on an embedded GPU system (NVIDIA Jetson AGX Xavier).
6% of LUTs, 43. 04% of Flip-flops, 82. 17% of BRAMs, 79% of DSPs, 189. 14 GOP/s throughput, and 10. 36 W power consumption. The implementation also achieves a frame rate of 30 FPS with the MS-COCO reference dataset, but does not provide the evaluation accuracy of the algorithm. The Vivado HLS tool is used to synthesize and implement the object detection algorithm on the SoC. A different work for accelerating convolution operations is proposed by  [12]. This work proposes an FPGA-based architecture to accelerate convolution operations to provide AI functionality to edge devices. The architecture is implemented in Verilog and can be used with different FPGA devices. It can compute simultaneously on multiple feature channels and filter cores, increasing computational parallelism. Their experiences show that the architecture can achieve 0.
224 GOPS on a single core when used on a single edge computing FPGA board and 4. 48 GOPS when fully utilized. Likewise, this paper  [13]  explores neural-network-based FPGA acceleration by accelerating general matrix multiplication (GEMM). They propose a design that exploits 8-bit quantization to improve bandwidth while preserving model accuracy, leveraging FPGAs for model parallelization and data reuse for high-performance, low-latency neural network inference. The design is implemented on Xilinx-based Zynq SoCs and tested on the MNIST dataset. The proposed method produces faster acceleration while maintaining nearly identical performance to the unquantized method. Additionally, the paper  [14]  presents a low-memory GEMM algorithm for CNN inference on Xilinx Virtex UltraScale+ VU9P FPGAs.
As a result, the low-power DNN topologies generated for the MNIST database achieved a high throughput of 3626 FPS and power consumption of only 0. 266 W. Much of the research on FPGA-based accelerators focusing on efficiency and costeffectiveness by utilizing lightweight object detection models and small datasets. These approaches aim to simplify development and enable real-time applications. However, our research takes a distinct approach by emphasizing the utilization of the full-weight YOLOv4 object detection algorithm on extensive autonomous driving datasets. This choice enables us to explore the algorithm's suitability for Advanced Driver Assistance Systems applications demanding high detection accuracy. The outcomes of our work hold paramount importance, as they will yield invaluable insights into the suitability of YOLOv4 for Advanced Driver Assistance Systems (ADAS) applications demanding high detection accuracy.
Materials and Methodology In autonomous driving applications, object detection models are typically based on deep-learning neural networks. Object detection is an essential part of autonomous driving, enabling autonomous cars to recognize and react to their environment. One of the most challenging aspects of autonomous driving is detecting and recognizing objects on the road, including vehicles, pedestrians, traffic signs, and traffic lights  [16]. Several object detection models exist in the literature, including YOLO, SSD, and Faster RCNN, among others. However, to determine the most suitable model for autonomous driving applications, it is essential to evaluate the accuracy of these models. This paper presents an evaluation of several state-of-the-art object detection models on the COCO dataset.
This framework contains pre-trained object detection and classification models. It allows users to evaluate model accuracy and inference time using virtual CPUs and GPUs on multiple datasets, such as COCO, a benchmark dataset commonly used for object detection tasks, including a large number of images with a variety of objects in different contexts. The mAP metric computes the average accuracy for each object class and averages the average accuracy of all classes. The third column refer to the number of parameters, which is the sum of all weights and biases of the neural network. For example, YOLOv4 has over 60 million trainable parameters. The results show that the YOLO series models outperform the other models on the COCO dataset, and YOLOv4 is the most accurate, with an mAP of 77. 40%. This signifies that the YOLO models are very effective for object detection on this dataset.
Datasets The KITTI dataset  [20]  is well-known for training and testing autonomous driving algorithms. It includes images, labels, and calibration data acquired from the onboard camera, lidar, and radar sensors of a vehicle driven on German public roads. The collection includes 7481 images of various locations, such as city streets, highways, and rural locations. In addition, the dataset structures road participants in eight different classes (car, van, truck, pedestrians, cycle, tram, and misc. ). Calibration information is required to convert sensor values to real-world coordinates. The dataset is suitable for a wide range of applications, including object detection, semantic segmentation, and 3D reconstruction. The Self Driving Car dataset  [21]  comprises driving in Mountain View, California, and surrounding cities during daylight hours.
It contains 97,942 labels divided into 11 classes, such as biker, car, pedestrian, traffic lights, trafficLight-Green, trafficLight-GreenLeft, trafficLight-Red, trafficLight-RedLeft, trafficLight-Yellow, trafficLight-YellowLeft, truck, and 15,000 images captured by a Point Grey camera at full HD resolution (1920 × 1200 px) at 2 Hz. The KITTI and Self Driving Car datasets are ideally suited to evaluate the performance of YOLOv4 in the autonomous driving domain. These datasets contain images of environments, such as highways and city streets, which is common in autonomous driving applications. In addition, they provide many annotated images, which can be used to train and accurately evaluate the object detection capabilities of YOLOv4. Furthermore, the images in these datasets are specifically designed to simulate real-world driving scenarios as shown in Figures  1  and 2.
Experimental Results on KITTI and Self Driving Car Dataset Yolov4 was thoroughly evaluated on several benchmark datasets and achieved excellent accuracy. On the MS COCO object detection dataset  [31] , Yolov4 achieved a mean average precision of 45. 6%. On the Open Images V4 dataset  [32] , Yolov4 achieved an average accuracy score of 39. 7%. Alternatively, Yolov4 was evaluated on the PASCAL VOC dataset, achieving an mAP of 71. 9%  [33]. In this work, we evaluate Yolov4 on two autonomous car datasets with three classes relevant to autonomous navigation (pedestrian, bicycle, car) under different input resolutions 608 × 608 px, 512 × 512 px, and 320 × 320 px. Table  3  shows the performance of YOLOv4 on the KITTI and Self-Driving Car datasets at three different resolutions. The results reveal that the model performs well overall, with an mAP of up to 89.
4% at the highest resolution, 608 × 608 px. However, the results vary considerably depending on the resolution used. The model achieved a mAP of 74. 18% and 87. 51% at the lowest resolution of 320 × 320 px and the highest resolution of 512 × 512 px, respectively. This result indicates that the model is more efficient when it receives more information, as higher resolution allows for more accurate object detection. Furthermore, the model detected objects in the KITTI dataset with high accuracy, as demonstrated by the mAP score. For evaluating the Self Driving Car dataset, the model achieves an mAP of 58. 19%, 57. 61%, and 47. 15% at a resolution of 608 × 608 px, 512 × 512 px, and 320 × 320 px, respectively. The results suggest that the model does not yield better performance. Several different factors might cause this.
First, as mentioned in Section 3, the resolution of this dataset is 1920 × 1200 px, and the current version of YOLOv4 may not be equipped to handle higher-resolution images. The model may not have sufficient capacity to process higher-resolution images without a significant increase in computational cost. Second, the dataset includes many duplicate bounding boxes for the same object, resulting in poor performance. Furthermore, the performance evaluation for the three classes, as shown in Figure  4 , shows that the model on KITTI with a resolution of 608 × 608 px achieves an average accuracy (AP) of 97. 15% for car detection, 81. 25% for pedestrian detection, and 89. 79% for bicycle detection. Similarly, for the self-driving car (see Figure  5 ), the model achieves an average accuracy (AP) of 77.
85%, 51. 05%, and 45. 66% for car, pedestrian and bicycle detection, respectively. We can notice that the class "Car" is more accurate than the other classes due to the fact that it occupies a large number of labels in both databases. In addition, when reducing the input size from 608 × 608 px to 320 × 320 px, we observed a proportional decrease in the average accuracy for all three classes. In the case of the KITTI dataset, as shown in Figure  4 , the average accuracy for the car class decreased by approximately 7. 44%, resulting in a value of 89. 71%. Additionally, for the pedestrian class, the average accuracy decreased by about 19. 31% to reach 61. 94%, while for the bicycle class, it decreased by about 18. 9% to reach 70. 89%. Similarly, for the Self Driving Car dataset, as shown in Figure  5 , we observed a decrease in the average accuracy for the car class by about 5.
93%, resulting in a value of 71. 92%. The pedestrian class experienced a decrease of approximately 14. 39% to 36. 66%, and the bicycle class decreased by approximately 12. 79% to 32. 87%. These results highlight the impact of scaling the input size on the accuracy performance of the model. The results indicate that the model is able to accurately identify multiple classes simultaneously, making it a viable option for multi-class object detection tasks. Overall, YOLOv4 performed better on the KITTI dataset with the three different resolutions tested than on the Self-Driving Car dataset. Using the highest resolution improved performance, as expected. However, YOLOv4 is not suitable for all autonomous driving datasets.
The utilization of logic resources such as LUTs and registers is relatively low, at about 12% and 12%, respectively. DSP block utilization is about 6%, while memory utilization is relatively high, at about 19%. To evaluate the effectiveness of our FPGA-based approach, we conducted experiments on two popular object detection datasets: KITTI, as shown in Figure  8 , and Self Driving Car, as shown in Figure  9. YOLOv4 convolution layers are implemented on a DE5a-Net DDR4 with a high-performance Intel Arria 10 FPGA. The communication between the FPGA and the host is done via PCIe. The convolution process is performed on the FPGA, and the results are transferred to the host PC to continue the YOLOv4 process, and then Figures  8  and 9  are displayed. Our results indicate that our FPGA architecture is wellsuited for object detection tasks and can efficiently detect and classify objects such as cars, pedestrians, and bicycles.
Overall, our results highlight the potential of FPGA-based implementations to perform effective object detection in real-world applications. Furthermore, the results of our proposed GEMM implementation on FPGA are promising and demonstrate its effectiveness in terms of performance. As shown in Figure  10 , with a resolution of 608 × 608 px, 512 × 512 px, and 320 × 320 px, our GEMM implementation achieved a mAP of 89. 40%, 87. 51%, and 74. 18% on KITTI, respectively-and 58. 19, 57. 61, and 47. 15% on the Self Driving Car dataset. Moreover, the performance evaluation for the three classes, as illustrated in Figure  11 , the model on KITTI with a resolution of 608 × 608 px achieves an average precision of 97. 15% for car detection, 81. 25% for pedestrian detection and 89. 79% for bicycle detection.
66% for pedestrian detection, and 32. 87% for bicycle detection. Our research results demonstrate the effectiveness of our proposed FPGA architecture for implementing the GEMM operation in YOLOv4 without affecting the accuracy. Our approach provides detection results comparable to those obtained with GPU implementations in Section 5 while offering significant performance advantages. These results demonstrate that the proposed GEMM implementation is an efficient way to process data in an FPGA environment and can be successfully used for object detection in autonomous driving applications. Additionally, our proposed implementation of GEMM on FPGA achieved a runtime of 38 ms with a cadence of 100 Hz on the KITTI dataset, which can reduce power consumption and improve the overall performance of embedded systems.
Conclusions This paper investigates the utilization of Field Programmable Gate Arrays for implementing YOLOv4 GEMM for autonomous driving applications. Our study analyzes the performance of YOLOv4 on two prominent autonomous driving datasets, namely KITTI and Self Driving Car. Notably, we have assessed the performance using various input sizes, specifically 608 × 608 px, 512 × 512 px, and 320 × 320 px. Furthermore, we have compared the detection performance achieved by GEMM operations implemented on an FPGA compared to those executed on a GPU. The outcomes derived from our investigation effectively demonstrate the immense potential of FPGAs in implementing YOLOv4 while preserving model accuracy. In future research, there are several avenues to further explore the potential of programmable gate arrays in accelerating YOLOv4.
It would be interesting to further analyze the performance of other versions of YOLO, such as YOLOv6, YOLOv7, and YOLOv8, on other autonomous driving datasets, such as BDD and Cityscapes, to gain a complete understanding of the effectiveness of these models. Alternatively, considering the complexity inherent in YOLOv4 as a neural network architecture, it becomes interesting to investigate the application of FPGA acceleration to other network components. This could involve exploring novel techniques for optimizing convolutional operations on FPGAs, including systolic networks, data preprocessing approaches, and parallelization techniques. Additionally, conducting comparative studies to evaluate the performance of optimized YOLOv4 implementations on FPGAs against CPUs, GPUs, and other FPGAs would provide invaluable information.
Motivation and Problem Statement Data graphs, including RDF knowledge graphs, as well as Property Graphs (PGs), are often used to represent and share data. More generally, any structured or semistructured dataset can be viewed as a graph, having: (i) an internal node for each structure element of the original dataset, e. g. , relational tuple, XML element or attribute, JSON map or array, URI in an RDF graph; (ii) a leaf node for each value in the dataset, e. g. , attribute value in a relational tuple, text node or attribute value in XML, atomic (leaf) value in a JSON document, or literal in RDF. The connections between the data items in the original dataset lead to edges in the graph, e. g. parent-child relationship between XML or JSON nodes, edges connecting each relational tuple node with their attributes, etc.
Entity-rich graphs Building on this idea of integrating heterogeneous data into graphs, the Connec-tionLens system  [4, 6]  has been developed to facilitate, for users lacking IT skills, such as data journalists, the exploration of datasets of various models, including relational/CSV, XML, JSON, RDF, and PGs. ConnectionLens turns any (set of ) datasets into a single graph as outlined above. For instance, the data graph at left in Fig. 1  features RDF triples about NASA spacecrafts (labeled edges), and an XML document describing presidents who attended spacecraft launches (tree with labeled node and unlabeled edges). ConnectionLens also includes Information Extraction modules, which extract, from any leaf node in the data graph, Named Entities (People, Locations and Organizations)  [4] , as well as other types of entities that journalists find interesting: temporal moments (date, time); Web site URIs; email addresses; and hashtags.
We designate any of these pieces of information as entities, and we model them as extra nodes, e. g. , in the data graph in Fig. 1 , organizations appear on a pink background, people on yellow and locations on green, respectively. Each entity is extracted from a leaf text node, to which it is connected by a dashed edge. When an entity is extracted from more than one text node, the edges connecting it to those nodes increase the connectivity of the graph, e. g. , "NASA" extracted from the nodes with IDs 15 and 17.
Entity paths Journalists are interested in the paths connecting entities in a given dataset. For instance, in Fig. 1 , they may want to know "how people are connected to places". Similarly, an article describes French real estate bought by family members of dictators abroad; here, journalists ask "what are the paths between people and cities (where real estate is)?". Importantly, we should consider paths irrespective of the direction of the edges in the data graph. This is because, depending on how the data is modeled, we may encounter  -----→ c ; both paths are interesting. Goal: efficient exploration of entity connections Journalist questions such as those above ask for data paths ending in entity pairs of certain kinds. When shown the set of corresponding labeled paths, users may pick one to further explore: how many pairs of entities are connected by each path? which entities are most frequent, e.
g. , in which cities are most properties located? how do the cities spread across countries, etc. Such analysis requires materializing the entity pairs connected by the paths, which may be very costly, if (i) the graph is large, and/or (ii) there are many paths (the latter is almost always true, especially since our paths may traverse edges in both directions. To mitigate this problem, PathWays includes a materialized view recommender and view-based rewriting module (Sec. 2), which significantly improve performance. Thus, PathWays enumerates paths between entities of user-selected types, (i) independently of the edge direction, (ii) asking for user input to focus on the paths most useful to them, and (iii) efficiently. To our knowledge, PathWays is the first system built for this flavor of graph exploration (see also Sec.
For each collection of nodes having text children, e. g. , C8 labeled name, the collection of these text children is denoted, e. g. , name#; entities (people, locations, etc. ) are extracted from such texts. Any path in the data graph also exists in the collection graph, thus PathWays enumerates paths on the smaller collection graph. Each path is then translated in a query over the data graph, to obtain data paths between actual entities. PathWays displays sample entity pairs connected by each path, e. g. , Nixon is connected to NASA because he attended a launch involving N. Armstrong. Users can then apply more aggregation/analysis on the entity pairs, look for frequent entities, etc. We will also show how PathWays optimizes path evaluation (see below). We will use real-life datasets, such as PubMed, the NASA dataset, RDF benchmarks, and GeoNames, to investigate connections between people and organizations, e.
View materialization and view-based rewriting When the data graph is large, paths are long, and/or many, evaluating path queries on the graph may be inefficient, despite the graph being extensively in-dexed. However, as illustrated at right in Fig. 1 , paths may overlap, e. g. , the edges connecting C1, C6 and C7 are common between the two highlighted paths. Leveraging this observation, PathWays identifies the subpaths common to at least two path queries. Then, with the help of a cost model, based on PostgreSQL's estimations, it materializes the most profitable shared subpath s 1 where profit is: the decrease in the total path evaluation cost if the subpath is materialized and its results used to evaluate the paths enclosing it, minus the cost to materialize the subpath. PathWays then rewrites every path query p 1 1 ,.
, p n1 1 containing s 1 , using it as a materialized view. Then, we remove p 1 1 ,. , p n1 1 from the path set, and, in a greedy fashion, again look for the most profitable common subpath s 2 for the remaining paths, etc. We stop when no subpath is profitable (materializing it costs more than its cost savings). The complexity of the above view selection algorithm is O(N 2 L + N 3 ), for N paths of length at most L. Sample performance saving On the NASA RDF dataset (100. 000 triples), we enumerated 100 paths, of length 2 to 8, between locations and people. Evaluating them all took 419 seconds, including 12 that timed-out at 30 seconds. PathWays found 89 common subpaths; 16 were selected by our algorithm, which rewrote 98 path queries using them as materialized views. Materializing the 16 paths took 0.
Related work and conclusion Many graph exploration methods exist, see, e. g. ,  [11]. Modern graph query languages such as GPML  [7]  (no implementation so far) or the JEDI  [1]  SPARQL extension allow asking for paths between nodes matching some query variables. Other systems interact with users to incrementally build SPARQL queries interesting for them, e. g. ,  [10]  for queries with aggregation. In keyword-based search (KBS, in short)  [2, 4, 6, 8, 12] , one asks, e. g. , for connections between "Sivel Aliev" (related to the Azeirbadjan president) and "Nice" (where she owns villas). KBS is handy when users know keywords (entities) to search for. Complementing the above, PathWays is focused on identifying and computing all paths between certain extracted entities, to give a first global look at the dataset content, for graphs obtained from multiple data models.