# Semi-Automatic Cloud-Native Video Annotation for Autonomous Driving
## 1 Introduction  
Automated Driving (AD) and Advanced Driver Assistance Systems (ADAS) are migrating the automotive sector from purely mechanical and electrical engineering to a broader realm of disciplines, including computer science, information technology (IT), and artificial intelligence (AI) [1, 2]. Generally speaking, two main aspects define the concept of AD: (i) Computerized driving, that makes reference to the control of the vehicle's functions (acceleration, steering, etc.) and (ii) navigation, which refers to the motion planning and to the knowledge of the car's environment. These two aspects lean on ADAS as gas and breaking control, lane departure warning, cruise control, or traction control among others. Likewise, ADAS use different data sources retrieved from sensors to describe the current state of the surroundings and the vehicle itself as Light Detection and Ranging (LIDAR) [3], radar, or cameras [4, 5], as well as networking (from a car to other vehicles or infrastructures as road IoT (Internet of Things) devices, public, or private clouds etc.). In order to manage the large amount of data from these sensors and the complexity of real road scenarios, ADAS have to rely on high quality machine learning analytics [6] (during this paper the term machine learning will generically refer to both machine learning and deep learning techniques). The performance of ADAS based on video analysis machine learning modules is inherently restricted by the quality and quantity of training and validation data available. A key bottleneck in achieving improvements in any application domain is the availability of ADAS datasets in real driving situations, with sufficient reliability (ground-truth), complexity, and coverage. Advanced approaches consist onHardware-in-the-Loop (HWiL) simulation, making use of simulated data to feed the ADAS component under test, in order to early identify and fix errors [7]. However, all the approaches must integrate human interactions (Human-in-the-Loop, HiL) to create annotations that can be considered as ground truth. Video annotation is an intense task due to the amount of information they contain. In addition to the large amount of information contained in a single image, in videos we must consider the temporal component, changes in the perspective of the scene, and the relationships of objects between frames. Because of this, each hour of collected video data requires more than 500 h of human intervention to annotate the images separately [8]. There is a wide variety of annotation tools that boost HiL, with different characteristics depending on how they operate and the functionality they provide [9; 10]. Some tools are designed with an orientation towards individual annotation [11; 12; 13], while others orient towards group annotation or even crowdsourcing [14]. There are tools that, as manual annotation is performed, complement them with automatic annotations (interactive annotation) [14; 15]. Among the most common annotations, available in almost all tools, for example is the ability to easily draw a rectangle or an ellipse. Some tools allow one to annotate a polygon when it is convenient (e.g., 3D objects) [16], keypoints (e.g., human pose [17]), temporal events [18], or the association of objects between frames of the video [12; 17]. Semi-automatic approaches exist to optimize the annotation process by reducing human interactions e.g, with recognition of human activity [19], detection of objects and segmentation throughout a video [20; 21; 22], as well as methods based on single frame detection such as [23; 24; 25]. Semi-automatic means they still need HiL interacting in some part of the annotation process. Despite the extensive diversity of the annotation tools described, there is still a need for new custom semi-automatic annotation algorithms and strategies to efficiently address video annotation that meets the computational complexity, data scale, and security requirements inherent in the automotive industry. In this scenario, our contributions can be summarized as follows:  
1.