Reviewerly/exHarmony huggingface dataset:
Efficient Backbone Architecture Search for Stereo Depth Estimation in Autonomous Driving

Recent advances in AutoML have extended Neural Architecture Search (NAS) beyond image classification to optimize dense prediction tasks. However, the existing works are inappropriate to search efficient backbone for deep learning based stereo matching, because their search spaces are not custom-designed according to the inherent requirements of the pixel-wise depth prediction. This paper proposes a differentiable architecture search specific for efficient stereo network backbone. In particular, the proposed method jointly optimizes the micro-architecture and the macro-architecture to search distinct cell structures and adaptive low-level features for stereo network backbone. The target architecture can be found within 3 GPU days using gradient-based optimization. The evaluation results on stereo datasets demonstrate that, by simply replacing the hand-crafted feature extraction with the searched backbone in a vanilla framework, the proposed network obtains much better disparity accuracy than the designs using existing NAS methods, and even achieves comparable performance compared with the state-of-the-art stereo networks that integrate various elaborate modules. Hence, the proposed NAS method is an efficient way to automate the stereo network architecture engineering.
Laplace Approximation for Real-Time Uncertainty Estimation in Object Detection

Object detection is a fundamental task in autonomous driving. Besides bounding-box-like detection algorithms, uncertainty estimation is necessary for safe and trust-worthy perceptions. Bayesian Neural Networks (BNNs) provide a reliable approach to address the challenge. However, it often becomes computationally prohibitive to apply them to modern large-scale neural networks. This work develops an efficient BNN by combining the Laplace Approximation (LA) with linearized inference. Specifically, we study the effectiveness and computational necessity of a diagonal Hessian approximation in the LA on over-parameterized networks. With numerous quantitative experiments on different types of interference, the proposed method demonstrates the ability for real-time and robust uncertainty description for autonomous driving.
Real-time Adaptive Background Subtraction for Traffic Scenarios at Signalized Intersections Based on Roadside Fish-eye Cameras

Background subtraction (BS) has been a norm for moving object detection along a classical computer vision pipeline, especially when the labelled data is largely unavailable. It has been widely used for infrastructure-based sensing such as traffic surveillance with roadside cameras. Existing BS algorithms focus on detecting moving objects, while the temporal motionless objects are neglected. This leads to performance degradation in particular at signalized intersections where vehicles may stop to wait in red. In this paper, we propose a hierarchical adaptive BS method which can eliminate the cumulative errors for those temporally static objects based on images from roadside fish-eye cameras in real-time. The proposed method is validated in both the CARLA simulation and the real-world environment. The results show that our method outperforms ViBe and LOBSTER by about 45% and 39%, respectively, on recall, without compromising too much in precision.
Dependency-aware Task Offloading via End-Edge-Cloud Cooperation in Heterogeneous Vehicular Networks

Task offloading is critical to enable various computation intensive applications in vehicular networks as well as accelerate the intellectualization of transportation system. In this paper, we first present a task offloading architecture in heterogeneous vehicular networks via end-edge-cloud cooperation. In particular, we investigate the scenario where each task may consist of multiple dependent subtasks, which have to be processed in sequence and in time. Then, a Dependency-Aware Task Offloading Problem (DATOP) is formulated as a Mixed Integer Nonlinear Programming problem (MINLP), aiming at minimizing the average service delay of tasks. Further, we design an offline method as a benchmark, which linearizes the problem into a Mixed Integer Linear Programming problem (MILP) and obtains the optimal solution through Lingo. Moreover, we propose a three-stage heuristic algorithm, which computes the critical path based on the derived task graph and schedules subtasks on the critical path via dynamic programming, and then the subtasks on non-critical paths are scheduled via a greedy strategy. Finally, we build the simulation model and conduct comprehensive performance evaluation, which demonstrates the effectiveness of the proposed online algorithm.
Design and implementation of obstacle detection system for powered wheelchairs

In this paper a detailed description of the design and implementation of a system capable of detecting various obstacles for wheelchairs is provided. This system is part of a larger project that its goal is to create intelligent applications for patients using powered wheelchairs. The system is implemented on an electric wheelchair using low-cost ultrasonic and infrared sensors. All tests took place in indoor environment inside our laboratories, and the performance of the system is found to be adequate for the purposes of our project.
Methodology to Adapt Neural Network on Constrained Device at Topology level

Artificial Intelligence is now ubiquitous as nearly every application domain has found some use for it. The high computational complexity involved in its deployment has led to strong research activity in optimizing its integration in embedded systems. Research works on efficient implementations of CNNs on resource-constrained devices (eg. CPU, FPGA) largely focus on hardware based optimizations such as pruning, quantization or hardware accelerator. However, most performance improvements leading to efficient solutions in terms of memory, complexity and energy are located at the NN topology level, prior to any implementation step. This paper introduces a methodology called ANN2T (Artificial Neural Network to Target) which adapts a pre-trained deep neural network to a designated device with given optimization constraints. ANN2T leverages its included simplifications and/or transformations to progressively modify the deep neural network layers in order to meet the optimization target. Experiment results obtained on microcontroller device show ANN2T produces valuable trade-offs. It achieved up to 33% MACC and 37% memory footprint reductions with no accuracy loss on ResNet-18 topology over the CIFAR-10 dataset. This fully-automated methodology could be generalized to targets such as CPUs, GPUs or FPGAs.
Analysis and Design of Distributed Data Cluster System

In order to solve the problems of dynamic construction of distributed data cluster and large amount of data synchronization data under the scenario of data consistency and low delay of data synchronization, this question has improved the existing distributed cluster technical scheme and proposed a dynamic construction of distributed data cluster control system. This system can provide dynamically configurable data node adding and removing service devices The server device provides data administrationservices to the data nodes, improves the processing efficiency and level of distributed data, ensures the data consistency between the data nodes, and provides low delay services for data reading and writing.
A Rigorous Optimization Approach for the Generic Derivation of Secondary Information from Digital Maps for Urban Scenarios

In the context of Highly Automated Driving (HAD) and Advanced Driver Assistance Systems (ADAS), we derive optimal stopping points for vehicles in urban scenarios. We develop a rigorous global optimization algorithm and apply it to a constrained maximization problem that is based on a high-definition map (HD map). Specific map feature functions describe the contribution of the points, curves, and polygons to the urban scenario model, while the perceived obstacles form infeasible areas. Our algorithm combines a branch-and-bound method using interval arithmetic with locally applied cubic Taylor expansions of the objective function. The polynomial maximization problem with box constraints is formulated as a root finding problem that we solve using an eigenvalue approach. We test the developed algorithm in urban scenarios and compare it to interval algorithms that either use local quadratic Taylor approaches or no local improvement at all.
Trajectory-planning using set-based motion-primitives considering model uncertainty and controller tracking errors

In this paper a trajectory planning approach based on set-based motion primitives is presented. Through the set-based motion primitives model uncertainty and the control error of a trajectory controller, due to imperfect tracking is considered. Due to the linearization approach of the vehicle dynamics, two main improvements compared to the state of the art are achieved. First, the resulting reachset of the controlled vehicle dynamics is linear and therefore more efficient to calculate compared to a nonlinear method. Secondly, the sequential composition condition, which has to be fulfilled when switching between two set-based primitives, can be satisfied efficiently online. This leads to more switching possibilities between the motion primitives during planning, and hence improves robustness of the collision avoidance behaviour in a dynamically changing environment. The trajectory planning approach is successfully demonstrated within an intersection scenario with a dynamic obstacle in a simulation framework.
Real-time Attention-Augmented Spatio-Temporal Networks for Video-based Driver Activity Recognition

Identifying driver behaviour and activities from in-cabin video cameras (especially the distracting non-driving activities), has been recently shown to be effective in enhancing the safety and the driving experience in smart and partially-automated vehicles. In the literature, the problem of video-based driver activity recognition is often tackled by using traditional deep learning-based human-action recognition systems. Despite their powerful capabilities, they seem not well-suited for video-based driver activity recognition, due to their complex and inefficient architecture that requires a huge amount of computational resources. Additionally, given the similarities of different non-driving activities that share the same pattern of upper body movements (e.g. drinking versus eating), it makes it harder for traditional human-action recognition systems to pick up or differentiate between these subtle changes. Thus, in this work we are proposing a novel framework based on an efficient spatio-temporal neural network architecture augmented with an attention mechanism that can differentiate between the subtle differences of similar non-driving activities. Our framework has been evaluated on one of the largest benchmark datasets for fine-grained recognition of driver activities and it has outperformed the state-of-art approach by more than 4% in the top-1 accuracy score with a boosting of 13x the run-time speedup during the inference.
Enhancing Flagman Gesture Recognition with Virtual Augmentation

A major challenge for autonomous vehicles is when an AV encounters a flagman who regulates traffic near a construction workzone area or at an automotive crash site. In such real-world scenarios, recognizing the flagman's gesture is an essential AV function that has not received much attention. A key piece of earlier work used a chroma-key (green background) screen to generate datasets and evaluate the accuracy of flagman gesture recognition. We find that this background-agnostic approach drops significantly in performance when the chroma-key background is replaced with images from real-world traffic scenarios. In this paper, we extend that baseline approach by adding contextual information and boosting the baseline's accuracy and robustness. First, we replace the chroma-key background to create virtually augmented (VA) images to capture real-world contexts. Next, we propose and evaluate three approaches: (1) Extract CNN features directly from the “raw” VA images, (2) Extracting CNN features after embedding flagman skeleton and prop information in the VA images, and (3) Utilize an attention mechanism via a transformer with padding masks. All three approaches outperform the baseline method. Notably, our Two-Stage Classifier with Transformer approach boosts the baseline's performance from an F-score of 32% to 80% in challenging VA traffic scenarios.
Automated driving system based on multi-source sensing method for unmanned detection vehicle in tunnel

With the development of Connected and Automated Vehicles (CAVs), related technologies are applied in many specific traffic scenarios. Especially in tunnels of highway, how to avoid traffic accidents attracts the attention of researchers. Therefore, this paper designs a novel multi-source sensing method for unmanned detection vehicle. Firstly, the Hector SLAM matching algorithm for multi-source sensing system is established by the fused data of camera, laser point cloud and inertial measurement unit (IMU), which can map the tunnel scenario and locate the real-time position. Secondly, a route planning algorithm in tunnels based on the combination of A* algorithm and dynamic window approach (DWA) is proposed. Thirdly, the detection method of cracks and fire in the tunnel is realized based on the improved faster region-based convolutional neural network (Faster R-CNN). Finally, through the field test in tunnel scenarios, experiment results have shown that the tracking and positioning error detected by the proposed method are no more than 0.2 meters. Meantime, the detection accuracy rate is 94.53%, and the recall rate is 86.33%, which can satisfy the detection requirement of driving safety in tunnels. It has provided an effective and feasible detection technology in tunnels based on autonomous vehicles.
Token Economy in the Mobility Ecosystem

Information technology is seen as the key to provide solutions for current and future challenges in the transportation and mobility ecosystem. Of those, the Distributed Ledger Technology (DLT) is considered as an emerging paradigm to establish a token economy ecosystem without trusted third parties. Hereby, the mobility and transportation infrastructure resources and their usage are regarded as assets, which are transformed into tradabe tokens, managed via distributed ledgers. The access to the infrastructure is restricted and guaranteed to the token owners. The design and implementation of distributed ledgers depends on the requirements of the respective use cases, leading to a DLT-based IT infrastructure consisting of many different distributed ledger implementations. We derive a generic architecture for the future mobility token economy by adapting and applying a DLT design decision process to the mobility ecosystem, including six potential use cases.
Road Traffic Image Stitching with Geometric Feature Protection

An ordinary traffic surveillance camera has difficulty capturing all the information due to its limited field of view. This paper proposes a method for stitching traffic images with geometric feature protection (GFP) to expand the view and keep the panorama as natural as possible. The core of this method is to build up a two-stage mesh optimization model integrating shape preservation, line protection, and boundary protection. In terms of boundary protection, a boundary regularization term is added to protect the boundaries of the initial stitched images to further achieve a regular display effect. The experiments show that the proposed method can handle traffic images well. It further enhances the alignment of lines and regularizes image boundaries whereas maintaining a high degree of similarity, making panoramas more natural and visually acceptable than several representative methods.
Guiding Belief Space Planning with Learned Models for Interactive Merging

Safely navigating complex, interactive situations is one of the major challenges in planning for automated vehicles. The key difficulty is that drivers have unobservable properties determining their driving style. To make optimal decisions, the belief over latent driver states has to be considered, which is computationally challenging. For this reason, online planning algorithms suffer from the curse of dimensionality in highly interactive scenarios. On the other hand, learned policies often have difficulties generalizing to new environments with different driving styles. We propose to use policies trained in belief space as heuristics to guide online belief space planning algorithms, thereby alleviating the curse of dimensionality. We evaluate the proposed approach in a cooperative merging scenario.
Identifying and Deactivating Unusable Modalities to Improve Multimodal CNN Robustness for Road Scene Understanding

Multimodal CNNs have become one of the most reliable solutions for many computer vision tasks in recent years. However, in the field of ADAS, the robustness of the latter is essential, in order to guarantee a reliability in road traffic actors detection in case of sensor malfunction or breakdown. A noisy input modality can not only bring no useful information, but also disturb the good functioning of the CNN. Nonetheless, when a modality is unusable, it is preferable to ignore it, which a multimodal CNN can hardly do without a strategy designed for it. In this paper, we propose Noise Augmentation, a data augmentation technique that produce unusable modalities during CNN training, and Modality Activator Model (ModAM), a CNN that preprocess input modalities, identify and deactivate those unusable. The experiments on KITTI object detection benchmark under several degraded conditions show that the combination of our 2 contributions makes multimodal CNNs significantly more robust, whatever the type of noise applied and on any input modality, without lowering their overall performances when all sensors are well functioning.
Detection of distant eye-contact using spatio-temporal pedestrian skeletons

Eye contact is an important factor in determining whether a pedestrian is aware of a vehicle. Most conventional eye contact detection methods rely on direct eye-gaze estimation based on eye measurements. This approach is difficult with distant pedestrians, especially in road environments. In contrast, the presence or absence of eye contact from a pedestrian can be determined based on information on their whole body and movement, such as their facial orientation, posture, and turn. In this study, we propose a method to detect eye contact from even distant pedestrians whose direction of gaze would be difficult to measure. The method captures the relationship between face and body information and its changes over time using spatio-temporal skeleton graph convolution. In an experiment using in-vehicle camera images, an accuracy of 88.6% was achieved. The results demonstrate the effectiveness of the proposed approach.
Parameter Extraction of Road Layout Based on Domain Adaptation

Scene layout estimation is crucial for decision planning of intelligent vehicles and scene library enriching of the intelligent driving test. However, current methods are usually limited to pixel-level representations that do not naturally support inference and planning required for intelligent vehicles and cannot directly provide structured samples for the scene library. To overcome the above issue, we propose a novel parameter extraction framework to estimate the parametric layout of various road scenes, which contains three stages: semantic segmentation stage, occlusion reasoning stage and parameter prediction stage, responsible for compressing redundant information of RGB images, restoring the occluded area and generating parametric layout of the road, respectively. Specifically, a residual-based occlusion reasoning method is adopted, which adds pre-acquired residuals to the original reasoning results for preserving small-size objects as much as possible. And a mixture of domains is introduced in the domain-adaptation-based parameter prediction, which avoids discriminator only guided by hard label, leveraging the inter-domain information. Experiments results on KITTI demonstrate that every module in our parametric layout estimation model is necessary, and the proposed method is practicable to provide more structured scene information to the vehicles and apply to enriching scene library.
How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies

Autonomous driving has the potential to revolutionize mobility and is hence an active area of research. In practice, the behavior of autonomous vehicles must be acceptable, i.e., efficient, safe, and interpretable. While vanilla reinforcement learning (RL) finds performant behavioral strategies, they are often unsafe and uninterpretable. Safety is introduced through Safe RL approaches, but they still mostly remain un-interpretable as the learned behavior is jointly optimized for safety and performance without modeling them separately. Interpretable machine learning is rarely applied to RL. This work proposes SafeDQN, which allows making the behavior of autonomous vehicles safe and interpretable while still being efficient. SafeDQN offers an understandable, semantic trade-off between the expected risk and the utility of actions while being algorithmically transparent. We show that SafeDQN finds interpretable and safe driving policies for various scenarios and demonstrate how state-of-the-art saliency techniques can help assess risk and utility.
Investigations of Different Classes Hybrid Deep Learning Networks and Analysis of Their Efficiency in Forecasting

Investigations of hybrid GMDH neuro-fuzzy networks and neo-fuzzy networks in the problem of forecasting in the financial sphere were carried out. The problem of forecasting NASDAQ stock prices with an interval of 1 day, 1 week and month was considered In the process of forecasting the hyperparameters of hybrid fuzzy neural networks were optimized, namely the number of inputs, the number of rules and the size of the sliding window. Optimal structures of hybrid fuzzy neural networks for different prediction intervals were synthesized. Experimental studies of the efficiency of forecasting hybrid neo-fuzzy and neuro-fuzzy networks were carried out and compared with conventional DL networks according to MAPE criterion. Training time of hybrid DL networks was estimated and compared.
VR-based implementation of interactive laboratory experiments in optics and photonics education

Within the framework of a developed blended learning concept, a lot of experience has already been gained with a mixture of theoretical lectures and hands-on activities, combined with the advantages of modern digital media. Here, visualizations using videos, animations and augmented reality have proven to be effective tools to convey learning content in a sustainable way. In the next step, ideas and concepts were developed to implement hands-on laboratory experiments in a virtual environment. The main focus is on the realization of virtual experiments and environments that give the students a deep insight into selected subfields of optics and photonics.
Research of Periodic Properties of the Generator Based on m-ary Codes

The development of high technologies put forward new requirements for the stability of the cryptographic means used in the post-quantum cryptoperiod (the emergence of a full-scale quantum computer). One of these mechanisms are pseudo-random number generators, which are used in symmetric and asymmetric cryptosystems and authentication mechanisms, while providing both additional and session cryptographic strength. The paper investigates the periodic properties of pseudo-random number generators on m-ary codes, the stability of which is based on the complexity-theoretic problem of syndromic decoding of a random code. An improved method for generating pseudo-random sequences is proposed, which allows generating sequences with a guaranteed period.