title: Development of a Realistic Radar Ray Tracing Simulator and Its Application to Deep Learning Based Radar Imaging
title: Entwicklung eines realistischen Radar-Ray-Tracing-Simulators und seine Anwendung auf Deep Learning basierte Radarbildgebung
Creators: name: Schüßler, Christian
Publisher: name: Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)
Year: 2024
Subjects: subject: Radar Simulation, Ray Tracing, Deep Learning Based Radar Imaging

description: This thesis is part of a cumulative dissertation with the following articles: [1] C. Schüßler, M. Hoffmann, J. Bräunig, I. Ullmann, R. Ebelt and M. Vossiek, "A Realistic Radar Ray Tracing Simulator for Large MIMO-Arrays in Automotive Environments," in IEEE Journal of Microwaves, vol. 1, no. 4, pp. 962-974, Oct. 2021, doi: 10.1109/JMW.2021.3104722. [2] C. Schüβler et al., "Achieving Efficient and Realistic Full-Radar Simulations and Automatic Data Annotation by Exploiting Ray Meta Data from a Radar Ray Tracing Simulator," 2023 IEEE Radar Conference (RadarConf23), San Antonio, TX, USA, 2023, pp. 1-6, doi: 10.1109/RadarConf2351548.2023.10149641. [3] C. Schüßler, M. Hoffmann, I. Ullmann, R. Ebelt and M. Vossiek, "Deep Learning Based Image Enhancement for Automotive Radar Trained With an Advanced Virtual Sensor," in IEEE Access, vol. 10, pp. 40419-40431, 2022, doi: 10.1109/ACCESS.2022.3166227. [4] C. Schuessler, M. Hoffmann and M. Vossiek, "Super-Resolution Radar Imaging With Sparse Arrays Using a Deep Neural Network Trained With Enhanced Virtual Data," in IEEE Journal of Microwaves, vol. 3, no. 3, pp. 980-993, July 2023, doi: 10.1109/JMW.2023.3285610. Since I used some figures of my own work published under a non Open Access License from IEEE (https://magazines.ieeeauthorcenter.ieee.org/choose-a-publishing-agreement/avoid-infringement-upon-ieee-copyright/): In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Friedrich-Alexander-Unversität Erlangen-Nürnberg’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink. If applicable, University Microfilms and/or ProQuest Library, or the Archives of Canada may supply single copies of the dissertation.
description: Due to the rise of deep learning, neural networks (NN) have been extensively applied to radar signal processing for object detection, classification, and road segmentation in the automotive domain. Since the angular resolution of typical automotive radar units is limited, the scientific community have made large efforts to enhance the resolution of radar sensors even above the Nyquist sampling theorem. Recently, NNs have been successfully applied for resolution enhance- ment and starting to succeed over classical methods, such as MUSIC, ESPRIT or Compressed Sensing (CS). The majority of works, which utilize NNs for direction-of-arrival (DoA) estimation only work with simulated point-like scatterers to generate the channel values at a single range- Doppler detection, which is clearly not realistic enough to resemble complex urban scenarios in which clutter and multi-path effects can degrade the beamforming process considerably. Also, by only considering single range-Doppler detections and not the complete radar data, a significant amount of information is not available to the NN. Moreover, for detection and segmentation tasks, compared to camera images or lidar point-clouds, radar signals are hard to interpret. This is even true for radar point-clouds. Since they are com- monly much sparser compared to their lidar counterpart, labeling objects manually is a challenging task. The data interpretation is even harder for radar raw data, such as range-Doppler data or even complete 3D range-Doppler-angle data cubes. In this case, radar data is even more unfamiliar to the human eye, especially compared to camera images. Therefore, manual annotation of radar data is even challenging for experts in the radar field and almost impossible for unexperienced users. This work addresses the use of realistic radar simulations and its application to deep learning tasks in the field of radar signal processing, especially in the field of radar image enhancement and super- resolution radar-imaging for sparse antenna arrays in the automotive domain. In the first part, the development of a sufficient physically accurate simulation model is proposed. The implemented simulator is distinctive to existing work in respect of being able to simulate large amounts of data on 3D mesh descriptions, which can be found abundantly in computer games or other 3D automotive simulation programs. There are no constraints on the triangle meshes being loaded, so that it can deal with almost any 3D model. Moreover, it can simulate complete radar data, meaning fast-time, slow-time and even huge numbers of antennas very efficiently by implementing novel optimizations steps. Compared to most other works, it does not aim to simulate the RCS of (often metallic) objects most accurately. Instead, it mainly focuses on effects such multi-path, clutter and explicitly modelling also rough surfaces by heuristic scattering models, so that a large amount of triangles to model uneven surfaces is not necessary, and also typically not available in existing 3D models. In addition, it implements a novel signal decomposition method, which allows data annotation on the raw signal level. This makes fully automatic high-quality annotation of raw radar data possible, which would be impossible to achieve manually. For example, pedestrians can be automatically separated from cars, or multi-path clutter from direct reflections. The second part of this work describes the implementation of deep-learning based image enhance- ment algorithms based on realistic simulation data generated by the proposed simulation approach. The main idea is to simulate a realistic digital twin of the real radar unit and simulate another enhanced virtual sensor, which is equipped with a higher number of antennas and is simulated without multi-path and noise. The NN can then learn to transfer the image of the virtual realistic input sensor to the simulated enhanced one. This approach leads to a significant reduction of noise and clutter. Even though the NN is solely trained on simulated data, it is still able to enhance radar images of real measurements almost as good as for simulated data. In the next part of this thesis, the image features were extended and the approach was applied on sparse antenna array configurations. In this case, the NN could not only generate ambiguity- free and to some extent super-resolution radar images, outperforming even conventional DoA algorithms, but also existing NN approaches trained on point-like targets. The novelty apart from using realistic training data, is that complete range-angle images are used and not only the antenna channel data of single range-Doppler detections.