Patent Description:
Document <CIT> discloses a method for image reconstruction by solving a linear inverse problem using artificial intelligence modules agnostic to image acquisition settings.

Document <CIT> discloses a method for radar imaging by fusing measurements of various antennas having unknown position perturbations.

Document <CIT> discloses methods for using deep convolutional networks for signal recovery using compressive sensing hardware.

Document <NPL> provides a review on solving inverse problems in imaging.

Document<NPL>describes a deep learning framework for inverse problems in imaging.

High resolution radar imaging is used in a variety of remote sensing applications including synthetic aperture radar (SAR) and through-the-wall radar imaging (TWI). A down-range resolution of a radar is controlled by a bandwidth of a transmitted pulse, and a cross-range (azimuth) resolution depends on an aperture of a radar array. Generating a large physical aperture is practically achieved by deploying a number of distributed antennas or arrays, each having a relatively small aperture. The distributed setup of antennas allows for flexibility of radar platform placement, reduces operational and maintenance costs, and adds robustness to sensor failures. Leveraging prior knowledge of a scene, such as sparsity, precise knowledge of antenna positions and a full synchronization of received signals has been shown to significantly improve radar imaging resolution.

A major challenge in the radar imaging using the distributed setup is being able to identify the positions of antennas due to inaccurate calibration or various position perturbations. Although modern navigation systems such as Global Positioning System (GPS) can measure positions, possible position errors due to the position perturbations are beyond scope of high-resolution distributed radar imaging. For example, for a vehicle mounted radar system, as the vehicle is moving along a predesigned trajectory, the position perturbations are introduced due to non-smooth road surface or varying driving velocity and direction. These position perturbations can be as large as several wavelengths of a radar center frequency. Consequently, applying standard reconstruction techniques without accounting for the position perturbations produces out-of-focus radar images.

There are multitude of solutions that addressed radar autofocus problem, particularly in the SAR setting, by developing tools that compensate for the antenna position errors. However, this problem is ill-posed and solving this problem is a computationally demanding process with a difficult to find solution. To that end, some methods impose additional constraints on the radar autofocus problem to make this problem tractable. However, those additional constraints are not always desirable.

Therefore, there is a need for radar imaging systems and methods for autofocusing of antennas having unknown position perturbations.

It is an object of some embodiments to provide a radar system and a method for generating a radar image of a scene by fusing measurements of various antennas having unknown position perturbations. Specifically, it is an object of some embodiments to formulate a neural network denoiser based radar autofocus problem for producing the radar image of the scene.

The above problems are solved by the subject-matter according to the independent claims. The embodiments are based on an understanding that a fundamental challenge that arises in distributed array imaging (i.e., distributed antennas setup) comes from uncertainties caused by one or a combination of position ambiguities and clock ambiguities of each antenna of the set of antennas. Some embodiments of are based on the recognition that radar autofocus problems of the distributed antennas with the position ambiguities can be an ill-posed problem with a vast number of unknowns. Specifically, when the radar autofocus problem is formulated as recovering a correct radar image from incorrect measurements caused by an incorrect radar operator encoding the position ambiguities, each measurement of a region of interest (ROI) includes an error caused by the position ambiguities. Further, due to non-linearity of relationships between the measurements and the errors in the positions of the antennas, each sample of the measurements from the same antenna can have a different error, thereby increasing a number of unknowns in a model of the radar autofocus problem.

For example, a radar image is to be recovered by processing F-dimensional frequency-domain measurements <MAT> from M distributed antennas with the position ambiguities. The position ambiguity can be modeled as a time-domain convolution with the measurements, or equivalently, as a gain and phase ambiguity in a frequency-domain of the measurement, that is, <MAT> where Dĝm is a diagonal matrix with a phase correction vector ĝm ∈ CF on its diagonal entries and where gm is also of dimension F, Am is a radar operator defined by an assumed position of the antennas, and x is an unknown radar image of size sqrt(N) by sqrt(N) resulting in a total of N unknown values for the radar image x. N is generally much larger than F. Every new measurement ym adds F equations but also results in F + N unknowns. Therefore, for M measurements, a resulting system of equations includes MF equations with MF + N unknowns. This is problematic since a number of unknowns will always be larger than a number of equations irrespective of a number of the measurements.

Some embodiments are based on realization that the radar autofocus problem can be reformulated as recovering an incorrect radar image from correct measurements and a correct radar operator. On one hand, such a reformulation does not make sense. However, some embodiments are based on the realization that the incorrect radar image determined via such a formulation of the radar autofocus problem can relate to the correct radar image through a linear shift. Due to this linearity, each sample of the measurements from the same antenna represents the correct radar image with the same linear shift.

To that end, the position ambiguity can be modeled as a spatial shift kernel in a domain of the radar image x. Specifically, a new measurement model is given by <MAT> where * is a spatial convolution operator and hm is a sqrt(P) by sqrt(P) shift kernel that captures the position ambiguity. Here, P is much smaller than N and is also smaller than F. As a result, collecting M measurements results in a system of equations with MF equations and MP + N unknowns. Therefore, there exists a suitable number of measurements M, such that MF is larger than MP + N. Specifically, when M > N/(F-P), then the radar autofocus problem may be solved.

Therefore, the measurements of each antenna are modelled as a product of the radar operator, the radar image, and the shift kernel.

Some embodiments are based on further realization that a regularizer can be utilized for both the radar image x and the shift kernel hm to reduce the necessary number of measurements M. One example of a regularizer for the radar image x is a fused Lasso regularizer. According to an embodiment, true objects in the radar image x have a radar signature that is sparse, i.e. most of the radar image x is zero-valued, and locations of nonzero entries tend to cluster together. The fused Lasso regularizer assigns a small cost to radar images that are sparse and when the locations of the nonzero entries cluster together. As a result, for a radar image x with K nonzero entries in its spatial gradient domain, a true number of unknowns of the radar image x can be represented by K log(N) unknowns. Similarly, the shift kernels hm are one-sparse and have a true number of unknowns represented by log(P). Therefore, a resulting system of equations includes MF equations with M log(P) + K log(N) unknowns when the fused Lasso regularizer is used. Thus, it is sufficient for M to be larger than log(N)/(F - log(P)) to be able to solve the radar autofocus problem.

As an example, consider a radar autofocus problem where F = <NUM>, N = 100x100 = <NUM>, P = 10x10 = <NUM>, and K = <NUM>. For M = <NUM> measurement vectors ym, the measurement model (<NUM>) results in a system with <NUM> equations with approximately <NUM> + <NUM> times log(<NUM>) = <NUM> unknowns. On the other hand, the new measurement model (<NUM>) results in a system with <NUM> equations and approximately <NUM> times log(<NUM>) + <NUM> times log(<NUM>) = <NUM> unknowns. This reduction in the number of the unknowns of the radar autofocus problem allows to solve the radar autofocus problem in an efficient manner.

Some embodiments are based on recognition that the image shifts (i.e., the shift kernels) and the radar image can be determined using an alternating optimization. For example, the radar image is updated while fixing the image shifts and the image shifts are updated while fixing the radar image.

In the optimization according to the invention, at first, an estimate of the radar image is produced. Next, a set of image shifts corresponding to different uncertainties of the antennas are estimated. Then, the estimate of the radar image is updated based on the determined set of image shifts, such that, for each of the antennas, the estimate of the radar image shifted by the corresponding image shift fits the measurements of the antenna. The step of estimating the set of image shifts and the following step of updating the estimate of the radar image are conducted iteratively until a terminal condition is met. When the terminal condition is met, a focused radar image is outputted.

According to an embodiment, the estimate of the radar image is produced by minimizing a difference between radar reflections and modelled measurements synthesized based on transmitted pulses and erroneous antenna positions Not in accordance with the invention, a regularizer is applied on the estimated radar image to filter noise from the estimated radar image. In an embodiment, the regularizer may be the fused Lasso regularizer that includes a one norm regularizer and a total variation (TV) regularizer. The one norm regularizer imposes sparsity on the estimated radar image, while the TV regularizer reduces noise in the estimated radar image to produce a filtered radar image. The steps of minimizing the difference between the radar reflections and the modelled measurements and applying the regularizer are conducted iteratively until convergence. Upon convergence, the estimate of the radar image is outputted.

The embodiments are based on observation that the regularizer (e.g., the fused Lasso regularizer) used in the alternating optimization to filter the estimated radar image may not be suitable when the noise in the estimated radar image is large because the noise may introduce image-domain artifacts that also tend to cluster together and therefore resemble a shape of true targets. As a result, the alternating optimization may focus on a false object and consequently produce incorrect antenna position correction.

The invention is based on realization that, to mitigate such a problem, a neural network denoiser can be used instead of the regularizer. The neural network denoiser includes a neural network trained to denoise the estimated radar image. In other words, the neural network denoiser filters the noise from the estimated radar image. The neural network denoiser is advantageous over the regularizer because of high modelling power of the neural network and its ability to represent signal structure that surpasses an explicit sparsity in image domain and its gradient.

To that end, for estimating the radar image, instead of applying the regularizer on the estimated radar image, the neural network denoiser is applied.

The embodiments are on recognition that artifacts/noises in the estimated radar image are due to iterative update steps that occur during the minimization of the difference between the radar reflections and the modelled measurements. The artifacts/noises due to the iterative update steps are unknown and unpredictable and may not be same as noise that the neural network is trained on. Therefore, the estimated radar image may include noises that the neural network denoiser is not trained on. When such radar images are applied to the neural network denoiser, the neural network denoiser may inject errors in the output radar image. Therefore, merely replacing the regularizer with the neural network denoiser is not sufficient and may pose problems.

The embodiments are based on realization that to manage the noises that the neural network denoiser is not trained on, the estimated radar image is filtered, prior to applying the estimated radar image to the neural network denoiser, by applying a forward radar operator and an adjoint radar operator on the estimated radar image. The forward and the adjoint operator filter the noise from the estimated radar image that the neural network is not trained on, to produce a filtered radar image. Further, the filtered radar image is applied to the neural network denoiser. As a result, the neural network denoiser does not inject errors in the output radar image and reconstruction of the radar image remains stable.

According to an embodiment, the neural network denoiser that denoises a filtering of the estimate of the radar image, includes a residual Unet architecture. The residual Unet architecture is symmetric and includes two parts, namely, left part and right part. The left part is called contracting path, which is constituted by a general convolutional process. The right part is expansive path, which is constituted by transposed 2d convolutional layers. Additionally or alternatively, in some embodiments, the neural network denoiser includes a denoising convolutional neural network architecture.

The embodiments are based on realization that since the forward and the adjoint radar operators filter the noise from the estimated radar image, that the neural network denoiser is not trained on, the neural network denoiser need not be trained with a large training dataset including all possible noises that may be observed in the alternative optimization. Thus, the neural network denoiser can be trained with a simpler training dataset. In embodiments, the simpler training dataset includes input-output pairs (zm, xm), where training input images zm are back projected images formed by applying the back projection operator AmH to example radar measurements ym = Amxm + n, such that, zm =AmHym; and the output images xm are ground truth object radar images, and where n represents added noise. As the neural network denoiser can be trained with the simple training dataset, the training of the neural network denoiser becomes easier and faster.

Additionally, some embodiments are based on recognition that the neural network denoiser based radar autofocus problem of the present disclosure is capable of reconstructing the objects with fewer measurements. This feature allows the embodiments of the present disclosure to be deployed in an object tracking system where radar characteristics of the objects are determined at a rate that is faster than motion of the objects. Such a process allows to continuously detect the objects and perform object tracking-by-detection.

Moreover, detections of moving objects received from the radar reflections may sometimes provide a partial view of the object when the object has a large extension relative to a beam width of the radar. In such scenarios, the neural network denoiser can be trained to reconstruct a partial shape of the object that is intended to be tracked instead of reconstructing point objects of strong reflectors of the object. The reconstructed partial shapes can then be used to facilitate additional modules that determine a current state of the object, where the state of the object includes information on size, orientation, and direction of the moving object.

Accordingly, one embodiment discloses a system configured for generating a radar image of a scene. The system comprises a set of antennas, at least one processor; and a memory having instructions stored thereon that, when executed by the at least one processor, cause the system to: receive radar measurements of a scene collected from the set of antennas, wherein the radar measurements are measurements of reflections associated with a radar pulse transmitted to the scene; generate the radar image of the scene by solving a sparse recovery problem, wherein the sparse recovery problem is configured to determine a set of image shifts of the radar image corresponding to different uncertainties of the antennas, wherein the set of antennas are under uncertainties caused by one or a combination of position ambiguities and clock ambiguities of each of the antennas, and update an estimate of the radar image, based on the determined set of image shifts of the radar image, until a termination condition is met, such that, for each of the antennas, the estimate of the radar image shifted by the corresponding shift of the radar image fits the radar measurements of the antenna, wherein the sparse recovery problem is solved with a neural network denoiser that denoises a filtering of the estimate of the radar image; and render the radar image when the termination condition is met. Further, the neural network denoiser is trained with a training dataset, wherein the training dataset includes input-output pairs (zm, xm), where training input images zm are back projected images formed by applying a back projection operator AmH to example radar measurements ym = Amxm + n, such that zm =AmHym, output images xm are ground truth object radar images, and n represents added noise; the filtering of the estimate of the radar image is performed by applying a forward radar operator and an adjoint radar operator to the estimate of the radar image; the forward radar operator and the adjoint radar operator are configured to filter noise from the estimate of the radar image that the neural network denoiser is not trained on, to produce a filtered radar image; and the filtered radar image is applied to the neural network denoiser.

Accordingly, another embodiment discloses a method for generating a radar image of a scene. The method comprises receiving radar measurements of the scene collected from a set of antennas, wherein the radar measurements are measurements associated with reflections of a radar pulse transmitted to the scene; generating the radar image of the scene by solving a sparse recovery problem, wherein the sparse recovery problem is configured to determine a set of image shifts of the radar image corresponding to different uncertainties of the antennas, wherein the set of antennas are under uncertainties caused by one or a combination of position ambiguities and clock ambiguities of each of the antennas, and update an estimate of the radar image, based on the determined set of image shifts of the radar image, until a termination condition is met, such that, for each of the antennas, the estimate of the radar image shifted by the corresponding shift of the radar image fits the radar measurements of the antenna, wherein the sparse recovery problem is solved with a neural network denoiser that denoises a filtering of the estimate of the radar image; and rendering the radar image when the termination condition is met. Further, the neural network denoiser is trained with a training dataset, wherein the training dataset includes input-output pairs (zm, xm), where training input images zm are back projected images formed by applying a back projection operator AmH to example radar measurements ym = Amxm + n, such that zm =AmHym, output images xm are ground truth object radar images, and n represents added noise; the filtering of the estimate of the radar image is performed by applying a forward radar operator and an adjoint radar operator to the estimate of the radar image; the forward radar operator and the adjoint radar operator are configured to filter noise from the estimate of the radar image that the neural network denoiser is not trained on, to produce a filtered radar image; and the filtered radar image is applied to the neural network denoiser.

Accordingly, yet another embodiment discloses a non-transitory computer-readable storage medium as defined in claim <NUM>.

In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure may be practiced without these specific details. In other instances, apparatuses and methods are shown in block diagram form only in order to avoid obscuring the present disclosure.

As used in this specification and claims, the terms "for example," "for instance," and "such as," and the verbs "comprising," "having," "including," and their other verb forms, when used in conjunction with a listing of one or more components or other items, are each to be construed as open ended, meaning that that the listing is not to be considered as excluding other, additional components or items. The term "based on" means at least partially based on. Further, it is to be understood that the phraseology and terminology employed herein are for the purpose of the description and should not be regarded as limiting. Any heading utilized within this description is for convenience only and has no legal or limiting effect.

<FIG> shows a block diagram of a radar system <NUM>, according to an embodiment of the present disclosure. The radar system <NUM> includes a set of antennas <NUM> and a system <NUM> for generating the radar image of the scene. The scene may be a region of interest that includes one or more objects. The one or more objects may include stationary objects and/or moving objects. The set of antennas <NUM> includes M distributed antennas 101a, 101b,. , <NUM> that can communicate with each other. The system <NUM> includes a memory <NUM>, a processor <NUM>, and a user interface <NUM>. The set of antennas <NUM> is configured to transmit radar pulses to the scene and receive corresponding reflections. The reflections of the radar pulses transmitted to the scene are referred to as radar measurements (hereinafter 'measurements'). The measurements may be stored in the memory <NUM>. The processor <NUM> is configured to solve a sparse recovery problem to produce an auto-focused high resolution two-dimensional (2D) radar image of the scene. The user interface <NUM> is configured to render the produced radar image.

<FIG> is a schematic illustrating the radar system <NUM>, according to embodiments of the present disclosure. The radar system <NUM> may be an airborne platform or a vehicle mounted platform that includes at least one moving antenna, and a set of M distributed moving receiver platforms or receivers. For instance, the set of antennas <NUM> may be moving in space along a predefined trajectory. To that end, the set of antennas comprises at least one antenna, such as the antenna 101a, the antenna 101b, the antenna 101c, and the antenna 101d. It may be understood by one of ordinary skill in the art that four antennas (101a - 101d) are shown in <FIG> for exemplar purpose. In practice, any number of antennas may be used to achieve the functionalities of the radar system <NUM> described herein, without deviating from the scope of the present disclosure.

Radar pulses <NUM> are transmitted from the at least one antenna 101a, to illuminate objects <NUM> situated in a region of interest (ROI) <NUM>, and corresponding reflections <NUM> reflected by the objects <NUM> are recorded by the distributed antennas 101a, 101b, 101c and 101d. The reflections <NUM> are characterized as a weighted combination of delayed pulses, where complex weights depend on specific object reflectivity and antenna patterns. Given the transmitted pulses <NUM> and the reflections <NUM>, the radar image can be generated in a range-azimuth plane according to corresponding weights and delays. Azimuth resolution of the radar image depends on a size of an array aperture, and a range resolution depends on a bandwidth of the transmitted pulses <NUM>.

A fundamental challenge that arises in distributed array imaging (i.e., distributed antennas setup) comes from uncertainties caused by one or a combination of position ambiguities and clock ambiguities of each antenna of the set of antennas <NUM>. As used herein, the position ambiguities correspond to error or uncertainty in positions of the set of antennas <NUM>. As used herein, the clock ambiguities indicate that clocks of the set of antennas <NUM> may or may not be synchronized, the set of antennas <NUM> can be either synchronous or asynchronous. While advanced positioning and navigation systems, such as global navigation satellite system (GPS/GNSS) and inertial navigation system (INS) provide somewhat accurate location information, remaining uncertainty in antenna positions can span multiple wavelengths. As a result, the measurements contain a gain and phase ambiguity when inexact antenna positions are used as reference. Consequently, applying standard reconstruction techniques without accounting for the uncertainty in the antenna positions produces out-of-focus radar images.

<FIG> are schematics, that when viewed together, illustrate distortion that affects a measured time domain signal of each antenna due to the uncertainty (or error) in the antenna positions, when measuring reflection of the object <NUM>, according to embodiments of the present disclosure. Further, <FIG> are schematics illustrating effect of the position perturbations of the different antennas, on aligning in time of the reflections.

Further, some embodiments of are based on the recognition that radar autofocus problems of the distributed antennas with the position ambiguities can be an ill-posed problem with a vast number of unknowns. Specifically, when the radar autofocus problem is formulated as recovering a correct radar image from incorrect measurements caused by an incorrect radar operator encoding the position ambiguities, each measurement of the ROI <NUM> includes an error caused by the position ambiguities. Further, due to non-linearity of relationships between the measurements and the errors in the positions of the antennas (such as the antennas 101a - 101d), each sample of the measurements from the same antenna can have a different error, thereby increasing a number of unknowns in a model of the radar autofocus problem.

To that end, the position ambiguity can be modeled as a shift kernel in a domain of the radar image x. Specifically, a new measurement model is given by <MAT> where * is a spatial convolution operator and hm is a sqrt(P) by sqrt(P) shift kernel that captures the position ambiguity. Here, P is much smaller than N and is also smaller than F. As a result, collecting M measurements results in a system of equations with MF equations and MP + N unknowns. Therefore, there exists a suitable number of measurements M, such that MF is larger than MP + N. Specifically, when M > N/(F-P), then the radar autofocus problem may be solved.

<FIG> is a schematic illustrating a mapping between the set of antennas <NUM> having perturbed positions and measuring the reflection of a single object to the set of antennas <NUM> having uniform linear positions and measuring shifted versions of the same object from <FIG>, according to embodiments of the present disclosure. Further, <FIG> is a schematic of a signal model that measured reflections of a single object at the perturbed antenna positions is equivalent to measuring shifted versions of the same object at the erroneous positions of the set of antennas <NUM>.

<FIG> is a schematic illustrating a relationship between shifted object images 119a, 121a, 123a, 125a and a true object image <NUM> convolved with a set of shift kernels 119b, 121b, 123b, 125b, according to an embodiment of the present disclosure. Measured reflections 101ax, 101bx, 101cx, 101dx in <FIG> correspond to measurements of the true object <NUM> by the set of antennas <NUM> located at perturbed positions in <FIG>. These same measured reflections also correspond to radar reflections of shifted objects 119a, 121a, 123a, 125a by the set of antennas <NUM> located at erroneous antenna positions in <FIG>, which in turn are shown to be equivalent to reflections of the true object <NUM> that is convolved with the set of shift kernels 119b, 121b, 123b, 125b in <FIG>.

As an example, consider a radar autofocus problem where F = <NUM>, N = <NUM>×<NUM> = <NUM>, P = <NUM>×<NUM> = <NUM>, and K = <NUM>. For M = <NUM> measurement vectors ym, the measurement model (i) results in a system with <NUM> equations with approximately <NUM> + <NUM> times log(<NUM>) = <NUM> unknowns. On the other hand, the new measurement model results in a system with <NUM> equations and approximately <NUM> times log(<NUM>) + <NUM> times log(<NUM>) = <NUM> unknowns. This reduction in the number of the unknowns of the radar autofocus problem allows to solve the radar autofocus problem in an efficient manner.

Further, some embodiments are based on recognition that the set of shift kernels (hereinafter `a set of image shifts') and the radar image can be determined using an alternating optimization. For example, in an embodiment, the radar image is updated while fixing the set of image shifts and the set of image shifts are updated while fixing the radar image.

<FIG> shows a block diagram of the alternating optimization for estimating the radar image and the set of image shifts, according to an embodiment of the present disclosure. The alternating optimization is executed by the processor <NUM>. At block <NUM>, a radar image is estimated based on the transmitted pulses <NUM> and radar reflections <NUM>. <FIG> shows a graph illustrating transmitted pulses <NUM>, according to an embodiment of the present disclosure. <FIG> shows a graph illustrating radar reflections <NUM>, according to an embodiment of the present disclosure. The estimation of the radar image is explained in detail below with reference to <FIG>.

At block <NUM>, a set of image shifts corresponding to different uncertainties of the set of antennas <NUM> are estimated. The estimation of the set of image shifts is explained in detail below with reference to <FIG> and <FIG>. Further, at block <NUM>, the estimated radar image is updated based on the set of image shifts. The updating of the estimated radar image <NUM> is explained in detail below with reference to <FIG>.

The steps of estimating the set of image shifts followed by updating the radar image are conducted iteratively <NUM> until a termination condition is met. When the termination condition is met, a focused radar image <NUM> is outputted.

<FIG> shows a block diagram of a method for estimating the radar image, according to an embodiment of the present disclosure. In an embodiment, the measurements of each antenna are modelled based on the transmitted pulses <NUM> and erroneous antenna positions. At block <NUM>, a difference between the radar reflections <NUM> and the modelled measurements is minimized to produce an estimate of the radar image <NUM>. Not in accordance with the claimed invention, at block <NUM>, a regularizer is applied on the estimated radar image <NUM> to filter noise from the estimated radar image <NUM>. The regularizer may be a fused Lasso regularizer that includes a one norm regularizer and a total variation (TV) regularizer. The one norm regularizer imposes sparsity on the estimated radar image <NUM>, while the TV regularizer reduces noise in the estimated radar image <NUM> to produce a filtered radar image.

Further, the measurements are modeled using the filtered radar image, the transmitted pulses <NUM>, and the erroneous antenna positions. Further, a difference between the radar reflections <NUM> and the modeled measurements is minimized to produce an estimate of the radar image, and subsequently, the regularizer is applied on the estimated radar image to produce a filtered radar image. The steps of minimizing the difference between the radar reflections and the modelled measurements and applying the regularizer are conducted iteratively until convergence <NUM>. Upon convergence, an estimate of the radar image <NUM> is outputted.

The embodiments are based on observation that the regularizer (e.g., the fused Lasso regularizer) used in the alternating optimization to filter the estimated radar image (e.g., the estimated radar image <NUM>) may not be suitable when the noise in the estimated radar image is large because the noise may introduce image-domain artifacts that also tend to cluster together and therefore resemble a shape of true objects. As a result, the alternating optimization may focus on a false object and consequently produce incorrect antenna position correction.

The embodiments are based on realization that, to mitigate such a problem, a neural network denoiser can be used instead of the regularizer. The neural network denoiser includes a neural network trained to denoise the estimated radar image <NUM>. In other words, the neural network denoiser filters the noise from the estimated radar image <NUM>. The neural network denoiser is advantageous over the regularizer because of high modelling power of the neural network and its ability to represent signal structure that surpasses an explicit sparsity in image domain and its gradient.

According to the invention, for estimating the radar image, instead of applying the regularizer on the estimated radar image <NUM>, the neural network denoiser is applied <NUM>, as shown in <FIG>.

The embodiments are based on recognition that artifacts/noises in the estimated radar image <NUM> are due to iterative update steps that occur during the minimization of the difference between the radar reflections and the modelled measurements. The artifacts/noises due to the iterative update steps are unknown and unpredictable and may not be same as noise that the neural network is trained on. Therefore, the estimated radar image <NUM> may include noises that the neural network denoiser is not trained on. When such radar images are applied to the neural network denoiser, the neural network denoiser may inject errors in the output radar image <NUM>. Therefore, merely replacing the regularizer with the neural network denoiser is not sufficient and may pose problems.

The embodiments are based on realization that to manage the noises that the that the neural network denoiser is not trained on, the estimated radar image <NUM> can be filtered, prior to applying <NUM> the neural network denoiser, by applying <NUM> a forward radar operator and an adjoint radar operator on the estimated radar image <NUM>, as shown in <FIG>. The forward and the adjoint radar operators filter the noise from the estimated radar image <NUM> that the neural network denoiser is not trained on, to produce a filtered radar image <NUM>. Further, the neural network denoiser is applied <NUM> on the filtered radar image <NUM>. As a result, the neural network denoiser does not inject errors in the output radar image <NUM> and reconstruction of the radar image remains stable.

The steps of minimizing <NUM> the difference between the radar reflections and the modelled measurements, applying <NUM> the forward and the adjoint radar operators, and applying <NUM> the neural network denoiser are conducted iteratively until convergence <NUM>. Upon convergence, the estimate of the radar image <NUM> is outputted.

<FIG> shows an exemplar estimate of the radar image <NUM>, according to an embodiment of the present disclosure.

After determining the estimate of the radar image <NUM>, the set of image shifts corresponding to different uncertainties of the set of antennas <NUM> are estimated. The estimation of the set of image shifts is explained in detail below with reference to <FIG> and <FIG>.

<FIG> shows a block diagram of a method for estimating the set of image shifts, according to an embodiment of the present disclosure. In an embodiment, the measurements are modelled based on the transmitted pulses <NUM>, initial image shifts, and the radar image <NUM>. At block <NUM>, a difference between the radar reflections <NUM> and the modelled measurements is minimized. At block <NUM>, one norm regularization is applied to enforce a single non-zero entry in the set of image shifts. The steps of minimizing <NUM> the difference between the radar reflections and the modelled measurements and applying <NUM> the one norm regularization <NUM> are conducted iteratively <NUM> until convergence. Upon convergence, an estimate of the set of image shifts <NUM> is outputted.

Further, at block <NUM>, the estimated set of image shifts <NUM> is aligned according to average of true positions of the set of antennas <NUM> to produce a new estimate of a set of image shifts <NUM>. <FIG> shows a schematic illustrating aligning of the estimated set of image shifts <NUM> according to average <NUM> of true positions of the set of antennas <NUM>, according to an embodiment of the present disclosure. The average <NUM> of true positions of the set of antennas <NUM> is known. The estimated set of image shifts <NUM> is aligned such that an average <NUM> of the estimated set of image shifts is equal to the average <NUM> of the true positions of the set of antennas <NUM>.

Further, the estimated radar image <NUM> is updated based on the new estimate of set of image shifts <NUM>, such that, for each of the set of antennas <NUM>, the estimate of the radar image shifted by the corresponding image shift fits the measurements of the antenna. The updating of the estimated radar image <NUM> is explained in detail below with reference to <FIG>.

<FIG> shows a block diagram of a method for updating the estimated radar image <NUM>, according to an embodiment of the present disclosure. In an embodiment, the measurements are modelled based on the transmitted pulses <NUM>, the new estimate of set of image shifts <NUM>, and the radar image <NUM>. At block <NUM>, a difference between the radar reflections <NUM> and the modelled measurements is minimized to produce an estimate of the radar image. At block <NUM>, the forward and the adjoint radar operators are applied on the estimated radar image to produce a filtered radar image. Further, at block <NUM>, the neural network denoiser is applied on the filtered radar image to denoise the filtered radar image.

The steps of minimizing <NUM> the difference between the radar reflections and the modelled measurements, applying <NUM> the forward and the adjoint radar operators, and applying <NUM> the neural network denoiser are conducted iteratively <NUM> until convergence. Upon convergence, an updated radar image <NUM> is outputted.

<FIG> shows an exemplar updated radar image <NUM>, according to an embodiment of the present disclosure.

The steps of estimating the set of image shifts followed by updating the radar image are conducted iteratively until the termination condition is met. When the termination condition is met, the focused radar image <NUM> is outputted. The termination condition may be a number of iterations or a condition where the updated radar images of two consecutive iterations remain unchanged.

<FIG> shows an exemplar focused radar image <NUM>, according to an embodiment of the present disclosure. The aforesaid problem of generating the focused radar image <NUM> using the alternating optimization is referred to as the sparse recovery problem.

In an embodiment, the neural network denoiser that denoises a filtering of the estimate of the radar image, includes a residual Unet architecture. Additionally or alternatively, in an alternate embodiment, the neural network denoiser includes a denoising convolutional neural network architecture.

<FIG> shows a residual Unet architecture <NUM>, according to an embodiment of the present disclosure. The residual Unet architecture <NUM> is symmetric and includes two parts, namely, left part <NUM> and right part <NUM>. The left part <NUM> is called contracting path, which is constituted by a general convolutional process. The right part <NUM> is expansive path, which is constituted by transposed 2d convolutional layers.

The embodiments are based on realization that the forward and the adjoint radar operators filter the noise from the estimated radar image, that the neural network denoiser is not trained on, therefore the neural network denoiser need not be trained with a large training dataset including all possible noises that may be observed in the alternative optimization. Thus, the neural network denoiser can be trained with a simpler training dataset. In embodiments, the simpler training dataset includes input-output pairs (zm, xm), where training input images zm are back projected images formed by applying the back projection operator AmH to example radar measurements ym = Amxm + n, such that, zm =AmHym; and the output images xm are ground truth object radar images, and where n represents added noise. As the neural network denoiser can be trained with the simple training dataset, the training of the neural network denoiser becomes easier and faster.

Additionally, some embodiments are based on recognition that the neural network denoiser based radar autofocus problem of the present disclosure (explained in <FIG>) is capable of reconstructing the objects with fewer measurements. This feature allows the embodiments of the present embodiments to be deployed in an object tracking system where radar characteristics of the objects are determined at a rate that is faster than motion of the objects. Such a process allows to continuously detect the objects and perform object tracking-by-detection.

Moreover, detections of moving objects received from the radar reflections may sometimes provide a partial view of the object when the object has a large extension relative to a beam width of the radar. In such scenarios, the neural network denoiser can be trained to reconstruct a partial shape of the object that is intended to be tracked instead of reconstructing point objects of strong reflectors of the object. The reconstructed partial shapes can then be used to facilitate additional modules that determine a current state of the object, where the state of the object includes information on a size, an orientation, and/or a position of the object.

<FIG> shows a schematic of a vehicle <NUM> including a tracker <NUM> for determining a state of a moving object, according to an embodiment of the present disclosure. The tracker <NUM> is associated with the system <NUM>. The vehicle <NUM> may be any type of wheeled vehicle, such as a passenger car, a bus, or a rover. Also, the vehicle <NUM> can be an autonomous or semi-autonomous vehicle. In one embodiment, a steering system <NUM> of the vehicle <NUM> is controlled by the tracker <NUM>. Additionally or alternatively, the steering system <NUM> may be controlled by a driver of the vehicle <NUM>.

In some embodiments, the vehicle <NUM> includes an engine <NUM>, which can be controlled by the tracker <NUM> or by other components of the vehicle <NUM>. In some embodiments, the vehicle <NUM> includes an electric motor in place of the engine <NUM> which is controlled by the tracker <NUM> or by other components of the vehicle <NUM>. The vehicle <NUM> can also include one or more sensors <NUM> to sense the surrounding environment. Examples of the sensors <NUM> include distance range finders, such as radars. In some embodiments, the vehicle <NUM> includes one or more other sensors <NUM> to sense its current motion parameters and internal status. Examples of the one or more other sensors <NUM> include global positioning system (GPS), accelerometers, inertial measurement units, gyroscopes, shaft rotational sensors, torque sensors, deflection sensors, pressure sensor, and flow sensors. The sensors, such as the one or more sensors <NUM> and the one or more other sensors <NUM>, provide information to the tracker <NUM>. The vehicle <NUM> may be equipped with a transceiver <NUM> enabling communication capabilities of the tracker <NUM> through wired or wireless communication channels with the system <NUM>. The tracker <NUM> includes a processor, and a memory that stores instructions that are executable by the processor. The processor can be a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The memory can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems.

<FIG> shows a schematic illustrating controlling of the vehicle <NUM> based on the state of a moving object such as a vehicle <NUM>, according to an embodiment of the present disclosure. The vehicles <NUM> and <NUM> are moving along a road <NUM> and the vehicle <NUM> is behind the vehicle <NUM>. The system <NUM> associated with the vehicle <NUM> is communicatively coupled to the set of antennas <NUM> shown in <FIG>. The set of antennas <NUM> may be covering a region of interest including the road <NUM>. The set of antennas <NUM> may transmit pulses to the region of interest and receive corresponding reflections. Further, the system <NUM> receives the reflections from the set of antennas <NUM> and generates a radar image by solving the sparse recovery problem described in previous embodiments. The radar image indicates locations of point detections in a space of the vehicle <NUM>. For instance, the radar image provides a set of point detections of the vehicle <NUM>. The radar image has no artifacts, less noise, is generated by computationally effective techniques described in previous embodiments and is a better sampling of a distribution of possible detections for the vehicle <NUM>. The radar image is input to the tracker <NUM>. In an embodiment, the tracker <NUM> determines a state of the vehicle <NUM> based on the received radar image, a motion model and a compound measurement model of the vehicle <NUM>. The motion model and the compound measurement model may be stored in the memory of the tracker <NUM>. According to an embodiment, the compound measurement model includes multiple probabilistic distributions constrained to lie on a contour of the vehicle <NUM> with a predetermined relative geometrical mapping to a center of the vehicle <NUM>.

Additionally, in some embodiments, the tracker <NUM> generates control inputs based on the state of the vehicle <NUM> for controlling the vehicle <NUM>. The control inputs, for example, include commands specifying values of one or combination of a steering angle of wheels of the vehicle <NUM>, a rotational velocity of the wheels, and an acceleration of the vehicle <NUM>. The control inputs aim to keep the vehicle <NUM> within particular bounds of the road <NUM> and aims to avoid the vehicle <NUM>. For example, the control inputs cause the vehicle <NUM> to navigate along a trajectory <NUM> to safely pass the vehicle <NUM>.

<FIG> shows a block diagram of a method <NUM> for generating a radar image of a scene, according to an embodiment of the present disclosure. At block <NUM>, the method <NUM> includes receiving radar measurements collected from a set of antennas (such as the set of antennas <NUM>). The radar measurements, for example, correspond to the reflections <NUM> explained in <FIG>.

At block <NUM>, the method <NUM> includes obtaining an estimate of the radar image. The estimated of the radar image is obtained as explained in <FIG>.

At block <NUM>, the method <NUM> includes determining a set of image shifts of the radar image corresponding to different uncertainties of the set of antennas. The set of image shifts is determined as explained in detail in <FIG> and <FIG>.

At block <NUM>, the method <NUM> includes updating the estimate of the radar image, based on the determined set of image shifts of the radar image, such that, for each of the antennas, the estimate of the radar image shifted by the corresponding shift of the radar image fits the measurements of the antenna. The estimate of the radar image is updated as explained in detail in <FIG>.

At block <NUM>, the method <NUM> includes determining if a termination condition is met. If the termination condition is not met, then further a new set of image shifts is determined. The steps of determining the set of image shifts followed by updating the estimate of the radar image are executed iteratively until the termination condition is met. The termination condition may be a number of iterations or a condition where the radar images of two consecutive iterations remain unchanged.

If the termination condition is met, then, at block <NUM>, the method <NUM> includes outputting the radar image.

The formulation of the neural network denoiser based radar autofocus problem is mathematically described below.

An image of the ROI <NUM> is to be recovered by processing F - dimensional frequency-domain measurements <MAT> from M distributed antennas <NUM> that suffer from position ambiguity. The present disclosure has developed an image reconstruction framework <FIG>, wherein a perturbation in the antenna positions results in a measured signal that corresponds to an image-domain convolution model as illustrated in <FIG>, <FIG>. More precisely, by denoting a radar propagation matrix at correct antenna positions by Ãm, and denoting the corresponding matrix at incorrect positions by Am, results ỹm = Ãmx ≠ Amx. However, the only provided measurements are ỹm and the matrices Am. The position ambiguity can be modeled as a time-domain convolution with the measurements, or equivalently, as a gain and phase ambiguity in the frequency-domain of the radar signal, that is, <MAT> where Dĝm is a diagonal matrix with a phase correction vector ĝm ∈ CF on its diagonal entries, and nm is a measurement noise vector. The system in (<NUM>) is ill-posed in general since for any M measurements, which leaves us with MF equations and NW+N unknowns. Alternatively, an aspect of the present disclosure is to represent the gain and phase ambiguity as an image-domain convolution <FIG> where a two-dimensional spatial shift kernel hm is applied to the radar image x, i.e., <MAT>.

Under this particular model, the shift kernels are one-sparse vectors with unknown support locations, thereby reducing unknown degrees of freedom to Mlog(F)+N.

The present disclosure includes considering a two-dimensional radar imaging scenario in which M distributed antennas are used to detect K objects. The objects are located within a spatial region of interest that is discretized on a grid Ω ⊂ R<NUM>, |Ω| = N and N = Nx × Ny with Nx and Ny specifying a number of grid points in horizontal and vertical directions. Denote by l ∈ Ω a spatial position of a grid-point in Ω.

Let Γ ⊂ R<NUM>, |Γ| = M be a set of all the spatial locations of the M antennas. Without loss of generality, assume that a subset of the antennas function as transmitter/receivers while the remaining antennas are only receivers. A transmitting antenna at position r ∈ Γ emits a time-domain pulse p(t) with frequency spectrum P(ω), where ω=2πf is an angular frequency and f ∈ B is an ordinary frequency in a signal bandwidth B, |B| = F.

Denote by ym := y(rm, r'm) and by Am := A(rm, r'm) the corresponding measurement vector and imaging operator of antenna pair (rm, r'm) indexed by m. Let ỹm = rm, + em and r̃'m = r'm + e'm be perturbed transmitter and receiver positions, respectively, where em and e'm, denote positioning errors. The received antenna measurement ỹm: = y(r̃m, r̃'m) observes scene reflectivity x through the perturbed imaging operator Ãm: = A(r̃m, r̃'m) , i.e., <MAT>.

Since the operator Ãm is unknown, the received measurements ỹm are defined as a function of Am and x.

The present disclosure uses approaches for radar autofocus that utilize a gain and phase correction in frequency measurement to describe ỹm in terms of Am and x. More precisely, let ĝm ∈ CF be a complex valued vector corresponding to Fourier transform of a time-domain kernel gm ∈ RM. The received measurement is expressed as in (<NUM>). Therefore, given M measurements ỹm, m ∈ {<NUM>. M}, the radar autofocus problem is regarded as a bilinear inverse problem in both the reflectivity image x and the phase correction vectors ĝm for all m.

The system in (<NUM>) has F equations with F+N unknowns, which makes it severely ill-posed. Even in a case where x is sparse, the problem remains ill-posed since a general phase correction vector ĝm continues to have F degrees of freedom. In order to make the problem tractable, the kernels gm = <MAT> can be assumed to be shift kernels, which reduces its degrees of freedom to a singe phase angle. However, the approximation that gm is a shift operator is only valid in the far field regime and where the position error can be approximated by a one-dimensional shift in a down-range direction of virtual antenna array.

The present disclosure also considers an alternate model to the convolution with a shift kernel in the measurement-domain by switching the convolution to the image-domain. Let <MAT>, Nh ≤ min{Nx, Ny} be a vectorized two-dimensional shift kernel of size Nh × Nh. Under new model, the received signal of the antenna pair indexed by m is written as in (<NUM>).

The present disclosure considers the image-domain convolution model that can be expressed in the spatial Fourier domain as <MAT> <MAT> where F<NUM> is a two-dimensional Fourier transform operator applied to the vectorization of a matrix, ĥm = F<NUM>hm and x̂ = F<NUM>x denote two-dimensional Fourier transforms hm and x, respectively, and Dĥm is a diagonal matrix with ĥm on the diagonal. The present disclosure presents a block coordinate descent approach for computing the radar reflectivity image x and the spatial convolution filters hm from noisy measurements ỹm.

Initially, first is to incorporate into the model in (<NUM>) the prior information that the radar image x is sparse and has a shape that can be learned from training data, and that the kernels hm are two-dimensional shift operators.

Therefore, a neural network denoising operator is used to refine the estimate of the radar image x, wherein a regularizer Rx(·) is added for x, and an ℓ<NUM> norm regularizer Rx(·) to hm. The overall optimization problem can be described as follows <MAT> where <NUM> is all one vector, and as before, ĥm = F<NUM>hm and x̂ = F<NUM>x. Parameters µ and λ are regularization parameters controlling a tradeoff between signal priors and data mismatch cost.

The neural network based regularizer Rx(x) can be imposed implicitly through action of a neural network denoiser <IMG>(zm) that is learned using training data examples composed of input-output pairs (zm, xm), where training input images zm are back projected images formed by applying a back projection operator AmH to example radar measurements ym = Amxm + n, such that, zm =AmHym; and output images xm are ground truth object radar images, and where n represents added noise.

Alternatively, the neural network based regularizer can be applied explicitly, given the neural network denoiser <IMG>(zm), using a framework called regularization-by-denoising (RED), such that the regularizer is defined as <MAT>.

On the other hand, a property of the shift kernel requires that every hm is one sparse with a nonzero entry equal to one. Since hm is nonnegative with a sum of its entries equal to one, the only regularization required is ℓ<NUM> norm penalty: <MAT>.

<FIG> shows a block coordinate descent algorithm 900a for solving (<NUM>), according to some embodiments of the present disclosure. The problem (<NUM>) is nonconvex and at least one goal of the present disclosure regarding this context is to find a stationary point to the problem (<NUM>). Therefore, the block coordinate descent algorithm 900a alternates between descent steps for each of x and hm, for all m. <FIG> illustrates a block diagram of the steps involved in the algorithm shown in <FIG>. At step <NUM>, the shift kernels hm are all initialized to no-shift kernel h<NUM>, an Nh × Nh zero-valued matrix with a central entry set equal to one.

At step <NUM>, an operator <MAT> is determined for all m. At step <NUM>, x is determined by executing a fista subroutine. For each descent step, a small number of iterations of fast iterative shrinkage/thresholding algorithm (FISTA) adapted to the appropriate regularizer of x <NUM> or hm <NUM> are applied. Moreover, every descent step of hm, produces an estimate h̃m which does not necessarily satisfy shift kernel properties, since only a small number of FISTA iterations are run. Therefore, at step <NUM>, a projector P(h̃m) is used onto a space of shift kernels which sparsifies h̃m by setting its largest entry that is closest to center to one and setting the remaining entries to zero.

<FIG> shows an algorithm 900b of FISTA subroutine for updating hm, according to some embodiments of the present disclosure. <FIG> shows an algorithm 900c of FISTA subroutine for x, according to some embodiments of the present disclosure. In general, FISTA can be used to solve convex optimization problems of the form <MAT> where D(u) is a smooth data fidelity cost function and R is a regularizer which can be a non-smooth function. In context of the block coordinate descent algorithm, subroutine fista(Am, R, ym, uinit, T) is defined as an algorithm that runs T iterations of FISTA procedure with a data fidelity cost function D(u), regularizer R, and initial guess uinit. The data fidelity cost function is specified by (<NUM>) as <MAT> where u refers to either the image x or the sequence of convolution kernels hm. The forward operator with respect to the image x given the estimates of the kernels <MAT> at iteration t is defined as <MAT>.

Similarly, the forward operator with respect to hm given the estimate of the image xt at iteration t is defined as <MAT>.

Note that the expression for D in (<NUM>) is separable in hm for every m. Therefore, the FISTA subroutines for updating hm for every m are described in <FIG>. Function T+(z; τ) is a non-negative soft-thresholding operator that sets all values of z less than tau to zero. In <FIG>, updating procedure is specific for using explicit regularizing functions, such as, one-norm penalty or a fused Lasso penalty. A nonlinear thresholding function that is explicit to a penalty function that is used is denoted by T+(z; αµ). In <FIG>, updating procedure is specific for using a neural network denoiser, where the neural network denoiser is denoted by T(vt). A filtering step is performed in step <NUM> of the algorithm 900c to eliminate the noise that the neural network denoiser was not trained on during its training.

<FIG> is a block diagram of a computer system <NUM> of the radar system contemplated by the present disclosure, in accordance with some embodiments of the present disclosure. The computer system <NUM> is in communication with the set of radar platforms or antennas <NUM> and can store collected data in a memory <NUM> that is processed by a processor <NUM> of the computer system <NUM>. The computer system <NUM> can include a human machine interface or user interface <NUM> that can connect the computer system <NUM> to a keyboard <NUM> and display device <NUM>. The computer system <NUM> can be linked through a bus <NUM> to a display interface <NUM> adapted to connect the computer system <NUM> to a display device <NUM>, wherein the display device <NUM> can include a computer monitor, camera, television, projector, or mobile device, among others.

The computer system <NUM> can include a power source <NUM>, depending upon the application the power source <NUM> may be optionally located outside of the computer system <NUM>. The processor <NUM> may be one or more processors that can be configured to execute stored instructions, as well as be in communication with the memory <NUM> that stores instructions that are executable by the processor <NUM>. The processor <NUM> can be a single core processor, a multi-core processor, a computing cluster, or any number of other configurations. The processor <NUM> is connected through the bus <NUM> to one or more input and output devices. The memory <NUM> can include random access memory (RAM), read only memory (ROM), flash memory, or any other suitable memory systems.

Still referring to <FIG>, the computer system <NUM> can also include a storage device <NUM> adapted to store supplementary data and/or software modules used by the processor <NUM>. For example, the storage device <NUM> can store historical data relating to predesigned radar platform trajectories, radar operating frequency bandwidth, transmitted waveform, estimated signal-to-noise ratio, image data relating to object recognition, imaging results using simulated noisy data with different methods dealing with position errors, among other things. The storage device <NUM> can include a hard drive, an optical drive, a thumb-drive, an array of drives, or any combinations thereof.

Still referring to <FIG>, a printer interface <NUM> can also be connected to the computer system <NUM> through the bus <NUM> and adapted to connect the computer system <NUM> to a printing device <NUM>, wherein the printing device <NUM> can include a liquid inkjet printer, solid ink printer, large-scale commercial printer, thermal printer, UV printer, or dye-sublimation printer, among others. A network interface controller (NIC) <NUM> is adapted to connect the computer system <NUM> through the bus <NUM> to a network <NUM>. The image data or related image data, among other things, can be rendered on a display device, imaging device, and/or printing device via the network <NUM>.

Still referring to <FIG>, the image data or related image data, among other things, can be transmitted over a communication channel of the network <NUM>, and/or stored within the computer's storage device <NUM> for storage and/or further processing. Further, the image data or related image data may be received wirelessly or wire from a receiver <NUM> or transmitted via a transmitter <NUM> wirelessly or wire, the receiver <NUM> and transmitter <NUM> are both connected to the computer system <NUM> through the bus <NUM>.

The computer system <NUM> may be connected to external sensors <NUM>, one or more input devices <NUM>, other computers <NUM> and other devices <NUM>. The external sensors <NUM> may include motion sensors, inertial sensors, a type of measuring sensor, etc. The external sensors <NUM> may include sensors for, speed, direction, air flow, distance to an object or location, weather conditions, etc. The input devices <NUM> can include, for example, a keyboard, a scanner, a microphone, a stylus, a touch sensitive pad or display.

The following description provides exemplary embodiments only, and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the following description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments.

Specific details are given in the following description to provide a thorough understanding of the embodiments. However, understood by one of ordinary skill in the art can be that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements in the subject matter disclosed may be shown as components in block diagram form in order not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. Further, like reference numbers and designations in the various drawings indicate like elements.

Also, individual embodiments may be described as a process which is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be rearranged. A process may be terminated when its operations are completed but may have additional steps not discussed or included in a figure. Furthermore, not all operations in any particularly described process may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the function's termination can correspond to a return of the function to the calling function or the main function.

The embodiments of the present disclosure include coherent distributed radar imaging by allowing location ambiguities, and on autofocusing for a single sensor array by distributed sensing with multiple sensors. In particular, a multi-static radar imaging approach where one transmitting/receiving radar platform and multiple receiving radar platforms are moving towards a region of interest (ROI) with position perturbations. The embodiments of the present disclosure detect objects inside the ROI. Due to inaccurate positioning and motion errors, the actual array positions are perturbed up to several times a central radar wavelength. Although image resolution of each sensor array may be low due to its small aperture size, a high-resolution image can be formed by jointly processing outputs of all distributed arrays with well-compensated position errors. The embodiments of the present disclosure assume a sparse scene and is realized iteratively by solving series of optimization problems for compensating position-induced phase errors, exploiting object signatures, and estimating antenna positions.

The embodiments of the present disclosure also provide for autofocus radar imaging for generating a radar image of objects situated in an area of interest using a single moving transmit radar platform or combination transmitter/receiver along with multiple spatially distributed moving radar receiver platforms or receivers. The moving radar receivers are perturbed with unknown position errors up to several radar wavelengths.

Various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages and/or programming or scripting tools, and also may be compiled as executable machine language code or intermediate code that is executed on a framework or virtual machine. Typically, the functionality of the program modules may be combined or distributed as desired in various embodiments.

Claim 1:
A system (<NUM>) configured for generating a radar image of a scene, the system (<NUM>) comprising: a set of antennas (<NUM>), at least one processor (<NUM>); and a memory (<NUM>) having instructions stored thereon that, when executed by the at least one processor (<NUM>), cause the system (<NUM>) to:
receive radar measurements of the scene collected from the set of antennas (<NUM>), wherein the radar measurements are measurements associated with reflections of a radar pulse transmitted to the scene;
generate the radar image of the scene by solving a sparse recovery problem, wherein the sparse recovery problem is configured to:
determine a set of image shifts of the radar image corresponding to different uncertainties of the set of antennas (<NUM>), wherein the set of antennas (<NUM>) are under uncertainties caused by one or a combination of position ambiguities and clock ambiguities of each of the antennas, and
update an estimate of the radar image, based on the determined set of image shifts of the radar image, until a termination condition is met, such that, for each of the antennas, the estimate of the radar image shifted by the corresponding shift of the radar image fits the radar measurements of the antenna (<NUM>), and
wherein the sparse recovery problem is solved with a neural network denoiser that denoises a filtering of the estimate of the radar image; and
render the radar image when the termination condition is met;
characterized in that
the neural network denoiser is trained with a training dataset, wherein the training dataset includes input-output pairs zm,xm, where training input images zm are back projected images formed by applying a back projection operator AmH to example radar measurements ym = Amxm + n, such that zm =AmHym, output images xm are ground truth object radar images, and n represents added noise;
the filtering of the estimate of the radar image is performed by applying a forward radar operator and an adjoint radar operator to the estimate of the radar image;
the forward radar operator and the adjoint radar operator are configured to filter noise from the estimate of the radar image that the neural network denoiser is not trained on, to produce a filtered radar image; and
the filtered radar image is applied to the neural network denoiser.