Title: AdaPKC: PeakConv with Adaptive Peak Receptive Field for Radar Semantic Segmentation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Radar Semantic Segmentation

Adaptive Peak Convolution
Abstract: Deep learning-based radar detection technology is receiving increasing attention in areas such as autonomous driving, UAV surveillance, and marine monitoring. Among recent efforts, PeakConv (PKC) provides a solution that can retain the peak response characteristics of radar signals and play the characteristics of deep convolution, thereby improving the effect of radar semantic segmentation (RSS). However, due to the use of a pre-set fixed peak receptive field sampling rule, PKC still has limitations in dealing with problems such as inconsistency of target frequency domain response broadening, non-homogeneous and time-varying characteristic of noise/clutter distribution. Therefore, this paper proposes an idea of adaptive peak receptive field, and upgrades PKC to AdaPKC based on this idea. Beyond that, a novel fine-tuning technology to further boost the performance of AdaPKC-based RSS networks is presented. Through experimental verification using various real-measured radar data (including publicly available low-cost millimeter-wave radar dataset for autonomous driving and self-collected Ku-band surveillance radar dataset), we found that the performance of AdaPKC-based models surpasses other SoTA methods in RSS tasks. The code is available at https://github.com/lihua199710/AdaPKC.

AdaPKC: PeakConv with Adaptive Peak Receptive Field for Radar Semantic Segmentation

Abstract

Deep learning-based radar detection technology is receiving increasing attention in areas such as autonomous driving, UAV surveillance, and marine monitoring. Among recent efforts, PeakConv (PKC) provides a solution that can retain the peak response characteristics of radar signals and play the characteristics of deep convolution, thereby improving the effect of radar semantic segmentation (RSS). However, due to the use of a pre-set fixed peak receptive field sampling rule, PKC still has limitations in dealing with problems such as inconsistency of target frequency domain response broadening, non-homogeneous and time-varying characteristic of noise/clutter distribution. Therefore, this paper proposes an idea of ada ptive peak receptive field, and upgrades PKC to AdaPKC based on this idea. Beyond that, a novel fine-tuning technology to further boost the performance of AdaPKC-based RSS networks is presented. Through experimental verification using various real-measured radar data (including publicly available low-cost millimeter-wave radar dataset for autonomous driving and self-collected Ku-band surveillance radar dataset), we found that the performance of AdaPKC-based models surpasses other SoTA methods in RSS tasks. The code is available at https://github.com/lihua199710/AdaPKC.

1 Introduction

As a common remote sensing device, radar exhibits superior robustness in complex environments ( e.g. , varying weather and lighting conditions) compared to cameras, and it is more cost-effective and resilient in extreme weather scenarios compared to LiDARs. Benefiting from the physical advantages of radar sensors and the powerful capabilities of deep learning techniques, modern deep learning-based radar signal interpretation has become a hot research topic in the field of radio frequency detection technology. It has been extensively explored in autonomous driving [30, 19, 34, 6], UAV surveillance [9, 17], sea monitoring [27, 21, 28], etc. Considering the similar dense representations between radar frequency maps and optical images, most of these works directly transfer convolution networks or modules developed for optical signals to radar perception tasks, such as radar object detection (ROD) and radar semantic segmentation (RSS), and they have achieved impressive performance. Nevertheless, without specific design for the inherent characteristics of radar signals, these approaches fail to fully liberate the potential of deep learning techniques.

Recently, PKCIn-Net [32] introduced an innovative convolution operator named PeakConv (PKC), tailored for the efficient analysis of radar signals, and this operator seamlessly integrates the advan-

Car

Car

Cyclist

Car

Car

Cyclist

Car

Cyclist

Car

(a) Synchronized camera image of the first frame

(b) Radar frequence map

2019-09-16-12-56-51-000253: RD Input

2019-09-16-12-55-51-000252: RD Inpu

)ofu

Angle()

tages of classic radar detectors [22] with common convolution networks [19, 6]. For radar signals, the frequency responses of objects comprise target echoes and interference, and share a distinct peak-shaped pattern, thus most classic radar detection methods [22, 10, 25, 23] build peak detection algorithms upon constant false alarm rate (CFAR) criteria. Extending from cell averaging-CFAR (CA-CFAR) [22], PKC explicitly embeds a similar band-pass peak enhancement mechanism in a standard convolution operator for better characterising target signatures in radar signals. Concretely, following the guard-reference policy of CA-CFAR, it first estimates interfering signals with the center unit/cell and reference units outside predefined guard bands. Then, with estimated interference, it finishes noise suppression for each cell under test (CUT) in feature space and enhances peak frequency response associated with objects of interest.

Despite its superior suitability for radar data than alternative convolution operators [31, 5, 33], there exists even greater potential for PKC to learn peak frequency response of radar signals. Research on CFAR detectors [10, 25, 23, 11] reveals that there exist significant variations in target signature and associated interference within radar signals, rendering the predefined reference cells in CA-CFAR inadequate for precisely locating interfering signals, and this limitation can also be observed in PKC. To provide a clearer depiction, let us delve deeper into these variations present within radar signals, as illustrated in Fig. 1. On the target side , since multi-dimensional radar tensors are generated through a sequence of cascading fast Fourier transformations (FFTs), target signatures along different dimensions exhibit distinct degrees of frequency response tailing (broadening). Additionally, the broadening degrees of different instances would also be influenced by target categories or states, i.e. , the relative distances, azimuth or velocity from the radar. On the interference side , the noise or clutter distribution commonly exhibits non-homogeneous and time-varying characteristics. However, since the PKC kernel always gathers the reference units at fixed locations for noise estimation, i.e. , the predefined peak receptive field (PRF), the dynamic variations in both targets and interference degrade its performance. In short, the fixed PRF essentially limits the learning ability of PKC , thus, it hinders the RSS model from obtaining better performance.

Motivated by the adaptive selection of reference cells in classic CFAR detectors [10, 25, 23], in this work we introduce two novel data-adaptive band-pass filtering mechanisms aimed at upgrading PKC to adaptively adjust its PRF for each CUT in a data-driven manner, namely ada ptive P ea kC onv (AdaPKC). Concretely, both versions of AdaPKC first measure the correlation between CUT and its

alternative reference units in high-dimensional feature representations, then select proper reference units and integrate them seamlessly with PKC to effectively take care of the fluctuating dynamics of radar signals. The main contributions of our work are:

We present the first attempt specially tailored for radar signal processing to dynamically adjust the receptive field for convolution operators. Concretely, we propose a novel updated version of PKC, termed as AdaPKC, which can adaptively adjust the PRF (or reference units) at cell-level granularity. And two different implementation versions are provided, which both exhibit enhanced flexibility and robustness in handling fluctuating radar signals compared to original PKC.

To better release the learning ability of AdaPKC, a fine-tuning technology with a thresholding on-line switch is presented. With such technology, the same AdaPKC-based model can even achieve better performance with less computational cost.

To verify the effectiveness of AdaPKCs, quantitative and qualitative experiments are conducted on various real-measured large scale radar datasets including CARRADA [20] collected from a low-cost FMCW ( ≈ 77 GHz) radar in autonomous driving scenario and self-collected dataset recorded from a Kurz-under (Ku) band ( ≈ 17 GHz) radar for UAV surveillance and sea monitoring. Results show that AdaPKC-based models achieve SoTA RSS performance and our fine-tuning strategy further brings visible improvements, verifying the scope of application of AdaPKCs.

2 Related Work

Receptive field (RF) adjustment. RF is crucial for modern deep convolution models, affecting the granularity of modeling primitives, computation architecture and their representation capabilities, etc. The rational use of RF can directly improve the representation ability of the models, e.g. enlarging the scope [31], multi-scale modeling [2], and dynamically changing shapes [5, 33]. Beyond that, the concept of RF has also been applied to Transformers [7, 16]. These methods are proposed for vision tasks and have achieved significant results. However, compared with conventional convolution, the improvement is not satisfactory enough on radar signals. Recently, by fully considering the characteristics of radar signals, the concept of PRF has been proposed, which makes the original convolution have the ability of band-pass filtering and noise suppression [32]. However, the bandwidth of filtering is pre-set, which hinders the adaptive ability of PKC to radar data. To this end, this paper attempts to study a data-driven PRF adjustment method, and introduces the concept of adaptive PRF (AdaPRF), so as to further improve the RSS performance. Considering that the adaptive adjustment of the suppression (guard) bandwidth and the non-suppression (reference) units is essentially a dynamic adjustment of RF, and from this point of view, the content studied in this paper is related to deformable convolution (DefConv) [5, 33]. Unfortunately, the dynamic RF technology of DefConvs cannot solve the problem in hand, for the following reasons: i) Differences of signaling mechanism. DefConv uses the visual prior information that the target is visually deformed geometrically, which cannot be directly corresponded to the radar signal. ii) Mismatched prediction method . DefConv generates new RF through prior prediction, i.e. , the regular RF is still used to infer the sampling point outside regular RF. However, in radar signal, the interference (noise/clutter) with large entropy, is often difficult or even impossible to predict. At present, a better way is under the premise of observation, i.e. , posterior measurement or statistics. iii) Different mechanisms of representation . Noise suppression is not required to be considered by DefConv, thus it does not need to distinguish between the center unit and surroundings during calculation. To this end, a novel adaptive RF adjustment method is required.

Radar semantic segmentation. Benefiting from the reliable perceptual capabilities, convolutional neural networks (CNNs) play an indispensable role in existing RSS networks for radar frequency maps processing, whether in pure CNN models [13, 8, 19, 32] or transformer-assisted CNN models [34, 12, 6]. RSS-Net [13] utilizes a fully convolutional neural network with encoder-decoder structure to recognize targets in radar scans, and it incorporates an atrous spatial pyramid pooling (ASPP) [2] module to gather multi-scale spatial information. RAMP-CNN [8] employs parallel branches to extract features from multiple views and adopts 3 Dconvolutions to better capture temporal information. TMVA-Net [19] leverages these techniques to develop a multi-view RSS model, which is capable of making semantic predictions across multiple views simultaneously. T-RODNet [12] integrates Swin Transformer [16] modules into a CNN-based RSS model to strengthen its modeling capability. TransRSS [34] and TransRadar [6] introduce attention blocks into the multi-view feature fusion stage to enhance the fusion effectiveness. Recently, as the first fundamental convolution operator tailored for radar signal processing, PKC [32] is proposed. Compared to Dilated Convolution [31]

and Deformable ones [5, 33], PKC demonstrates superior RSS performance. However, it is inherently constrained by its fixed peak receptive field, posing challenges in achieving consistent interference (noise/clutter) suppression under the dynamic and time-varying nature of radar signals. By contrast, our AdaPKCs overcome this limitation well by implementing novel data-adaptive band-pass filtering mechanisms.

3 Method

In this section, the proposed two versions of AdaPKC with different AdaPRF mechanisms are first introduced in § 3.1. Then, the proposed fine-tuning strategy to further uncover the potential of AdaPKC-based RSS models is presented in § 3.2. We design these models using both multiview [19, 32] and single-view frameworks, which are comprehensively described in Appendix A.2 and A.3 for saving space.

3.1 AdaPKC

3.1.1 AdaPKC ξ : PKC w/ Metric-based AdaPRF

To ensure the reliability of estimating the proper unit-level PRF, i.e. , AdaPRF, for AdaPKC in radar signals, we establish the estimation process in a posterior way: we first define a set of candidate PRFs within the neighbourhood of the center unit, aligning with the local peak response of targets and the local scanning process of convolution, and then design a measuring criterion to evaluate these PRFs, estimating AdaPRF primarily occupied by interfering signals. Motivated by classic CFAR detectors [10, 25, 23], we first demonstrate how to estimate AdaPRF in an explicitly measuring way, referred to as metric-based AdaPRF (AdaPKC ξ ). To illustrate the mechanism of AdaPKC ξ , we begin by defining the search space of alternative PRFs. Reviewing the PRF definition in previous work [32] we can see that, the PRF for center unit x c encompasses x c itself and a set of sampled reference units { x ( i ) r } N r i =1 , and the area of reference units is governed by horizontal- and verticalsymmetry guard bandwidth b G ≜ { b G x , b G y } and reference bandwidth b R ≜ { b R x , b R y } , as illustrated in Fig. 2-(a). Following this definition, the PRF adjustment corresponds precisely to the adjustment of the reference unit set, thus we can define the PRF search space by defining the candidate sets of reference units with the adjustment ranges for the guard bandwidth and reference bandwidth. Given that adjusting the reference bandwidth leads to a drastic change in the number of sampled reference units compared to adjusting the guard bandwidth, in AdaPKC ξ we keep anytime-fixed b R ≜ { b R x = 1 , b R y = 1 } and denote the set of K guard bandwidth candidates as Ω G ≜ { b G k } K k =1 = { b G | b G min | x ≤ b G x ≤ b G max | x , b G min | y ≤ b G y ≤ b G max | y } , generating K sets of reference units as {{ x ( i ) r | k } N k i =1 } K k =1 correspondingly. As a result, AdaPRF estimation in AdaPKC ξ is equivalent to selecting an appropriate reference unit set from these candidate sets for each CUT (or center unit), as illustrated in Fig. 2-(b).

For better explanation, we divide the observed radar signals into three subsets: i) signals reflected directly from a target, S t; ii) the target-interfering noise, i.e. , the noise coupled with the signal that partially leaks out of the target, S t-n; and iii) the target-independent noise, S n. In practice, it is the part that from S t-n really causes misjudgment. Therefore, classic CFAR detectors focus on filtering out such noise either by the extreme value [10, 25] or the median value [23] in amplitude domain of signals. Motivated by this idea, our AdaPKC ξ centers its attention on collecting reference units predominantly occupied by such noise.

However, as a learnable module, AdaPKC ξ needs to process the representation tensors of radar signals, implying that the measurement of the reference units should be conducted on feature space. For some CUT x c = ψ ( s ; W ) and its candidate reference unit x r = ψ ( s ′ ; W ) , where s ∈ S t , s ′ ∈ S t ∪ S t-n ∪ S n, and ψ ( · ; W ) ∈ R C denotes a convolution layer with shared weights, W . Then, AdaPKC ξ should be responsible for transforming these features into a metric space that explicitly delineates their correlation with the target. This transformation is achieved by utilizing the inner product of representations between CUT and its candidate reference unit, i.e. , x c x ⊤ r , sharing a similar spirit with the matched filter concept in conventional radar signal processing [22] and attention in [26].

(b) AdaPRF estimation in

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

𝑔𝑔

(sort

Ξ

)

K candidate PRFs

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

|

𝒚𝒚

𝐆𝐆

𝒃𝒃

𝐦𝐦𝐀𝐀𝐦𝐦

|

𝒚𝒚

𝐆𝐆

Input feature map

K

×

H

×

W

K =

𝑏𝑏max

|

𝑥𝑥

G

-𝑏𝑏min

|

𝑥𝑥

G

+1

⋅

𝑏𝑏max

|

𝑦𝑦

G

-𝑏𝑏min

|

𝑦𝑦

G

+1

Estimation criterion

AdaPRF

Reference unit under confirmation

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

|

𝒙𝒙

𝐆𝐆

𝒃𝒃

𝐦𝐦𝐀𝐀𝐦𝐦

|

𝒙𝒙

𝐆𝐆

Metric

𝑘𝑘

Selection

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

𝑏𝑏

𝑥𝑥

G

𝑏𝑏

𝑦𝑦

G

𝑏𝑏

𝑦𝑦

R

𝑏𝑏

𝑥𝑥

R

𝑘𝑘

†

(a) Fixed PRF in PKC

•

★

Cell Under Test (CUT),

𝐦𝐦

𝒄𝒄

Guard unit

Sampled reference unit,

𝐦𝐦

𝒓𝒓

𝜉𝜉1

,

𝜉𝜉2

, … ,

𝜉𝜉K

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

𝑏𝑏

𝑥𝑥

G

𝑏𝑏

𝑦𝑦

G

𝑏𝑏

𝑦𝑦

R

𝑏𝑏

𝑥𝑥

R

𝐦𝐦

𝑟𝑟↑

Reference bandwidth:

𝐛𝐛

R

≜ 𝑏𝑏𝑥𝑥

R

,

𝑏𝑏

𝑦𝑦

R

Guard bandwidth:

𝐛𝐛

G

≜ 𝑏𝑏𝑥𝑥

G

,

𝑏𝑏

𝑦𝑦

G

𝐦𝐦

𝑟𝑟↓

𝐦𝐦

𝑟𝑟←

𝐦𝐦

𝑟𝑟→

Under this definition, these measures exhibit the following statistical properties,

where, E ( · ) is the expectation and ∥ · ∥ 2 denotes the L 2 norm. From Eq. (1), we can see that the inner product transformation assigns three statistical boundaries to x r from S t , S t-n and S n: for s ′ ∈ S t-n , the expectation E ( x c x ⊤ r ) consistently exhibits smaller value than case for s ′ ∈ S t and larger value than s ′ ∈ S n, with a notable separation between their respective magnitudes. This attribute significantly serves to facilitate the subsequent localization of reference units from target-interfering noise.

Then we elucidate the process of translating the K available sets of reference units (or PRFs) into the previously discussed metric space. Since different sets may comprise varying numbers of units, we uniformly sample N ( 16 by default) units as representatives, as illustrated in Fig. 2-(b). For the center unit x c and its k th reference unit set { x ( i ) r | k } N i =1 , let R k = { x c , { x ( i ) r | k } N i =1 } denote its corresponding PRF. Then the correlation value (or metric score), ξ k for the k th PRF w.r.t. x c is formulated as

where, C is the feature dimension; σ ( · ) is the sigmoid function which normalizes ξ k to (0 , 1) .

With the correlation values Ξ = { ξ k } K k =1 for all alternative PRFs, we can select the appropriate PRF R † , which effectively encompasses target-interfering noise. In view of the attribute presented in Eq. 1, we employ the maximum value of the first-order gradient of Ξ as the selection criterion and please refer to Appendix B.1 for the detailed analysis. Then, we have final selection strategy as follows,

where arg max operator retrieves the index corresponding to the maximum value in g [ sort (Ξ)] ; g is the difference function; sort is the descending sort operator. After obtaining R † , AdaPKC performs a convolution operation similar to PeakConv, detailed in Appendix A.1.

3.1.2 AdaPKC θ : PKC w/ Learning-based AdaPRF

In AdaPKC ξ , a measuring criterion is established in Eq. 3 based on prior knowledge of radar signals, providing a non-parametric way to achieve AdaPRF. Differing from AdaPKC ξ , AdaPKC θ learns to

7

×

1 Conv

1

×

7 Conv

BN

BN

2

×

H

×

W

Input feature map

Guard unit under confirmation

Optimal guard bandwidth

2

×

H

×

W

𝒃𝒃

↑

𝐆𝐆

𝑏𝑏

↑

G

𝑏𝑏

↓

G

𝑏𝑏←

G

𝑏𝑏→

G

Modulated sigmoid function

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

𝐆𝐆

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

𝐆𝐆

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

𝐆𝐆

𝒃𝒃

𝐦𝐦𝐦𝐦𝐦𝐦

𝐆𝐆

𝑏𝑏min

G

𝑏𝑏max

G

𝑏𝑏min

G

𝑏𝑏max

G

𝒃𝒃

↓

𝐆𝐆

𝑏𝑏max

G

𝑏𝑏min

G

𝑏𝑏max

G

𝑏𝑏min

G

𝐆𝐆

𝒃𝒃→

𝐆𝐆

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

𝑏𝑏→

G

𝑏𝑏

↓

G

𝑏𝑏

𝑦𝑦

R

𝑏𝑏

𝑥𝑥

R

𝑏𝑏←

G

𝑏𝑏

↑

G

(a) Example of candidate PRF

Guard bandwidth:

𝐛𝐛

G

G

,

𝑏𝑏

↓

G

,

𝑏𝑏←

G

,

𝑏𝑏→

G

𝐦𝐦

𝑟𝑟←

𝐦𝐦

𝑟𝑟→

ℛ

=

𝐦𝐦

𝑐𝑐

,

𝐦𝐦

𝑟𝑟↑

},

𝐦𝐦

𝑟𝑟↓

,

𝐦𝐦

𝑟𝑟←

, {

𝐦𝐦

𝑟𝑟→

Candidate PRF:

𝐦𝐦

𝑟𝑟↑

𝐦𝐦

𝑟𝑟↓

(b) Estimation network for optimal guard bandwidth

•

★

Cell Under Test (CUT),

𝐦𝐦

𝒄𝒄

Guard unit

Sampled reference unit,

𝐦𝐦

𝒓𝒓

build the criterion in a task-driven manner, employing a small network to estimate AdaPRF, i.e. , a parametric way. Thus, given center unit x c and its K candidate PRFs, {R k } K k =1 , the natural way to locate AdaPRF R † , is to define a function f ( · ; θ ) with learnable parameters θ , which is used to produce the likelihood of each R k being R † , then we can have

At a rough glance, there is no obvious difference between Eq. 4 and Eq. 3. However, AdaPKC θ involves joint optimization of estimation network parameters ( i.e. , θ ) and segmentation model parameters. The use of arg max operation will lead to the loss of gradient information of θ , as a result, the optimization of the estimation network cannot be driven by the segmentation task. To this end, we transform the discrete estimation problem into a continuous form. Firstly, following the setup of anytime-fixed b R ≜ { b R x = 1 , b R y = 1 } in AdaPKC ξ , estimating R † can be equivalently translated into estimating the optimal guard bandwidth b G † . Subsequently, to ensure that the estimation of b G † retains gradient information, the estimation network is designed to generate the continuousvalued b G † , instead of the likelihoods for alternative guard bandwidths. Finally, the derivable linear interpolation is used to associate the continuous-valued b G † with discrete spatial coordinates.

Concretely, given an input feature map X ∈ R C × H × W , our AdaPKC θ is responsible for obtaining an appropriate guard bandwidth B G † = { b G † h,w } H,W h =1 ,w =1 ∈ R 4 × H × W . For enhancing the expressive capability of AdaPKC, the horizontal- and vertical-symmetry b G design in the original PRF is extended to a quadruple form, b G = { b G ↑ , b G ↓ , b G ← , b G → } , so that the shape of PRF would enjoy free change in four directions, i.e. , top , bottom , left , right , as illustrated in Fig. 3-(a), resulting in more diverse band-pass filters. As shown in Fig. 3-(b), we use a small network with two paralleled conv blocks, g Hrz ( · ) : R C × H × W → R 2 × H × W and g Vtc ( · ) : R C × H × W → R 2 × H × W to estimate b G ∈ R 4 for each unit of X in horizontal ( ←→ ) and vertical ( ↑↓ ) directions, respectively. To ensure the directional consistency, the horizontal branch possesses kernels with a size of 1 × (2 b G max +1) , and the kernel size of the vertical branch is (2 b G max +1) × 1 , correspondingly. All four directions have the same lower bound, b G min = 1 and the same upper bound, b G max = 3 by default.

Then, for each x c , its AdaPRF R † = { x c , { x ( i ) † r } N i =1 } ( N = 16 by default) can be obtained by the following two steps:

Step 1. Guard bandwidth estimation:

where, each b G † h,w in B G † is the learned guard bandwidth for x c of the spatial coordinate ( h, w ) ; together with a constant b G max -b G min , the sigmoid activation σ ( · ) can modulate the input values to (0 , b G max -b G min ) ; β ( · ) and ⊕ denotes the BatchNorm and Concat operator, respectively; 1 ∈ Z 4 × H × W is an all-one cube.

Step 2. Reference unit sampling: for better illustration, we first define the default PRF with guard bandwidths equal to 1 in [32] as R 0 = { x c , { x ( i ) r } N i =1 } , which is shown in Fig. 2-(a). let p c = ( h c , w c ) and p ( i ) r = ( h ( i ) r , w ( i ) r ) denote the spatial coordinate of the current center unit x c and one of its reference unit x ( i ) r in R 0 , respectively. Then we split { x ( i ) r } N i =1 into four subsets as { x ( j ) r ↑ } j M =1 , { x ( j ) r ↓ } j M =1 , { x ( j ) r ← } j M =1 and { x ( j ) r → } j M =1 , according to the four directions, where M = N/ 4 , as indicated by the braces drawn in Fig. 2-(a). By using b G † obtained from Step 1 , we can sample R † = { x c , { x ( i ) † r } N i =1 } through linear interpolation, as exemplified in Fig. 3-(a). Taking the top direction as an example, each x ( j ) † r ↑ can be obtained by

Sampling values for the rest directions and back-propagation of gradients in linear interpolation are demonstrated in Appendix B.2.

3.2 Fine-tuning AdaPKC with Thresholding On-line Switch

To further release the learning ability of AdaPKC, a fine-tuning strategy is proposed. To improve interpretability, we focus mainly on optimizations for the explicit AdaPRF estimation version, i.e. , AdaPKC ξ . The motivation comes from two perspectives. i) Model confidence : the representation ability and confidence of the model gradually improve with training, hence, during early training phase the less representative features may result in unreliable metric scores calculated in Eq. 2, consequently affecting the selection of AdaPRF and misleading the subsequent learning process. ii) Data sparsity : AdaPKC is responsible for the PRF adaptation for both background units ( x c / ∈ S t ) and target units ( x c ∈ S t ). However, due to the sparsity of target occupation w.r.t. the radar detection range, most units in the input feature map are background units, thus the heavy class-imbalance hinders the model's concentration on target points and their AdaPRF optimization.

To address the above issues, we propose to fi ne-tune AdaPKC ξ with a t hresholding o n-line s witch (FiTOS), and please refer to Appendix C for a visual illustration. Firstly, a pre-trained PKC model with pre-defined PRFs is used to initialize the AdaPKC-based model to be optimized, so that AdaPKC can have a warm-start before PRF adjustment . Thus, the risk of obtaining unreliable metric scores is greatly reduced. Secondly, considering that the majority of background units are occupied by locally similar noise, i.e. , their monotonic correlation/metric curves, sort (Ξ) , are relatively flat, FiTOS introduces a confidence threshold τ to filter out these background units in spatial dimension on-the-fly . Specifically, if the steepness of the metric curve, i.e. , max { g [ sort (Ξ)] } , is below the threshold τ , then the corresponding unit is considered a background unit, and we retain the initial PRF, i.e. , switch off PRF adjustment. Otherwise, we adopt the newly estimated AdaPRF, i.e. , switch on PRF adjustment. As a result, the PRF selection criterion in Eq. (3) is modified as follows,

where k 0 denotes the index of pre-defined PRF in the pre-trained model.

4 Experiments

To verify the effectiveness of our methods, we conduct quantitative and qualitative experiments on two public multi-view radar datasets and our self-collected single-view radar dataset. For simplicity of notations, AdaPKC ξ -based and AdaPKC θ -based multi-view RSS model is denoted as AdaPKC ξ -Net and AdaPKC θ -Net, respectively. The proposed single-view baseline model is named KuRALS-Net. Additionally, we append 'FiT' in the upper right corner of the models to indicate the use of FiTOS.

4.1 Datasets and Training Setups

CARRADA [20] dataset is recorded by a low-cost FMCW radar in millimeter wave band ( ≈ 77 GHz). It comprises camera-radar synchronised multi-view radar recordings in various scenarios, and contains four categories of objects: pedestrian , cyclist , car and background . The dimensions of provided range-angle-Doppler (RAD) tensors are 256 × 256 × 64 and support multi-view (RA and RD views) RSS task. The dataset splits are the same as in [32, 19]. CARRADA-RAC [32] dataset is derived from CARRADA and mainly calibrates the original RA annotations, see Appendix D.1 for details.

KuRALS dataset is self-collected by a Ku rz-under band ( ≈ 17 GHz) surveillance R adar, which is recorded in multiple scenarios, including A erial vehicles, L and targets and ships on the S ea surface. Different from other public datasets, e.g. , CARRADA [20] and CRUW [29], KuRALS aims at exploring the performance of deep models in the field of monitoring radar, hence offering a greater range field ( ≦ 6 . 4 km) and higher Doppler resolution ( ≈ 0 . 198 m/s). The comprised RD tensors are stored as 2 D matrices of size 2048 (range) by 128 (Doppler). This dataset contains 9 sequences of radar recordings and there exist four moving object categories: UAV , pedestrian , vehicle and ship .

Training Setups . Following previous works, all models are evaluated with Intersection over Union (IoU) and Dice scores. These metrics are averaged across all classes on the test subset for model performance comparison, yielding mean IoU (mIoU) and mean Dice (mDice). Implementation details are presented in Appendix D.2.

4.2 Investigation of AdaPKC Mechanism

To investigate the working mechanism of AdaPKCs, a series of comparison analysis is conducted on CARRADA benchmark. We first compare AdaPKCs with PeakConv and a manual PRF adjustment method. Results in Tab. 1 demonstrate that, both versions of AdaPKC exhibit significant enhancements in RSS performance compared to PeakConv, especially in the RA view, and they incur affordable additional computational complexity and inference speed overhead. Considering the severer signal tailing effect in RA view, this suggests that AdaPKC can better handle situations with frequency response ambiguity. For better illustration, we analyze the distribution of guard bandwidths in Appendix E.1. Additionally, since manually adjusting PRFs directly affects the RSS performance of PeakConv-based models, as discussed in [32], in this work we undertake a similar exploration experiment within a broader range of guard bandwidths. Specifically, different guard bandwidths in range dimension, b G R ∈ { 1 , 2 , 3 , 4 , 5 } , are tested, while the guard bandwidths in angle and Doppler dimensions are fixed at 1 for controlling variables. To ensure consistent parameter counts under different bandwidth settings, we adopt the same strategy of uniform sampling as in AdaPKC ξ . As shown in Tab. 2, presetting a proper PRF globally in a hyper-parameter way can indeed help the PeakConv-based model achieve better performance. However, the manual adjustment way cannot cater to each unit, and the computational cost of traversing the guard bandwidth in all dimensions and directions is quite large. In contrast, AdaPKC completely automates the PRF adjustment, and the adjustment granularity reaches the unit-level. With this unit-level PRF adaptation capability, AdaPKCs demonstrate superior RSS performance compared to the manual adjustment way.

Furthermore, a comparative analysis between AdaPKCs and DefConvs (DefConv and DefConvV2) [5, 33] is conducted under the same RSS framework. Results in Tab. 3 show that, sharing the similar unitlevel dynamic RF adjustment spirit with AdaPKC, DefConvs demonstrate better RSS performance than regular convolution (Conv). However, due to the three task-mismatched reasons discussed in § 2, DefConvs exhibit inferior applicability compared to AdaPKC, highlighting AdaPKC's suitability for

achieving adaptive receptive fields in radar signals. For supplementary purposes, we also show the comparison results between AdaPKC and dynamic CFAR detectors [10, 25, 23] in Appendix E.2.

As further explorations, we present an investigation into adaptive sampling strategies for reference band in Appendix E.3. Additionally, in Appendix E.4 we demonstrate a training strategy to enlarge the search space of alternative PRFs under fixed resource constraints for AdaPKC ξ .

4.3 Comparison with State-of-The-Art (SoTA)

Our methods are further compared with fashionable visual segmentation models and existing SoTA RSS solutions. The quantitative results on the CARRADA benchmark are illustrated in Tab. 4 and the qualitative comparisons are presented in Appendix F.3. Both AdaPKC ξ -Net and AdaPKC θ -Net outperform previous RSS models, including pure CNN models [13, 8, 19, 32] and transformerassisted CNN models [12, 6]. Compared to the baseline model PKCIn-Net, AdaPKC ξ -Net exhibits improvements in both RD and RA views, with a particularly notable enhancement in the RA view. AdaPKC θ -Net purely relies on task-driven PRF adjustment, achieving a better performance balance between two views. Additionally, without consuming extra training resources, our proposed FiTOS strategy further enhances RSS performance of AdaPKC ξ -Net, achieving an overall superiority over AdaPKC θ -Net.

Results on CARRADA-RAC dataset are shown in Tab. 5. Similar to the trend on CARRADA, the evaluation results exhibit the superior performance of our methods than these RSS baseline models. AdaPKC θ -Net still shows a relatively balanced performance improvement in both views. AdaPKC ξ -Net demonstrates a more pronounced improvement in the RD view, while

AdaPKC ξ -Net FiT shows a greater enhancement in the RA view. We speculate that this might be attributed to the collaborative training of different views in the multi-view segmentation task.

4.4 RSS Performance on KuRALS

To verify the effectiveness of AdaPKCs in other application scenarios, we conduct comparative experiments on KuRALS dataset. Quantitative results are shown in Tab. 6 and the qualitative comparisons are presented in Appendix F.5. Compared to conventional visual segmentation models, our proposed KuRALS-Net offers a stronger baseline and AdaPKCs further boost the RSS performance of KuRALS-Net, validating their application potential in surveillance radar detection scenarios.

4.5 Ablation Study for FiTOS

Since the threshold τ plays a vital role in FiTOS, we study the impact by testing different values of τ from 0 . 1 to 0 . 9 . The summary results on mDice are shown in Fig. 4 and the corresponding mIoU results are presented in Appendix F.6. The optimal outcome for AdaPKC ξ -Net FiT in RD and RA view is obtained with τ = 0 . 6 and 0 . 7 , respectively. It is observed that the performance of AdaPKC ξ -Net FiT declines when τ is either too large or too small. When τ goes larger, fewer target units adjust their PRFs during the fine-tuning stage, rendering AdaPKC ξ less effective. Conversely, when τ is small, AdaPKC ξ tends to focus on background units with random fluctuations, resulting in training confusion. Compared to PKCIn-Net, AdaPKC ξ -Net FiT consistently demonstrates superiority under different τ s, highlighting the essentiality of PRF adaptation for the original PKC. When compared to AdaPKC ξ -Net, AdaPKC ξ -Net FiT exhibits superior performance with most values of τ , demonstrating the efficacy of the fine-tuning strategy. Especially, in RD view, the RSS performance shows significant improvement within a broad range of τ . Taking mDice as an example, it consistently surpasses 73 . 5% for τ ∈ [0 . 5 , 0 . 8] , indicating a strong level of robustness w.r.t. τ . 72.0% 72.5% 73.0% 73.5% 74.0% 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 mDice

53.5%

54.0%

54.5%

55.0%

55.5%

56.0%

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

mDice

RA View

72.5%

73.0%

73.5%

74.0%

74.5%

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

mDice

TAPKCIn-Net

TAPKCIn-Net

PKCIn-Net

-𝐍𝐍𝐍𝐍𝐍𝐍

𝑭𝑭𝑭𝑭𝑭𝑭

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

-Net

PKCIn-Net

𝜏𝜏

𝜏𝜏

5 Conclusion

This work delves deeply into the convolution operator for radar signals, PKC, and improves upon it. Due to the design of PRF, PKC filters out the reference units corresponding to interfering signals in a band-pass filtering manner. Compared to other convolution operators in deep learning, PKC obtains a more robust representation by using the center unit and the reference unit to cancel each other out and then weighted fusion. However, the fixed suppression bandwidth ( i.e. , guard bandwidth) setting limits the adaptability of PKC to signal diversity. Based on this, we propose a method for adaptive adjustment of the PRF, and provide two effective solutions based on metrics and learning, i.e. , AdaPKC ξ and AdaPKC θ . In addition, to further boost the learning ability of AdaPKC, a novel fine-tuning strategy is presented. To fully verify the effectiveness of AdaPKC, different real-measured radar datasets are used for experimental analysis. The results show the superior performance of AdaPKC in RSS tasks.

References

Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. CoRR , abs/1706.05587, 2017.

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 , pages 833-851, Cham, 2018. Springer International Publishing.

D. Comaniciu and P. Meer. Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence , 24(5):603-619, 2002.

Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to algorithms . 2022.

Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei. Deformable convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) , Oct 2017.

Yahia Dalbah, Jean Lahoud, and Hisham Cholakkal. Transradar: Adaptive-directional transformer for real-time multi-view radar semantic segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 353-362, 2024.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020.

Xiangyu Gao, Guanbin Xing, Sumit Roy, and Hui Liu. Ramp-cnn: A novel neural network for enhanced automotive radar object recognition. IEEE Sensors Journal , 21(4):5119-5132, 2021.

Daniel Gusland, Sigmund Rolfsjord, and Børge Torvik. Deep temporal detection - a machine learning approach to multiple-dwell target detection. In 2020 IEEE International Radar Conference (RADAR) , pages 203-207, 2020.

V Gregers Hansen. Constant false alarm rate processing in search radars. In IEE Conf. Publ. no. 105," Radar-Present and Future" , pages 325-332, 1973.

Ahsan Jalil, Hassan Yousaf, and Muhammad Iram Baig. Analysis of cfar techniques. In 2016 13th International Bhurban Conference on Applied Sciences and Technology (IBCAST) , pages 654-659. IEEE, 2016.

Tiezhen Jiang, Long Zhuang, Qi An, Jianhua Wang, Kai Xiao, and Anqi Wang. T-rodnet: Transformer for vehicular millimeter-wave radar object detection. IEEE Transactions on Instrumentation and Measurement , 72:1-12, 2022.

Prannay Kaul, Daniele de Martini, Matthew Gadd, and Paul Newman. Rss-net: Weakly-supervised multi-class semantic segmentation with fmcw radar. In 2020 IEEE Intelligent Vehicles Symposium (IV) , pages 431-436, 2020.

D. Kingma and J. Ba. Adam: A method for stochastic optimization. In ICLR , 2015.

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision , pages 2980-2988, 2017.

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10012-10022, 2021.

Zhang Liwen, Pan Jian, Zhang Youcheng, Chen Yuanpei, Ma Zhe, Huang Xuhui, and Sun Kewu. Capturing temporal-dependence in radar echo for spatial-temporal sparse target detection. Journal of Radars , 12(R22228):356, 2023.

Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , June 2015.

Arthur Ouaknine, Alasdair Newson, Patrick Pérez, Florence Tupin, and Julien Rebut. Multi-view radar semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 15671-15680, October 2021.

Arthur Ouaknine, Alasdair Newson, Julien Rebut, Florence Tupin, and Patrick Pérez. Carrada dataset: Camera and automotive radar with range- angle- doppler annotations. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 5068-5075, 2021.

Qizhe Qu, Weijian Liu, Jiaxin Wang, Binbin Li, Ningbo Liu, and Yong-Liang Wang. Enhanced cnn-based small target detection in sea clutter with controllable false alarm. IEEE Sensors Journal , 23(9):1019310205, 2023.

M. A. Richards. Fundamentals of Radar Signal Processing, Second Edition . Fundamentals of Radar Signal Processing, Second Edition, 2005.

Hermann Rohling. Radar cfar thresholding in clutter and multiple target situations. IEEE Transactions on Aerospace and Electronic Systems , AES-19(4):608-621, 1983.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015 , pages 234-241, Cham, 2015. Springer International Publishing.

Gerard V Trunk. Range resolution of targets using automatic detectors. IEEE Transactions on Aerospace and Electronic Systems , (5):750-755, 1978.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

Hao Wan, Xiaoqing Tian, Jing Liang, and Xiaofeng Shen. Sequence-feature detection of small targets in sea clutter based on bi-lstm. IEEE Transactions on Geoscience and Remote Sensing , 60:1-11, 2022.

Jingang Wang and Songbin Li. Maritime radar target detection in sea clutter based on cnn with dualperspective attention. IEEE Geoscience and Remote Sensing Letters , 20:1-5, 2023.

Yizhou Wang, Zhongyu Jiang, Xiangyu Gao, Jenq-Neng Hwang, Guanbin Xing, and Hui Liu. Rodnet: Radar object detection using cross-modal supervision. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages 504-513, 2021.

Yizhou Wang, Zhongyu Jiang, Yudong Li, Jenq-Neng Hwang, Guanbin Xing, and Hui Liu. Rodnet: A real-time radar object detection network cross-supervised by camera-radar fused object 3d localization. IEEE Journal of Selected Topics in Signal Processing , 15(4):954-967, 2021.

F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In ICLR , 2016.

Liwen Zhang, Xinyan Zhang, Youcheng Zhang, Yufei Guo, Yuanpei Chen, Xuhui Huang, and Zhe Ma. Peakconv: Learning peak receptive field for radar semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17577-17586, 2023.

Xizhou Zhu, Han Hu, Stephen Lin, and Jifeng Dai. Deformable convnets v2: More deformable, better results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.

Hao Zou, Zhen Xie, Jiarong Ou, and Yutao Gao. Transrss: Transformer-based radar semantic segmentation. In 2023 IEEE International Conference on Robotics and Automation (ICRA) , pages 6965-6972, 2023.

Supplementary Material

A AdaPKC-based RSS Frameworks

A.1 AdaPKC

We formulate the detailed process of how AdaPKC conducts PeakConv on R † . For the simplicity of notation, we uniformly represent the R † acquired by the two AdaPKCs as R † = { x c , { x ( i ) † r } N i =1 } . R † can be readily incorporated into both PeakConv operators proposed in [32], including vanillaPKC and ReDA-PKC. Given ReDA-PKC's enhanced capability in utilizing interference and target information for noise suppression compared to vanilla-PKC, we choose ReDA-PKC as the baseline operator. Therefore, both AdaPKCs share the same structure as ReDA-PKC and differ only in the generation of R † . Analogous to the formalization of ReDA-PKC, we can formulate AdaPKC ( R † ; W ∈ R C in × N × C out ) : R C in → R C out for each x c as follows,

where, W is the general learning weights of AdaPKCs; V ec ( · ) is the vectorization operator.

A.2 Multi-view RSS Framework

For multi-view RSS tasks on public radar datasets, in alignment with the design of PKCIn-Net [32], we replace the ReDA-PKC in PKCIn-Net with AdaPKC ξ or AdaPKC θ . And this brings two multiview RSS models, denoted as AdaPKC ξ -Net and AdaPKC θ -Net correspondingly, whose overall frameworks are depicted in Fig. S1. The remaining components of both AdaPKC-Nets keep consistent with PKCIn-Net and TMVA-Net [19]: i) the designed networks take RD, RA and AD tensors as inputs and conduct semantic segmentation on both RD and RA views concurrently; ii) each encoding branch for a single view comprises two 3 D convolution blocks, allowing for the comprehensive utilization of temporal information; iii) an ASPP [2] module is used as a component of the encoding branch to aggregate multi-scale spatial information; iv) after the encoding phase, a Latent Space Encoder (LSE) aids in integrating the encoding features from multiple views and transmits them to the decoding section for the final prediction.

A.3 Single-view RSS Framework

For single-view RSS tasks on our self-collected radar dataset, we establish a lightweight baseline model called KuRALS-Net, the architecture of which is illustrated in Fig. S2. This model takes temporally contiguous multi-frame RD tensors as inputs, utilizing an Encoder with 3D convolutions to simultaneously gather temporal and spatial information. Then, the encoding features pass through 2D convolution blocks followed by an ASPP module for further information fusion. Finally, the fused features are fed into the Decoder to obtain pixel-wise semantic predictions in the RD view. For the validation of AdaPKC, we replace the 2D convolution blocks in KuRALS-Net with different AdaPKC blocks.

B Supplementary Details of AdaPKC

B.1 Analysis of Selection Criterion in AdaPKC ξ

We provide a detailed analysis of the selection criterion adopted in Eq. 3. In Eq. 2, the metric score ξ k for the k th PRF is calculated by averaging the metric scores of related reference units { x ( i ) r | k } N i =1 . For reference units mainly occupied by s ′ ∈ S t-n, the corresponding ξ k is statistically smaller than those primarily influenced by s ′ ∈ S t and larger than those controlled by s ′ ∈ S n. When it is reflected on the curve of sort (Ξ) , ξ k is likely to lie near the quickly descending position. In view of this

Max-Pool

1ⅹ1 2D Conv

Concatenation

Multi

-

View

Encoding Flow

Encoding Flow

RD

Encoder

LSE

AdaPKC

Block

RA

Decoder

RD

Decoder

ASPP

AD Encoding

Branch

RA Encoding

Branch

RD Encoding Branch

Multi-frames

RD input

RD output

prediction

RA output

prediction

Multi-frames

AD input

Multi-frames

RA input

Max-Pool

1ⅹ1 2D Conv

Concatenation

RD

Encoder

Conv

Block

RD

Decoder

ASPP

Multi-frames

RD input

RD output

prediction

attribute, we might adopt the median value of Ξ as the selection criterion. However, while the use of median value can locate ξ k where it is desired in some case, it is clear that the maximum value of the first order gradient is more robust for a variety of scenarios, as illustrated in Fig. S3. Therefore, to promote the robustness of this selection, we finally employ the maximum value of the first-order gradient.

B.2 Reference Units Sampling and Back-propagation in AdaPKC θ

The sampling values of reference units in down, left and right directions are obtained by the following forms of linear interpolation, accordingly:

sort(

Ξ

)

sort(

Ξ

)

sort(

Ξ

)

For back-propagation of gradients in these linear interpolations, we compute the gradient of Eq. S1 w.r.t. b G † = { b G † ↑ , b G † ↓ , b G † ← , b G † → } as

Since { x ( i ) † r } N i =1 have been split into four subsets based on their directions, and each x ( i ) † r is solely determined by the guard bandwidth offset in its corresponding direction, there only remain four nonzero items in { ∂ x ( i ) † r ∂ b G † } N i =1 , i.e. , ∂ x ( j ) † r ↑ ∂b G † ↑ , ∂ x ( j ) † r ↓ ∂b G † ↓ , ∂ x ( j ) † r ← ∂b G † ← and ∂ x ( j ) † r → ∂b G † → , where j ∈ [1 , M ] . For simplicity, we demonstrate the calculation procedure with ∂ x ( j ) † r → ∂b G † → as an example, and the remaining items follow a similar approach. From Eq. S4, we can get

With the combination of Eq. S5 and S6, we can obtain the desired gradients.

C Visual Illustration of FiTOS

Load weights

Confidence threshold,

PeakConv

PeakConv

Feature Map

Fixed PRF

Pretraining Flow

Finetuning Flow

AdaPRF

Thresholding

On-line Switch

𝝉𝝉

The detailed process of FiTOS strategy is illustrated in Fig. S4.

D Supplementary Details about Datasets and Training Setups

D.1 Detailed Description of CARRADA-RAC Dataset

The CARRADA-RAC [32] dataset is derived from CARRADA and mainly calibrates the original RA annotations. For the generation of both RD and RA annotations, CARRADA adopts a semiautomatic method relying on the Mean-Shift clustering [3]. However, in CARRADA, the clustering performance is seriously degraded by unreliable centroid initialization from optical images and inaccurate candidate search space in RA representation. To alleviate these issues, CARRADA-RAC proposes an RD association strategy to refine the initial centroid, and introduces a regionalized CFAR for readjusting the search space.

D.2 Implementation Details

Multi-view RSS . The input sizes of RA, AD and RD views are 256 × 256 , 256 × 64 and 256 × 64 , respectively. Both AdaPKC-Nets leverage a sequence of 5 input frames for temporal information aggregation, consistent with PKCIn-Net [32]. We train all these models on two NVIDIA-3090 GPUs and use the Adam optimizer [14] for training. The initial learning rate is 1 e -4 , and decays in a cosine manner by default. We train these models for 300 epochs with a batch size of 6 . For FiTOS, the 300 epochs are evenly distributed between the pre-training and fine-tuning stages, and we set τ = 0 . 6 for the fine-tuning stage by default. We train these models using a combination of weighted cross-entropy loss, Dice loss and coherence loss, configured with the recommended parameters outlined in [19].

Single-view RSS . In contrast to the multi-view RSS task, the experimental configurations for KuRALS dataset maintain consistency with the following two exceptions: i) the input includes only a single view of RD tensor after 0 -Doppler frequency elimination, with shape of 2048 × 124 ; ii) the weighted cross-entropy loss in multi-view RSS is substituted with weighted focal loss [15] to tackle the severer target-background imbalance in KuRALS.

E More Exploration of AdaPKC mechanism

E.1 Analysis of Guard Bandwidth Distribution in AdaPKC ξ -Net

We analyze the guard bandwidth distribution in AdaPKC ξ -Net from two perspectives: different categories and different views. The distribution histogram of guard bandwidths in AdaPKC ξ -Net is illustrated in Fig. S5. From the perspective of different categories , we observe that the guard bandwidth of the background class tends toward a uniform distribution, while that of the foreground class exhibits a more distinctly concentrated trend. For a better illustration, we present the variance values of different categories in Tab. S1, and the remarkable difference in variance between foreground and background classes confirms our observation. This observation aligns with the characteristics of our proposed metric score, indicating that AdaPKC ξ consistently enhances features of foregroundclass targets, thus improving their discriminability from the background class. From the perspective of different views , this concentrated trend appears more apparent in the RA view compared to the RD view. Concretely, RA view shows a clear concentration on the guard bandwidth of (2, 1), while RD view exhibits considerable proportions on all guard bandwidths except (1, 1). Moreover, this concentrated guard bandwidth in the RA view remains consistent across different AdaPKC layers, while that of the RD view transitions from (2, 3) in the first layer to (1, 2) in the second layer. This comparative analysis of different views indicates that the guard bandwidth selection in RA view demonstrates more confidence and ease for AdaPKC, which explains the better performance improvement of AdaPKC on the RA view. 0 0.1

(1, 1)

(1, 2)

(1, 3)

(2, 1)

The Probability Distribution of Guard Bandwidths

RA View

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

(1, 1)

(1, 2)

(1, 3)

(2, 1)

(2, 2)

(2, 3)

Probability

(R, D): bandwidth in the 1st layer of RD view

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

(1, 1)

(1, 2)

(1, 3)

(2, 1)

(2, 2)

(2, 3)

Probability

(R, A): bandwidth in the 1st layer of RA view

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

(1, 1)

(1, 2)

(1, 3)

(2, 1)

(2, 2)

(2, 3)

Probability

(R, D): bandwidth in the 2nd layer of RD view

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

(1, 1)

(1, 2)

(1, 3)

(2, 1)

(2, 2)

(2, 3)

Probability

(R, A): bandwidth in the 2nd layer of RA view

background

pedestrian

cyclist

car

(2, 2)

(2, 3)

E.2 Comparison between AdaPKC and Dynamic CFAR Detectors

We compare the performance of AdaPKC and dynamic CFAR detectors, and the results are shown in Tab. S2. The comparison reveals several significant limitations of CFAR methods: i) They can only detect foreground targets without the ability to categorize them; ii) They show poor target identification performance, struggling with complex target and interference scenarios; iii) They rely on manual parameter tuning and lack adaptive learning capabilities. Therefore, it is evident that enhancing radar target perception paradigms through deep learning is both necessary and practical, serving as one of the key motivations for PKC and AdaPKC.

E.3 Exploration of Adaptive Sampling Strategies for Reference Band

We demonstrate the exploration of adaptive sampling strategies for reference band. Sampling for reference band plays an indispensable role in multiple applications of peak convolutions: i) for PKC with different guard bandwidth settings, sampling fixed N reference points ensures consistent parameter counts; ii) for AdaPKCs, different x c within the feature map might claim diverse PRFs, necessitating reference points sampling to meet the requirement of convolution operations; iii) for future research on reference bandwidth adjustment, this sampling mechanism operates in a manner akin to that in AdaPKCs. For simplicity of exploration, we take the mentioned application in PKC as the subject of our experiment. In addition to the common uniform sampling, we explore two adaptive sampling strategies based on the similar principle of AdaPKC ξ : For the candidates of reference points { x ( i ) r } N r i =1 in a fixed reference band, we first compute their inner product (similarity) with x c , i.e. , x c x ⊤ r . Then, the first version (v1) of adaptive sampling selects the N least similar reference points to locate target-interfering noise, while the second version (v2) of adaptive sampling chooses the N reference points with intermediate similarity instead. To evaluate the effectiveness

of these sampling strategies, we conduct comparative experiments on the CARRADA dataset, with the guard bandwidth of PKC fixed at 2 for all dimensions. The evaluation results are shown in Tab. S3. Compared with uniform sampling, adaptive sampling v1 consistently shows improvement across two views, further affirming the effectiveness of our introduced metric. In the case of adaptive sampling v2, we observe a similar trend as with AdaPKC ξ , which exhibits a more pronounced performance enhancement for RA view. Nevertheless, in comparison to the adaptive guard bandwidth adjustment in AdaPKCs, these adaptive samplings of reference band encompass a more limited range of adjustable receptive field, thus demonstrating less improvements on RSS performance.

E.4 Exploration of Enlarging PRF Search Space in AdaPKC ξ

To achieve AdaPRF estimation in a wider range of PRF search space under fixed resource constraints, we propose a novel training strategy named Voting-driven Multi-round Training (Vot-MRT) . Due to limitations in GPU memory resources, AdaPKC ξ restricts the selection of AdaPRF within a local range. To determine the AdaPRF in a wider range of PRF search space, Vot-MRT employs a step-wise local optimization strategy, inspired by greedy algorithms [4]. Concretely, Vot-MRT involves multiple rounds of training, and each successive round provides a better initialization for the guard bandwidth search space, Ω G , of the latter round. Given that different pixels may pick distinct guard bandwidths, Vot-MRT implements a voting mechanism to gather the most concentrated guard bandwidth for the subsequent round. The detailed algorithm of Vot-MRT strategy is presented in Algo. 1.

To evaluate the effectiveness of Vot-MRT , we conduct a quantitative experiment on CARRADA dataset. We keep Ω G 0 consistent with Ω G denoted in § 3.1.1, and set the value of N to 3 by default, indicating that the original training process of AdaPKC ξ -Net will be repeated for three rounds in Vot-MRT . The results are shown in Tab. S4 and it can be observed that Vot-MRT brings visible segmentation performance improvements to AdaPKC ξ -Net by gradually enlarging the search space of alternative PRFs.

F Supplementary Experiment Results

F.1 Semantic Segmentation Results by Category on CARRADA

The category-wise semantic segmentation results on the CARRADA-Test dataset are shown in Tab. S5. Compared with TMVA-Net, PKCIn-Net employs PeakConv to replace the regular convolution, resulting in impressive performance improvements. Our proposed methods further enhance the capabilities of PeakConv, leading to significant enhancements across nearly all classes and views, with a minor decrease observed in the car class of RA view. Consequently, the proposed methods achieve superior RSS performance and a better trade-off among different classes, which still holds true when compared to other SoTA RSS models.

F.2 Performance vs. Complexity

The trade-off between performance and complexity of different RSS models is presented in Fig. S6.

F.3 Qualitative Comparisons on CARRADA

We present in Fig. S7 the qualitative comparisons of different methods on three frames from the CARRADA test split, with each frame exhibiting varying levels of interference with targets. In the

first frame, where the target is interfered with minor noise/clutter, results indicate that all methods can accurately locate and classify targets in relatively clean RD view. However, in the RA view, methods without peak convolution struggle to identify target regions completely in the presence of stronger interference. The interference suppression capability equips PKCIn-Net with superior identification performance, while AdaPKC ξ -Net FiT further strengthens this capability and recognizes complete target regions. In the second frame, as clutter interference on targets intensifies, PKCIn-Net can still accurately locate targets in both RD and RA views but faces challenges in differentiating target categories in the highly cluttered RA view. Nevertheless, AdaPKC ξ -Net FiT correctly classifies targets in both views. In the third frame, where the signal of the distant car target is weak and the clutter interference is strong, existing methods miss the car target whereas AdaPKC ξ -Net FiT can still identify it in the RD view. These results suggest that AdaPKC ξ -Net FiT exhibits stronger interference suppression and target recognition capabilities compared to PKCIn-Net and other RSS methods.

F.4 Semantic Segmentation Results by Category on KuRALS

Considering the limited samples of both ship and land vehicle classes, we merge the two classes into a single vehicle class to mitigate the negative impact of class-imbalance on model training. This class merging is primarily based on two observations: i) both ships and land vehicles belong to rigid transportation vehicles, whose reflected radar signals share considerable similarities; ii) ships and land vehicles are primarily used for marine monitoring tasks and land detection tasks, respectively, and there exist nearly no overlapping application scenarios between them, which minimizes the risk

mDice (%)

25

30

35

40

45

50

55

60

0

50

100

150

mDice (%)

25

27

29

31

33

35

37

39

41

43

45

0

50

100

150

mIoU (%)

35

40

45

50

55

60

65

70

75

80

0

50

100

150

mDice (%)

30

35

40

45

50

55

60

65

0

20

40

60

80

100

120

140

mIoU (%)

FCN

U-Net

DeepLabv3+

RSS-Net

RAMP-CNN

TMVA-Net

TransRadar

T-RODNet

PKCIn-Net

TAPKCIn-Net (Ours)

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

-𝐍𝐍𝐍𝐍𝐍𝐍

𝑭𝑭𝑭𝑭𝑭𝑭

(

𝐎𝐎𝐎𝐎𝐎𝐎𝐎𝐎

)

80

75

70

65

60

55

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

(a)

(b)

(c)

(d)

(e)

(f)

of practical application issues due to recognition confusion. As a result, we perform the segmentation task with four categories: background , UAV , pedestrian and vehicle . The semantic segmentation results by category on the KuRALS-Test dataset are presented in Tab. S6.

F.5 Qualitative Comparison on KuRALS

The qualitative results on KuRALS dataset are illustrated in Fig. S8. Compared to our baseline methods and other segmentation models, our AdaPKCs demonstrate more accurate localization and classification of targets.

(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

73.0%

F.6 More Results in Ablation Study for FiTOS 72.5%

Fig. S9 illustrates the segmentation results of mIoU for AdaPKC ξ -Net FiT with different values of τ ∈ { 0 . 1 , 0 . 2 , 0 . 3 , 0 . 4 , 0 . 5 , 0 . 6 , 0 . 7 , 0 . 8 , 0 . 9 } . 72.0% 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

60.5%

61.0%

61.5%

62.0%

62.5%

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

mIoU

43.0%

43.5%

44.0%

44.5%

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

mIoU

RA View

TAPKCIn-Net

TAPKCIn-Net

PKCIn-Net

-𝐍𝐍𝐍𝐍𝐍𝐍

𝑭𝑭𝑭𝑭𝑭𝑭

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀𝐀

𝝃𝝃

-Net

PKCIn-Net

𝜏𝜏

𝜏𝜏

G Broader Discussions

Limitations . We have collected a Ku-band continuous wave radar dataset to validate the effectiveness of our proposed method in surveillance radar detection scenarios. Nevertheless, pulse-Doppler radar is also commonly used in these scenarios and investigation on such radar datasets would further uncover the potentials and issues of our methods in practical applications. However, there is currently a lack of publicly available datasets for pulse-Doppler monitoring radar. To alleviate this issue, we will try to collect and release a pulse-Doppler radar dataset to enable more comprehensive validation and analysis of our methods and other works.

Societal Impacts . Our approach is applicable to various practical applications, such as perceptions for autonomous driving, UAV surveillance and marine monitoring. However, inappropriate usage may lead to decreased reliability, potentially resulting in safety and other issues.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: This paper discusses the limitations of our work in Section G.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: For each theoretical result, this paper provides the full set of assumptions and a complete (and correct) proof.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: This paper fully discloses all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: In abstract, this paper provides an link to the data and code, with sufficient instructions to faithfully reproduce the main experimental results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: This paper specifies all the training and test details in Section 4.1 and Section D.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: This paper reports error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: This paper provides sufficient information on the computer resources needed to reproduce the experiments in Section D.2.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: The paper discuss the potential societal impacts of our work in Section G.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: This paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The creators or original owners of assets, used in the paper, are properly credited and the license and terms of use are explicitly mentioned and properly respected.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: New assets introduced in the paper are well documented and the documentation is provided alongside the assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects. Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe PeakConv (PKC) model specialized for radar signal analysis effectively characterizes the target signatures of radar signals. However, the fixed predefined peak receptive field limits the performance of PKC due to significant variations in target features and associated interference within radar signals. To solve this problem, the authors propose AdaPKC, aimed at adaptively adjusting the peak receptive fields in PKC through two data-adaptive band-pass filtering mechanisms. The experimental results show that the proposed method is advanced to some extent.\n\nSTRENGTHS:\nOriginality: This paper introduces an adaptive method for adjusting the PRF based on an existing baseline, demonstrating some innovation.\nQuality: The paper is technically sound. The proposed dataset and the results of the proposed model are analyzed in detail.\nClarity: Overall, the paper is clearly written, although some sections require clearer articulation.\nSignificance: The general applicability of the proposed method needs further validation.\n\nWEAKNESSES:\n1.The explanation of Figure 1 is unclear, and the captions for Figures 2 and 3 do not sufficiently summarize the contents depicted, resulting in poor readability.\n2.The experiments lack an evaluation of the model's computational efficiency\n3.The effectiveness of the proposed method requires further validation.\n\nQUESTIONS:\n1.Please provide a clearer explanation of the content presented in Figure 1. Which part of the Figure 1(a) does the red ellipse in the first row of radar frequency map represent? Why does it disappear in the second radar frequency map of the first row? Why does the purple ellipse reappear in the second row? What do the corresponding image frames look like, and how are they related to these radar frequency maps? Please also supplement the captions for Figures 2 and 3 to facilitate reader comprehension.\n2.In Section 3.1.1, \"...we divide the observed radar signals into three subsets...\". How are these subsets divided? How are the expressions for each segment in Equation 1 derived?\n3.Please add an analysis of the computational efficiency of the model. How much computational cost is added by the two modules proposed?\n4.Dilated convolution can also adaptively adjust the receptive field of the convolution kernel. What are the advantages of your method compared to dilated convolution techniques?\n5. Would the method proposed in this paper still work if other baselines similar to PKC was used?\n\nLIMITATIONS:\nSee Weakness and Questions.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-05"}

comments: SUMMARY:
This paper presents a radar semantic segmentation method, AdaPKC, which combines PeakConv and Adaptive Peak Receptive Field (APRF) concepts. The author demonstrates extensive applicability of AdaPKC in radar perception including autonomous driving, drone surveillance, and ocean monitoring. The method significantly enhances radar semantic segmentation performance with robustness and scalability.

STRENGTHS:
1.The method's performance and efficiency have been demonstrated through experiments to exceed both a strong baseline in radar semantic segmentation.
2.The author successfully achieved incremental optimization in PeakConv [1], reaching state-of-the-art performance.
3.Rigorous ablation studies were conducted, providing solid evidence of the proposed method's efficacy.
4.The paper includes a good review of existing work and contributes to the development of radar semantic segmentation.

WEAKNESSES:
This method builds upon existing methods and is an improved version of the existing PeakConv (PKC). It combines them in a novel way. The novelty is limited for NeurIPS.

QUESTIONS:
1.Can you provide a more detailed analysis of the computational complexity?
2.Can you provide a more detailed analysis of the impact of the proposed AdaPKC on real-time performance?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

comments: SUMMARY:
This paper proposes an idea of adaptive peak receptive field, and upgrades PKC to AdaPKC based on this idea. Beyond that, a novel fine-tuning technology to further boost the performance of AdaPKC-based RSS networks is presented.

STRENGTHS:
The adaptive version of PeakConv (PKC) is motivated by the adaptive selection of reference cells in the classical radar detector, CFAR.

WEAKNESSES:
The adaptive version of PeakConv (PKC) is considered to be incremental work. The numerical results also indicated the AdaPKC performance improvement over PKC is very limited. 

The comparison with classical CFAR with adaptive reference cell selection is not included in the validation part.

QUESTIONS:
It is not clear how many annotated radar data is sufficient to train the proposed network. 
What is complexity of the proposed network? Is it running faster than CFAR with adaptive reference cell selection?

LIMITATIONS:
Limitation of the proposal is not discussed in the submission.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 2
DATE: 2024-07-13

comments: SUMMARY:
This paper works on the improvement of Radar semantic segmentation. Motivated by the limitation of learning ability of the SoTA PKC method due to the fixed peak receptive field (PRF), an adaptive version named AdaPKC is proposed.  The method can be metric-based and learning-based.  The advantages of the proposed method are validated by the extensive experiments  on two datasets.

STRENGTHS:
- The motivation of the study is practical, and the paper is well-written and easy to follow. The figures are well-illustrated .
- The experiments, visualization and analysis are extensive.
- The proposed AdaPKCs outperform the state-of-the-art baselines.

WEAKNESSES:
- The novelty of the proposed method is limited, considering the existing PKC work.
- Although the method is practical and lightweight, the performance gain is not significant.
- I also have concerns about the practicality of the problem setting for radar semantic segmentation in this work. It appears to be more similar to the object detection task found in other datasets. Additionally, other modalities such as cameras or LiDAR are usually available and can provide complementary information even in adverse weather conditions. Moreover, scanning Radar and 4D Radar sensors (such as ORR, Radiate, and K-Radar datasets), which offer much higher resolution, are also becoming increasingly popular.

QUESTIONS:
- How are the frame rates for $AdaPKC^θ$, $AdaPKC^ξ$ and PKC?
- Is any relationship between the guard bandwidth and the specifics of the Radar sensor?

LIMITATIONS:
The limitation of the evaluation of the method for pulse-Doppler radar is discussed.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-16