Searching Parameterized AP Loss for Object Detection
Year: 2022

Abstract:
Loss functions play an important role in training deep-network-based object detectors. The most widely used evaluation metric for object detection is Average Precision (AP), which captures the performance of localization and classification sub-tasks simultaneously. However, due to the non-differentiable nature of the AP metric, traditional object detectors adopt separate differentiable losses for the two sub-tasks. Such a mis-alignment issue may well lead to performance degradation. To address this, existing works seek to design surrogate losses for the AP metric manually, which requires expertise and may still be sub-optimal. In this paper, we propose Parameterized AP Loss, where parameterized functions are introduced to substitute the non-differentiable components in the AP calculation. Different AP approximations are thus represented by a family of parameterized functions in a unified formula. Automatic parameter search algorithm is then employed to search for the optimal parameters. Extensive experiments on the COCO benchmark with three different object detectors (i.e., RetinaNet, Faster R-CNN, and Deformable DETR) demonstrate that the proposed Parameterized AP Loss consistently outperforms existing handcrafted losses. Code shall be released.

heading: 1 Introduction
text: The past decade has witnessed the significant success of deep neural networks in object detection, in which loss functions play an indispensable role in training networks. To evaluate the object detection methods, the Average Precision (AP) metric is usually used, which captures the performance of localization and classification simultaneously. However, as in most object detectors [26], the training of localization and classification sub-tasks are driven by two separate losses (see Figure 1). For example, the L1/smooth-L1 [11] or GIoU [38] losses are usually employed for localization, while the cross-entropy or Focal [25] losses are usually used for classification. Such a mis-alignment between network training and evaluation may well lead to performance degradation.
To mitigate this mis-alignment issue, a straight-forward solution is to approximate the AP metric in network training. Because the AP metric is non-differentiable, many works [3, 5, 13, 40, 35] have explored hand-crafted losses based on the mathematical formula of the AP metric. [3] replaces the non-differentiable parts in the AP metric with hand-crafted differentiable approximations. The loss gradient is obtained by taking derivation with respect to the hand-crafted function. However, as both [5] and our experiments show, such a hand-crafted AP approximated loss produces lower
∗Equal contribution. †This work is done when Chenxin Tao and Zizhang Li are interns at SenseTime Research. ‡Corresponding author.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
performance than the commonly used cross-entropy and L1/smooth-L1 losses. Another line of works try to estimate the gradient for the AP metric directly [5, 13, 40, 35]. These works try to manually design the loss gradient. Therein, the AP approximated gradient only drives the training of the classification branch, while the training of the localization branch is still supervised by traditional regression losses. In practice, these methods still do not address the mis-alignment issue well.
The common issue of existing approaches is they do not optimize over the numerous approximations of the non-differentiable discrete AP metric. The AP metric itself is a piecewise constant function, whose differentiable approximations have infinite possibilities. The hand-crafted approximations / gradients may well be sub-optimal for driving network training. Instead of manually determining the surrogate loss form, we propose to approximate the non-differentiable parts with a family of continuous parameterized functions, which helps to represent the numerous AP approximations in a unified formula. Then, an efficient parameter search procedure is employed to find out the desired loss function, so as to optimize the trained object detector’s performance on the evaluation set with the AP metric. Because the parameterized AP approximations constitute a compact search space, the search process would be very effective.
To this end, we propose the Parameterized AP Loss, which is built on top of the AP metric to mitigate the mis-alignment between network training and evaluation. It utilizes parameterization to construct the search space so that the optimal parameters can be searched automatically. Specifically, we first explicitly reformulate the AP metric for object detection as a function of classification scores and box coordinates. Then we replace the non-differentiable components in the reformulated function with parameterized functions. Finally, to obtain the optimal loss function for network training, we search the parameters through a reinforcement-learning-based search process, which aims to maximize the AP score on the evaluation set.
We evaluate the searched Parameterized AP Loss on various object detectors, including RetinaNet [25], Faster R-CNN [37] and Deformable DETR [42]. Extensive experiments on the COCO benchmark [23] demonstrate that the proposed Parameterized AP Loss consistently outperforms existing delicately designed losses.
The main contributions of our work can be summarized as:
• By reformulating the AP metric and introducing differentiable parameterized substitutions, Parameterized AP Loss can represent numerous AP approximations in a unified formula, which captures classification and localization sub-tasks simultaneously in a single loss function.
• Instead of hand-crafting AP losses or gradient approximations, our approach automatically searches for the optimal parameter, optimizing for the trained object detector performance.
• Extensive experiments on different object detectors demonstrate that the searched Parameterized AP Loss consistently outperforms the existing losses.

heading: 2 Related work
text: Hand-crafted Loss Functions for Object Detection. Designing loss functions has been an active direction in the field of object detection for long. Cross-entropy and smooth-L1 losses are widely used for the classification and localization sub-tasks, respectively [26]. For the classification subtask, to mitigate the imbalance problem in one-stage detectors, Focal Loss [25] and GHM [17] propose to adjust the weight of loss or gradient for each predicted box. DR Loss [34] proposes to convert the classification problem into a ranking problem. For the localization sub-task, a series of
works [38, 41] introduce IoU variants for better localization. GFL [22] and GFLv2 [21] point out that the classification and localization branches are separately trained but compositely used in inference, so they merge the localization representation into the classification branch for a joint optimization. But such combination is still not consistent with the AP metric.
Recent works [5, 30] have noticed the mis-alignment issue between network training and evaluation, and try to address it with specifically designed surrogate loss functions. AP Loss [5] replaces the classification task with a ranking task derived from the AP metric via a hand-crafted errordriven gradient. aLRP Loss [30] extends the framework of AP Loss, and unifies classification and localization losses under the LRP metric [29].
These loss functions are all hand-crafted, and may well be sub-optimal in guiding network training. In contrast, our proposed Parameterized AP Loss is automatically searched, which can better align the network training and evaluation.
Direct Optimization for the AP Metric. Pioneer works [13, 40, 35] have also tried to use the AP metric as training objective. Due to the non-differentiable nature of the AP metric, these works adopt different methods to estimate the back-propagated gradient. [13] uses finite difference and linear envelope to derive the pseudo-gradients. [40] applies the loss-augmented inference for the gradient estimation. [35] resorts to reinforcement learning for policy gradient.
While these works take a step towards direct optimization for the AP metric, they all focus on the gradient for the classification task, ignoring the localization task, which may well be sub-optimal for the optimization of AP metric. By contrast, our proposed Parameterized AP Loss deals with these two tasks simultaneously in a single loss function, and thus better align training and evaluation processes.
Another line of works try to approximate the AP metric via interpolation or neural network[27, 32, 8]. [27] refactors the computation process and interpolates the loss value with differentiable functions. [32] learns an embedding for prediction and target, so the Euclidean distance between them approximates the metric value. [8] trains a network for sorting operation, so as to construct a differentiable surrogate. Although these methods use differentiable functions to approximate AP metric, they ignore the training process. Our proposed Parameterized AP Loss, however, is searched to directly optimize the AP metric during evaluation, and thus consider the training behavior of loss function. This leads to a more effective loss function for network training.
Searching Loss Functions for Object Detection. Recent works [28, 19] have also tried to search suitable loss functions for object detection. In these methods, loss functions are formulated as computational graphs composed of basic mathematical operators. Since the combinations of operators are randomly chosen, the search space is very sparse with a large number of unpromising loss functions. Thus, they have to design specific techniques to accelerate the search process. [28] also relies on hand-crafted initialization. Nevertheless, their searched loss functions only gain marginal improvement on detection tasks. On the other hand, these methods search for separate losses for the localization and classification sub-tasks, thus the mis-alignment between network training and evaluation still exists.
Our proposed Parameterized AP Loss constitutes a compact search space, where different loss functions are represented by different parameters. Since the parameterized search space is continuous, the search process would be more effective than the discrete combinatorial optimization in [28, 19]. Moreover, Parameterized AP Loss drives the localization and classification training with a single unified loss, which better mitigates the mis-alignment issue.
Hyper-Parameter Optimization. Previously, grid search or random search [1] are commonly used for hyper-parameter tuning. As the search space becomes more complex, many efficient hyperparameter optimization methods have been proposed. Bayesian optimization [2, 14] aims to build a prediction model from history data and use it to select the most promising hyper-parameters for evaluation. Bandit-based methods [15, 20, 10] view it as a resource allocation problem and allocate most of the computational resource to the most promising hyper-parameters. Evolutionary algorithms [36, 33] evolve the optimal hyper-parameters from a population of models. Reinforcement learning techniques [43] have also been used to explore the search space via sampling candidates and adjusting the sampling policy.
The introduced parameters in our proposed Parameterized AP Loss are indeed hyper-parameters, which fit the hyper-parameter optimization methods. In this work, we adopt reinforcement learning
to search for the optimal parameters because of its simplicity and efficiency. Other efficient hyperparameter optimization algorithms can also be employed.

heading: 3 Method
text: In this section, we present how Parameterized AP Loss is constructed to better align the training target with the evaluation metric. The overview of our method is illustrated in Figure 2.

heading: 3.1 Revisiting AP Metric
text: AP metric is the most widely used evaluation metric for object detection. Given an image, we assume an object detector outputs N detected bounding boxes for each category as B = {(bi, si)}Ni=1. Here, bi ∈ R4 denotes the box coordinates of the i-th prediction, and si ∈ R is its corresponding classification score. In the AP metric, these predictions will be matched with a set of ground-truth bounding boxes G. Each prediction will be assigned with zero or one ground-truth bounding box. Those predictions assigned to ground-truth bounding boxes constitute the positive set P , while other predictions form the negative set N . Then, the AP metric score is defined as the area under the precision-recall curve. Following the above notations, it can be written as
AP = 1 |P| ∑ i∈P p(i), (1)
where p(i) denotes the precision of the predictions, whose classification scores are higher than that of the i-th prediction. Following [5, 3], Eq. (1) can be explicitly calculated as the function of classification scores
AP({si}Ni=1) = 1 |P| ∑ i∈P 1− rank –(si) rank(si) = 1 |P| ∑ i∈P 1− ∑ j∈N H(sj − si) 1 + ∑ j∈B,j 6=iH(sj − si) , (2)
where rank–(si) and rank(si) denote the ranks of the classification score si among the negative prediction set N and the whole prediction set B, respectively. H(·) is the Heaviside step function,
H(x) = { 1, if x > 0, 0, otherwise.
(3)
Eq. (2) explicitly includes the classification scores into the AP calculation, which helps previous works [5, 3] to explore surrogate losses for the classification task. However, the localization task is also very important for object detection, which is just implicitly contained in Eq. (2).

heading: 3.2 Parameterized AP Loss
text: Instead of hand-crafting AP approximations for training networks, we propose to represent the numerous potential AP approximations in a unified parameterized formula, which is denoted as the Parameterized AP Loss. Specifically, we first explicitly reformulate the AP metric as a function of classification scores and box coordinates {(bi, si)}Ni=1 output by the detector. Then, the nondifferentiable components in the reformulated function are substituted with parameterized functions, so as to extend the AP metric to differentiable parameterized approximations.
Reformulating AP metric as Function of Classification Scores and Box Coordinates. Since Eq. (2) is only related to classification scores explicitly, it needs to be further extended as the function of box coordinates. In Eq. (2), the only part related to the localization task for object detection is the summation range (i.e., the positive set P and the negative set N ), which motivates us to replace the summation range by explicitly including the localization results.
To this end, we define the localization score for the i-th prediction as
l(bi) = { IoU(bi, bi∗), i ∈ P, 0, i ∈ N , (4)
where i∗ denotes the assigned ground-truth bounding box to the i-th prediction, and IoU(bi, bi∗) calculates the area overlap ratio between these two boxes. Here, box IoU is used to measure the
localization results, which follows the AP metric. Other measurements, such as GIoU [38] and L1 are also applicable. Results in Section 4.3 demonstrate that GIoU actually delivers the best results.
By further replacing the summation range in Eq. (2) using Eq. (4), the AP metric can be explicitly formulated as the function of classification scores and box coordinates:
AP({(bi, si)}Ni=1) = 1 |P| ∑ i∈B H ( l(bi) ) − ∑ j∈B,j 6=iH(sj − si) ( 1−H ( l(bj) )) 1 + ∑ j∈B,j 6=iH(sj − si) H ( l(bi) ) . (5)
Here, H ( l(bi) ) is the Heaviside step function with the localization score as input, which indicates whether the i-th prediction belongs to the positive set. We multiply each summation term with the appropriate indicator function to replace the summation range.
Extending AP metric to Differentiable Parameterized Approximations. Designing approximated loss functions for the AP metric may well mitigate the mis-alignment between network training and evaluation. We need to replace the non-differentiable Heaviside step function H(·) in Eq. (5) with its differentiable substitutions. Previous works [5, 30] have tried to address this problem with a specially designed error-driven update technique. Nevertheless, these hand-crafted methods are all based on heuristics and expertise, which may not be optimal to guide the training process.
In contrast to hand-crafting differentiable surrogates for the AP metric, we substitute the nondifferentiable components with parameterized functions, which helps to represent different AP approximations in a unified formula. To achieve this, we replace the non-differentiable Heaviside step function H(·) in Eq. (5) with parameterized functions f(·; θ) parameterized by θ. Then, the proposed Parameterized AP Loss can be obtained as
L = − 1 |P| ∑ i∈B f(l(bi); θ1)− ∑ j∈B,j 6=i f(sj − si; θ2)(1− f(l(bj); θ3)) 1 + ∑ j∈B,j 6=i f(sj − si; θ4) f(l(bi); θ5). (6)
Note that we normalize the input of f(x; θ) to x ∈ [0, 1] for a unified domain (see Section 4.1 for implementation details). The output range is also restricted to f(x; θ) ∈ [0, 1], which follows the value range of H(·). We adopt different parameters {θi}5i=1 in Eq. (6) to substitute different H(·) functions in Eq. (5), which makes the Parameterized AP Loss more flexible. Our experiments in Section 4.3 also demonstrate that such parameterization delivers better performance than that of employing shared parameters for different H(·) functions. We also block the gradient of f(sj − si; θ4) on the denominator, and empirically find such modification brings more stable training (see Appendix A.1).
Piecewise Linear Function. The parameterized function f(x; θ) can be of any family of differentiable functions. Here, we adopt the piecewise linear function for simplicity. The piecewise linear function is composed of linear segments over different input ranges, where the first point of one
segment coincides with the last point of the previous segment. Assuming a piecewise linear function f(x; θ) has M segments, the k-th segment is defined as
fk(x; θ) = yk+1 − yk xk+1 − xk · (x− xk) + yk, xk ≤ x < xk+1, k = 0, . . . ,M − 1. (7)
The k-th segment is controlled by two points (xk, yk) and (xk+1, yk+1). The coordinates of these control points build up the set of parameters θ.
Following [18], we apply the end-point constraint and monotonicity constraint to regularize the search space. The end-point constraint requires that the end points of the parameterized function should take the same value with the original function, while the monotonicity constraint requires the parameterized function to be monotonically increasing. These constraints can be easily applied via
x0 = 0, y0 = 0, xM = 1, yM = 1; (end-point constraint) 0 < xk − xk−1 xM − xk−1 < 1, 0 < yk − yk−1 yM − yk−1 < 1, k = 1, . . . ,M − 1. (montonicity constraint)
To apply the above restrictions in optimization, the specific form of the parameters is defined as
θ = {( xk − xk−1 xM − xk−1 , yk − yk−1 yM − yk−1 ) , k = 1, . . . ,M − 1 } . (8)
Such parameterization also makes each parameter independent, which simplifies the search.
Gradient Scale for Localization Branch. In object detection, to balance the training of the localization and classification branches, traditional detectors [31] usually add a loss weight to the localization branch. While the Parameterized AP Loss is a unified loss, the network itself still preserves independent branches for these two tasks. In order to balance the training weights, we multiply the gradient back-propagated through the localization branch with a gradient scale λ > 0 (see Figure 1), which serves similar effect with the traditional loss weight.
In practice, the actual searched parameter is θλ = 12 (log10 λ + 1). Such search parameterization form simplifies the search for small and large gradient scale λ values. We also empirically restrict the search range for θλ as (0, 1), i.e., λ ∈ (0.1, 10). The collection of the searched parameter for the gradient scale θλ and other parameters in the Parameterized AP Loss {θi}5i=1 are denoted as Θ, which could be optimized by efficient parameter search methods.

heading: 3.3 Searching for Optimal Parameters
text: The optimal parameter set Θ for the Parameterized AP Loss is found by optimizing the network performance on the validation set with the AP metric. The massive parameter space of Θ makes it impractical to determine the desired parameters manually. Efficient automatic search is thus necessary to find the optimal parameters.
Our search process is described in Algorithm 1. Specifically, we divide the training set into two subsets in the search process, Dtrain for training and Deval for evaluation, respectively. The whole search process is treated as a bi-level optimization problem, which can be formulated as
maxΘ R(Θ) = AP(ω ∗(Θ);Deval)
s.t. ω∗(Θ) = minω LΘ(ω;Dtrain), (9)
where LΘ is the Parameterized AP Loss with the loss parameter set Θ, ω stands for the network weights, and ω∗(Θ) indicates the trained network weights with the given loss parameterized with Θ. AP(ω;D) evaluates the AP metric for the network with weights ω on the given dataset D. To optimize Eq. (9), we optimize the inner level and outer level problem iteratively. At the inner level, we train a detector network on Dtrain with the certain loss parameter set Θ for one epoch. At the outer level, following [43, 18], we adopt reinforcement learning, a commonly used hyper-parameter optimization algorithm, to search for the optimal parameters. Other efficient search methods like evolutionary algorithm may also be applied. Specifically, we adopt the PPO2 [39] algorithm. Here, we consider a search process consisting of T rounds. In the t-th round, we sample S parameter set
Algorithm 1: Parameterized AP Loss Search Process Input: Network initial weights ω0, initial parameter set distribution (µ1, σ2I), training dataset Dtrain and evaluation dataset Deval for the proxy task Output: Searched optimal parameter set Θ∗ for t← 1 to T do
for i← 1 to S do Sample parameter set Θti ∼ Ntrunc[0,1](µt, σ2I); Inner-level network training with the initial network weights ω0, ω∗(Θti) = min LΘ(ω;Dtrain);
Evaluate the AP metric for the trained network as the corresponding reward, R(Θti) = AP(ω
∗(Θti);Deval); end Outer-level distribution update, µt+1 = argmax
µ
1 S ∑S i=1 R̃ t i(µ, µt,Θi);
end return Θ∗ = argmaxΘR(Θti), ∀t = 1, . . . , T, i = 1, . . . , S
samples {Θti}Si=1 independently from a truncated normal distribution [4] Ntrunc[0,1](µt, σ2I), where µt and σ2 are the mean and variance values, respectively. The truncated range of the distribution is set to [0, 1] so as to satisfy the monotonicity constraint of independent parameters in Eq. (8). For the i-th sample, the AP metric score of the network trained in the inner level is regarded as its reward Rti = R(Θ t i). In the PPO2 algorithm, the mean value of the truncated normal distribution for the next (t+ 1)-th round is updated as
µt+1 = argmax µ
1
S S∑ i=1 R̃ti(µ, µt,Θ t i). (10)
Here R̃ti(µ, µt,Θi) is calculated from the original reward R t i of each sample as
R̃ti(µ, µt,Θi) = min
( p(Θti;µ, σ 2I)
p(Θti;µt, σ 2I)
Rti,CLIP
( p(Θti;µ, σ 2I)
p(Θti;µt, σ 2I)
; 1− , 1 + ) Rti ) , (11)
where min(·, ·) outputs the smaller value between two inputs, p(Θti;µ, σ2I) denotes the PDF of given truncated normal distribution, and the CLIP function clips input value within 1− and 1 + as stated in PPO2 [39] for more stable and effective search. Note that the mean reward in the t-th round is subtracted from each Rti for better convergence. More implementation details are described in Section 4.1.

heading: 4 Experiments
text: We evaluate our approach on the COCO 2017 object detection benchmark2 [23] with various object detectors, including RetinaNet [25], Faster R-CNN [37] and Deformable DETR [42]. Note that for Faster R-CNN, we search the losses for both the RPN and R-CNN heads simultaneously.

heading: 4.1 Implementation Details
text: Loss Function Calculation. As described in Section 3.2, the inputs to the parameterized functions f(x; θ) are normalized to range [0, 1] for a unified domain. For classification score differences sj − si, because their original values are unbounded, we first clip them within the range of [−1, 1] following [5, 30]. The clipped values are further mapped to the range of [0, 1] via min-max re-scaling. For the localization scores l(bi), GIoU [38] is used as the measurement by default due to its good performance. The localization scores are also mapped to the range of [0, 1] via min-max re-scaling. The default value of number of segments M is fixed as 5. In experiments, the whole prediction set B in Eq. (6) consists of all predicted boxes in the current mini-batch for all categories. The positive
2COCO 2017 is publicly available under the Creative Commons Attribution 4.0 License. As far as we know, it does not contain any personally identifiable information or offensive content.
prediction set P and the negative prediction setN are determined by the original pre-defined training target assignment of each object detector.
Search Settings. In COCO 2017 [23], there are 118k images in the train subset. As described in Section 3.3, we randomly divide the original train subset into Dtrain and Deval, which constitute 113k training images and 5k evaluation images for the proxy task, respectively. In the inner-level network training of Algorithm 1, we train the object detectors for one epoch. For Deformable DETR [42], we also set the number of feature levels to 1 to further accelerate the training. In the outer-level PPO2 [39] updating of Algorithm 1, we sample S = 8 samples each round, and search for T = 40 rounds in total. The mean vector µ1 of the truncated normal distribution is initialized to make f(x; θ) = x. The standard deviation σ is initialized as 0.2, which decays linearly to 0 with respect to the search round. The clip operation in Eq. (11) is applied on each component value in Θ independently. The clip range is set to 0.1 following PPO2 [39]. To solve Eq. (10), we employ the Adam optimizer [16] for 100 iterations with a base learning rate of 0.01. In the first 30 iterations, linear warm-up from a learning rate of 0 is utilized.
Training Settings. After the parameter search, we re-train the object detectors with the searched Parameterized AP Loss on the COCO 2017 train subset, and evaluate them on the val subset, which consists of 5k images. For RetinaNet [25] and Faster R-CNN [37], ImageNet [7] pre-trained ResNet50 [12] with FPN [24] is utilized as the backbone, and the training settings strictly follow [5, 30]. For Faster R-CNN, we use a base learning rate of 0.024 and a batch size of 64 (8 images on each GPU). For RetinaNet, the base learning rate is set to 0.016 and the batch size is 64 (8 images on each GPU). For Deformable DETR, ImageNet [7] pre-trained ResNet-50 [12] is utilized as the backbone, and the training settings strictly follow [42], where the learning rate is set to 0.0002 and the batch size is 32 (4 images on each GPU). All the experiments are conducted on 8 NVIDIA V100 GPUs. Our method is implemented based on the open-sourced MMDetection codebase3 [6].

heading: 4.2 Main Results
text: Table 1 summarizes the performance of our approach and the existing losses on COCO 2017 [23]. Among these existing losses, Cross Entropy and Focal Loss [25] are commonly used for classification, while L1 and GIoU [38] are commonly used for localization. AP Loss [5] is a classification loss handcrafted for error-driven optimization of the AP metric, while aLRP Loss [30] is an error-driven method specifically designed for the LRP metric [29]. Compared with these handcrafted losses, Parameterized AP Loss can yield 1.5 ∼ 3.0 AP score gain. We also compared with AutoLossZero [19] and CSE AutoLoss [28], which are two AutoML-based losses. Our approach can obtain over 1.5 AP score improvement over them. We also seek to compare with the direct optimization methods [13, 40, 35], but their codes are not released and their original training settings differ too much from ours, which makes them difficult to be compared with. The searched parameterized functions f(x; θ) of our approach are demonstrated in Appendix A.1.
3MMDetection is an open-sourced codebase under the Apache-2.0 License.

heading: 4.3 Ablation Study
text: In this section, we ablate different design choices and the search effectiveness. RetinaNet [25] is employed as the baseline model in the following experiments.
Comparing Searched and Hand-crafted Substitutions for H(·). Here, we compare our searched parameterized function f(·; θ) with several handcrafted differentiable substitutions, including sigmoid, sqrt, linear, and square functions. Table 2 shows that the searched parameterized function demonstrates significant improvement. The low performance of hand-crafted substitutions indicates that it is non-trivial to hand-craft appropriate differentiable approximations. As analyzed in [5], the sigmoid substitution forH(·) actually may lead to non-convergence of the network training. Note that the linear function is also the initialization of our searched parameterized function, which indicates the effectiveness of the search process.
Comparing Separate and Shared Parameters for Different H(·). As stated in Section 3.2, we use separate parameters for different H(·) functions. We have also tried using shared parameters for all the H(·) functions. Table 3 shows that using separate parameters can yield nearly 3.0 AP improvement. That is because separate parameters can enhance the flexibility, so that different components in Eq. (6) can search for its own shape to better adjust the loss function.
Comparing with and without Gradient Scale λ. As described in Section 3.2, the searched gradient scale is adopted to serve similar effect as the traditional loss weight. We have also tried not using the gradient scale, i.e., λ is fixed as 1. Table 4 shows that the automatically searched gradient scale can indeed improve the performance.
Comparing Different Measurements for Localization Score l(bi). Table 5 shows the results of using L1, IoU and GIoU [38] for the localization score. GIoU and L1 produce similar results, both of which outperform the performance of IoU. We argue that this is because IoU will back-propagate meaningful gradients only if two boxes are overlapped, while GIoU and L1 do not have such problem.
Comparing Different Number of Segments in f(·; θ). Table 6 shows the results. Too less segments will limit the expressiveness of f(·; θ), which leads to significant drop in performance. However, too many segments may increase the search difficulty. In practice, 5 segments is enough.
Comparing PPO2 [39] and Random Search. Figure 3 shows that PPO2 can find better parameters more efficiently than random search, which suggests that loss function search is non-trivial and reinforcement learning helps to accelerate the search process.

heading: 5 Conclusion
text: In this paper, we proposed Parameterized AP Loss, which represents numerous AP approximations in a unified formula, and automatically searches for the optimal loss function. Parameterized AP Loss is a single unified loss function capturing the classification and localization sub-tasks simultaneously, which consistently outperforms existing loss functions on various object detectors. Although we have verified the effectiveness of parameterization in searching for optimal loss functions for the AP metric, there are still open questions about whether such technique can be extended to other non-differentiable metrics in different tasks, such as the widely used BLEU metric in machine translation, where the calculation of n-gram might be non-trivial to be parameterized.
Potential Negative Societal Impacts. Our searched losses share the same societal issues with other hand-crafted ones, that the trained object detectors may have inexplicable detection failures and suffer from data bias. The automatic search process in our method is laborsaving, which may also have a negative impact on social employment opportunities.
Acknowledgments and Disclosure of Funding
The work is supported by the National Key R&D Program of China (2020AAA0105200) and Beijing Academy of Artificial Intelligence.

Review summaries:
This paper describes a technique for estimating a loss for object detection that is well-aligned with the AP loss used at evaluation time. The work builds on a prior formulation of the AP loss as a function on classification scores, but converts the formulation to account for per-bounding box results, and also replaces the Heaviside function with parametrized, searchable functions. Experiments show the gain from the proposed technique using three backbone detectors, and compares to different prior losses.  Post-rebuttal: The rebuttal addressed my concerns.

Authors propose a loss for object detection, that is a relaxation of mAP suitable for use in gradient ascent. Where AP includes Heaviside (step) functions, the authors replace these by learnt piecewise-linear functions. These functions are trained to maximise AP on a held-out set achieved by a detection model that was itself trained using them as the loss (so they do not aim to approximate AP closely, but to derive a surrogate loss that is 'structurally similar' to AP and achieves good training performance). Experiments on COCO 2017 show accuracy improvements over baselines (trained with focal/XE + regression losses) and other techniques approximating AP.

This paper proposes a Parameterized AP Loss to better align the network training and evaluation in object detection. The main technical contributions include: 1) It proposes a parameterized loss function by reformulating the AP metric and replacing the non-differentiable component with parameterized functions. 2) It automatically searches for the optimal parameters of the proposed loss function through a reinforcement-learning-based search process. 3) Finally, it demonstrates competitive results on COCO. 

This paper proposes a parameterized AP loss. Parameterizd functions are introduced to substitute the non-differentiable components in the AP calculation. Then automatic parameter search algorithms can be employed to search the optimal parameters.