Discussion about the paper of name CenterNet

vauthority: <p>Hello everyone!<br>
Currently I’ve started reading the paper of name “CenterNet: Objects as Points”. The general idea is to train a keypoint estimator using heat-map and then extend those detected keypoint to other task such as object detection, human-pose estimation, etc. But the thing that confused me is how to splat the ground truth keypoint onto a heat-map by using Gaussian kernel. <div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/2/4/24ce160540d0dd002caa0c1c6ed45cee250c8ccf.png" data-download-href="https://discuss.pytorch.org/uploads/default/24ce160540d0dd002caa0c1c6ed45cee250c8ccf" title="Annotation 2020-05-11 162433"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24ce160540d0dd002caa0c1c6ed45cee250c8ccf_2_690x266.png" alt="Annotation 2020-05-11 162433" data-base62-sha1="5fAJ1u2zLSmT2MviQftnhBleRsP" width="690" height="266" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/2/4/24ce160540d0dd002caa0c1c6ed45cee250c8ccf_2_690x266.png, https://discuss.pytorch.org/uploads/default/original/3X/2/4/24ce160540d0dd002caa0c1c6ed45cee250c8ccf.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/2/4/24ce160540d0dd002caa0c1c6ed45cee250c8ccf.png 2x" data-dominant-color="E5E5DE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Annotation 2020-05-11 162433</span><span class="informations">768×297 93.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div>  What is object size-adaptive standard deviation? How to calculate the sigma for that formula.<br>
Best regards!</p>

tom: <p>From [30 - Hei Law, Jia Deng: CornerNet: Detecting Objects as Paired Keypoints]:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/f/ff0bfc7cb453de2b533414ebd980909d9143f97d.png" data-download-href="https://discuss.pytorch.org/uploads/default/ff0bfc7cb453de2b533414ebd980909d9143f97d" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff0bfc7cb453de2b533414ebd980909d9143f97d_2_527x499.png" alt="image" data-base62-sha1="AofqZR3qoDMF5sgMHRrjqzV906x" width="527" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff0bfc7cb453de2b533414ebd980909d9143f97d_2_527x499.png, https://discuss.pytorch.org/uploads/default/optimized/3X/f/f/ff0bfc7cb453de2b533414ebd980909d9143f97d_2_790x748.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/f/f/ff0bfc7cb453de2b533414ebd980909d9143f97d.png 2x" data-dominant-color="E0E1E3"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">816×773 252 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>So the reference without comment how they do something different would seem to indicate that they do something very similar, i.e. I expect them to see how much you can move the bounding box (keeping the size the same, if they only consider the center, I guess) while keeping IoU &gt;= 0.7. Without some thought, I wouldn’t know whether axial or some diagonal translation is crucial, but you could experiment with that. This would be the radius. Then you could take 1/3 like in the reference or so.<br>
Whether that is exactly what they do, I don’t know. However, I would expect to get radii that look somewhat similar to the ones pictured.</p>
<p>Best regards</p>
<p>Thomas</p>

vauthority: <p>Thanks for you quick reply!<br>
Thomas I checked the code of the CornerNet and it used a formula called unormalized gaussian2d, I googled but didn’t find out the result. Do you know where to derive that formula?</p>

tom: <p>The formula is just the one you have above for Y_xyc, no?</p>

vauthority: <p>Dear Thomas!<br>
I tried that formula but it outputs zero?<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/d/a/daa5e6f93d453471b05fe9cb66c52a338907f424.png" data-download-href="https://discuss.pytorch.org/uploads/default/daa5e6f93d453471b05fe9cb66c52a338907f424" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/d/a/daa5e6f93d453471b05fe9cb66c52a338907f424.png" alt="image" data-base62-sha1="vcfx0OhS1UmmIC6cytiZL04g3FG" width="690" height="397" data-dominant-color="F2F2F2"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">723×417 8.99 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
It’s that anything I missing?<br>
Best Regards!</p>

hqabcxyxz: <pre><code class="lang-python">def gaussian_radius(det_size, min_overlap=0.7):
    height, width = det_size

    a1 = 1
    b1 = (height + width)
    c1 = width * height * (1 - min_overlap) / (1 + min_overlap)
    sq1 = np.sqrt(b1 ** 2 - 4 * a1 * c1)
    r1 = (b1 + sq1) / 2

    a2 = 4
    b2 = 2 * (height + width)
    c2 = (1 - min_overlap) * width * height
    sq2 = np.sqrt(b2 ** 2 - 4 * a2 * c2)
    r2 = (b2 + sq2) / 2

    a3 = 4 * min_overlap
    b3 = -2 * min_overlap * (height + width)
    c3 = (min_overlap - 1) * width * height
    sq3 = np.sqrt(b3 ** 2 - 4 * a3 * c3)
    r3 = (b3 + sq3) / 2
    return min(r1, r2, r3)

def draw_gaussian(heatmap, center, radius, k=1):
    diameter = 2 * radius + 1
    gaussian = gaussian2D((diameter, diameter), sigma=diameter / 6)

    x, y = int(center[0]), int(center[1])

    height, width = heatmap.shape[0:2]

    left, right = min(x, radius), min(width - x, radius + 1)
    top, bottom = min(y, radius), min(height - y, radius + 1)

    masked_heatmap = heatmap[y - top:y + bottom, x - left:x + right]
    masked_gaussian = gaussian[radius - top:radius + bottom, radius - left:radius + right]
    if min(masked_gaussian.shape) &gt; 0 and min(masked_heatmap.shape) &gt; 0:  # TODO debug
        np.maximum(masked_heatmap, masked_gaussian * k, out=masked_heatmap)
    return heatmap
</code></pre>
<p>here is official code count sigma</p>