FCOS Implementation question

username: isalia99
cooked: <p>In the code of FCOS implemented in torchvision:</p>
<aside class="onebox githubblob" data-onebox-src="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/fcos.py">
  <header class="source">

      <a href="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/fcos.py" target="_blank" rel="noopener nofollow ugc">github.com</a>
  </header>

  <article class="onebox-body">
    <h4><a href="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/fcos.py" target="_blank" rel="noopener nofollow ugc">pytorch/vision/blob/main/torchvision/models/detection/fcos.py</a></h4>


      <pre><code class="lang-py">import math
import warnings
from collections import OrderedDict
from functools import partial
from typing import Any, Callable, Dict, List, Optional, Tuple

import torch
from torch import nn, Tensor

from ...ops import boxes as box_ops, generalized_box_iou_loss, misc as misc_nn_ops, sigmoid_focal_loss
from ...ops.feature_pyramid_network import LastLevelP6P7
from ...transforms._presets import ObjectDetection
from ...utils import _log_api_usage_once
from .._api import register_model, Weights, WeightsEnum
from .._meta import _COCO_CATEGORIES
from .._utils import _ovewrite_value_param, handle_legacy_interface
from ..resnet import resnet50, ResNet50_Weights
from . import _utils as det_utils
from .anchor_utils import AnchorGenerator
from .backbone_utils import _resnet_fpn_extractor, _validate_trainable_layers
</code></pre>



  This file has been truncated. <a href="https://github.com/pytorch/vision/blob/main/torchvision/models/detection/fcos.py" target="_blank" rel="noopener nofollow ugc">show original</a>

  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>It uses anchors (For example on line 396 anchor_generator is defined if there is no anchor_generator defined while instantiating the class). However in the <a href="https://arxiv.org/pdf/1904.01355.pdf" rel="noopener nofollow ugc">paper</a> we read that the model is completely anchor-free. Could someone explain the reason why anchors are used?</p>

username: Peter_Vennerstrom
cooked: <p>FCOS per pixel i,j predictions can be thought of as single per pixel anchors.  The use of the existing AnchorGenerator enables targets to be made at different scales.</p>

username: isalia99
cooked: <p>Yes it can be thought of that way, but the paper’s main idea and goal was to not use anchors and do everything anchor free. In the original implementation on their github page, they in fact don’t use anchors. So it seems weird to me that torchvision’s implementation uses them.</p>

username: Peter_Vennerstrom
cooked: <p>Just like FCOS, this implementation regresses a single set of 4 offsets per i,j location on the output feature maps along with classification and centerness.</p>
<p>The 4 offsets are predicted relative to the i,j location and normalized by size when supervised.</p>
<p>The code is a bit tricky to follow as some elements of ssd are repurposed for FCOS.  Anchors are used to hold stride information in their height and width to normalize the regressed box values.  Ultimately, these anchors are converted to i,j center locations and the offsets are applied to yield predicted bboxes.</p>
<p>Please see compute_loss() in the FCOSHead and the decode() method in BoxLinearCoder.</p>

username: isalia99
cooked: <p>Ah I see, it makes more sense now. Thank you for clarification</p>