# ACT-FRCNN: Progress Towards Transformer-Based Object Detection
## 3 Method and Materials  
In ACT-FRCNN, a detection network is used with Adaptive Clustering backbone. This addition allows us to determine bounding box classifications and coordinates. The original transformer's design is identical to the adaptive clustering transformer (ACT) we proposed. ACT reduces the calculation cost of the transformer without requiring retraining. Through a few rounds of knowledge distillation fine-tuning, we can narrow the performance gap between ACT and the base transformer [1]. Figure 1 shows a visual representation of the ACT-FRCNN architecture. The RPN generates multiple predictions for each location on the map to identify exciting areas likely to contain objects. Each prediction corresponds to a specific anchor on the map and has different sizes and aspect ratios. The first step is to determine if an object exists in that area, and then the system determines the coordinates of a box around the object. Parameterization allows us to calculate the bounding boxes as distances from the anchor boxes.  
\[\begin{array}{ll}t_{x}=(x-x_{a})/w_{a}&\qquad t_{x}^{*}=(x^{*}-x_{a})/w_{a}\\ t_{y}=(y-y_{a})/h_{a}&\qquad t_{y}^{*}=(y^{*}-y_{a})/h_{a}\\ t_{w}=\log(w/w_{a})&\qquad t_{w}^{*}=\log(w^{*}/w_{a})\\ t_{h}=\log(h/h_{a})&\qquad t_{h}^{*}=\log(h^{*}/h_{a})\end{array} \tag{1}\]The coordinates (\(x\), \(y\), \(w\), \(h\)) of the box's center, width, and height are denoted by \(x\), \(x_{a}\), and \(x^{*}\), whereas \(x\), \(x_{a}\), and \(x^{*}\) represent the prediction, anchor, and ground truth, respectively. When comparing the true offsets \(t^{*}\) to the projected offsets \(t\), the box training loss is equivalent to a Huber loss. The best candidates are used to RoI-pool regions from the feature map based on the RPN's thorough forecast of object-containing regions, yielding one feature proposal for each region. The same parameterization as before provides these pooled features to a pair of lightweight heads, which then generate bounding box regressions and predictions for object categories (or background).  
The DETR encoder includes features that allow it to use the attention mechanism to gather information from nearby positions. Our research shows that similar features in close physical proximity tend to produce identical attention maps. ACT must select prototypes representing the whole and communicate updated characteristics to its neighbor since its self-attention overlaps with its nearest neighbor. The more complex the encoder becomes, the more similar its characteristics grow as they start to share data. Also, each encoder layer shows a significant variation in the distributions of input features. These findings suggest using an adaptive method to determine the required number of prototypes for each data layer instead of relying on a predetermined set of cluster centers.  
To tackle the issue of encoder attention redundancy, ACT utilizes key-query attention for representative prototypes. These prototypes are created by clustering similar query features and then averaging them. The modified features of the prototypes are then shared with their neighbors by using the Euclidean distance on the query feature space. Due to the wide variety of encoder characteristics, we need an adaptive clustering method to categorize features based on their location in the input and encoder layers. Our technique called multi-round exact Euclidean locality sensitivity hashing (E2 LSH) [19] achieves query feature distribution-aware clustering.