# A Vision-Based Method for Simultaneous Instance Segmentation and Localization of Indoor Objects
## 3 System Overview and Method
### Feature Points Detection and Matching  
Image feature point matching algorithms typically consist of two steps: feature point extraction and matching. Building upon the semantic segmentation conducted in the preceding section, this paper proposes a method that integrates semantic information, edge information, and corner information for detecting feature points. The FAST algorithm is employed to efficiently extract local corner features of objects, while the Canny algorithm is utilized to identify edge feature points of objects with sparse textures. This ensures that these objects possess an adequate number of feature points while maintaining a more uniform distribution. Furthermore, we incorporate semantic information by employing improved BRIEF descriptors to describe these feature points. The complete implementation process of the algorithm is illustrated in Figure 3.  
In order to improve the noise immunity of the descriptor, Gaussian templates with window size \(9\times 9\) and variance \(\sigma=2\) are used to filter the image.  
\[G(x,y)=\frac{1}{2\pi\sigma^{2}}e^{\frac{-(x^{2}+y^{2})}{2\sigma^{2}}} \tag{4}\]  
Then select \(n\) comparison points in the \(S\times S\) pixel block around the feature point \(p\), and compare the intensity value of the two pixels, and the comparison rule is as follows:  
\[\tau(p;u,v)=\left\{\begin{array}{l}1,p(u)\leq p(v)\\ 0,others\end{array}\right. \tag{5}\]  
where \(p(u)\), \(p(v)\) is the grayscale value respectively. The BRIEF descriptor of the \(n\) dimension is obtained by logical operations.  
Select \(m\) comparison points and compare the semantic information of the two pixels, and the comparison rule is as follows:  
\[v(p;x,y)=\left\{\begin{array}{l}1,I(x)=I(y)\\ 0,others\end{array}\right. \tag{6}\]  
where \(I(x)\), \(I(y)\) is the semantic information respectively. The BRIEF descriptor of the \(m\) dimension is obtained by logical operations. Therefore, a binary descriptor of \(n+m\) bits is formed for the feature point \(p\), as shown in the following equation:  
\[f_{n}(P)=\sum_{1\leq i\leq n}2^{i-1}\tau(p;u,v)+\sum_{n+1\leq i\leq n+m}2^{i-1 }v(p;x,y) \tag{7}\]  
During the process of matching feature points in the left and right camera images, semantic information is effectively utilized to eliminate unmatched feature points. Subsequently, a brute force algorithm is applied to calculate the Hamming distance for each  
Figure 3: Fusion-based matching of multi-feature points.  
feature point. The minimum Hamming distance value is then compared against a predetermined threshold. If the calculated distance is less than or equal to this threshold, it indicates a match between two points; otherwise, they are considered mismatched. Furthermore, all qualifying features are sorted based on the number of identical elements in their respective bits, with those having the highest count being selected as matching features. It should be noted that no rotation or scale transformation occurs between corrected binocular images; therefore, polar constraints are incorporated into the matching algorithm. By searching for feature points within two pixels above and below the pole during the matching process, not only can accuracy be improved but also computational time can be reduced.