# Unbiased 3D Semantic Scene Graph Prediction in Point Cloud Using Deep Learning
## 3 Methods
### Network Design
#### 3.2.1 Feature Extraction and Coding  
Due to the specificity of the scene graph structure, which requires encoding the features of each node and edge from the point cloud, EdgeConv using DGCNN is chosen as the core part of the feature extraction network [5]. Unlike the previous approach [11; 14] that only considers using PointNet [4] for the geometric feature extraction of point clouds, EdgeConv serves as a high-performance point cloud classification model using graph theory [5]. Moreover, it can learn the point cloud's contextual features and aggregate the neighboring information in the high-dimensional feature space. As shown in Figure 3, we added a multi-scale aggregation mechanism to the original network architecture [30; 31] used to obtain the multi-scale point cloud features and thus design a new feature extraction network architecture, MP-DGCNN.  
Figure 1: Structure of the 3D scene semantic relation graph prediction model. It mainly includes point feature extraction and encoding (in Section 3.2.1), embedding knowledge learning (in Section 3.3), and relational graph inference with embedded feature fusion (in Section 3.4).  
Firstly, for the \(M\) instance objects in the scene, the point cloud data \(N\left(x,\ y,\ z\right)\) is the input object. After regularization and retaining the height to obtain \(N\left(x^{\prime},y^{\prime},z^{\prime},\ h\right)\), the farthest distance sampling algorithm (FPS) is used for each instance object in the scene to sample the number of points \(N\) = 1024, 256, 128. The network extracts the point feature to obtain the preliminary feature map. In order to go through the following graph neural network in the relational inference, the feature initialization of the nodes and edges needs to be performed.  
Node Feature Initialization: The nodes only need to go through one layer of the MLP-transformed feature space to obtain the initial node features \(X_{V}\in R^{M\times c_{point}}\), \(C_{point}=512\).  
Edge Feature Initialization: The edge in the graph \(E\left(i,\ j\right)\) that represents the object node in the scene graph \(V_{i}\) pointing to \(V_{j}\) for each pair of objects \(\left(V_{i},V_{j}\right)\) connects the  
Figure 3: Point cloud geometric feature extraction network (MP-DGCNN).  
Figure 2: The 3D scene graph prediction network. (**a**) First, enter the point cloud into the network. After MP-DGCNN extracts the initial point cloud geometric feature, \(X_{v}\), splice it with the centroid of the object, and after initialization of the node and edge features, obtain the features \(X^{\prime}_{v}\) and \(X^{\prime}_{e}\). (**b**) Knowledge learning network stage: acquire a priori knowledge in the form of embedding vectors by learning the labels of categories in the scene and then add category weights to obtain unbiased meta-embedding (UME). (**c**) Knowledge fusion stage: input \(X^{\prime}_{v}\) and \(X^{\prime}_{e}\), the first inference after ENA-GNN with the classifier, to obtain the initially predicted likelihood vector from the prediction results selected from the top five categories of UME and fuse them to re-run the inference process. (**d**) After the prediction, the scene graph triad is obtained, including objects and predicate relations.  
potential features \((X_{V_{i}^{\prime}},X_{V_{j}})\) with the center coordinates \(P_{c}\in R^{M\times(3+1)}\) of the two point sets. To better encode the relative spatial relationships (e.g., left and front), the edge feature encoding method is used, which is initialized using the concatenate scheme as follows:  
\[X_{\varepsilon_{(i,j)}}=\left(X_{v_{j}}\oplus\left(X_{v_{j}}-X_{v_{j}}\right) \right), \tag{2}\]  
where \(\oplus\) refers to the concatenate of the feature vector, after which the feature space is transformed to the same feature dimension as the node by MLP to obtain the initial edge embedding feature \(X_{\varepsilon}(i,j)\in R^{N\times(N-1)\times C_{point}}\).