# Deep Clustering of Student Code Strategies Using Multi-View Code Representation (CMVAE)
## 3 Methodology
### Pre-Training for Multi-View Representations  
After obtaining the structural and semantic representation of the code, a pre-training phase is performed. The aim is to learn an unsupervised representation of the code that reflects the code strategy by simultaneously optimizing the reconstruction loss of the autoencoder and the contrast loss between different views. An autoencoder consists of an encoder and a decoder and is commonly used in self-supervised learning for feature extraction. It works by reducing the discrepancy between the input and its reconstructed version, effectively capturing hidden patterns or features in the data. The advantage of this approach is that there is no need to label the data and the reconstruction target can be constructed solely from the input data. In this work, the code is treated as multi-view data, where the structural feature embedding \(X_{1}\) and the semantic feature embedding \(X_{2}\) are first extracted. These embeddings are then fed into separate transformer encoders, which generate the corresponding structural feature representation \(Z_{1}\) and semantic feature representation \(Z_{2}\). By using independent encoders to model the structural and semantic features separately, the uniqueness of each view is preserved. This prevents information loss from occurring in the early stages and ensures the completeness and complementarity of the structural and semantic representations in the latent space. The model employs an attention mechanism to fuse structural and semantic features to generate a unified code representation. The mechanism captures fine-grained relationships between different representations and dynamically assigns weights to structural features and semantic features according to task requirements. As a result, the contribution of eachfeature to the fused representation is dynamically adjusted and the final representation is projected into a unified potential space. The calculation is shown in Equation (10):  
\[Z=\text{Attention}(Z_{1},Z_{2}), \tag{10}\]  
where \(Z\) denotes the unified code representation, and \(Z_{1}\) and \(Z_{2}\) represent the structural and semantic features, respectively. The fused code representation then reconstructs the original structural features and semantic features through a decoder. The reconstruction loss, which is used to measure the error between the input and reconstructed data, ensures that the model retains as much input information as possible while preserving the full information from both views in the fused features, allowing for better capture of the global features of the structural features and semantic features of the code. The reconstruction loss is computed as shown in Equation (11):  
\[\mathcal{L}_{recon}=\|X-\hat{X}\|_{2}, \tag{11}\]  
where \(\|\cdot\|_{2}\) denotes the Euclidean distance between the input and the reconstructed representations. To better capture the commonality and differences between multi-view information in the representation, contrastive learning loss is incorporated. Each representation is normalized, and the structural and semantic representations are concatenated in different directions. To create pairs of positive samples, corresponding structural and semantic features of the same code were used. Cosine similarity was then applied to measure the similarity between these samples and the model was optimized using cross-entropy loss to encourage a more consistent alignment of structural and semantic features in the latent space. The contrast loss aims to enhance the fusion of multi-view data by increasing the alignment between structural and semantic representations.