# Supervised Contrastive Learning for 3D Cross-Modal Retrieval
## 1 Introduction  
During the times of social distancing, metaverse platforms emerged, digitally replicating social experiences. Platforms such as Roblox, Fortnite, Decentraland, and Meta Horizon provided interactive virtual environments for social interaction. In the current post-pandemic world, the adoption and progress of the metaverse have become slower, but immersive virtual experiences are provided through space computing use cases. Interoperability between different virtual platforms, in particular, the search and transfer of digital assets across platforms, is an active area of research [1]. This involves the need to search and analyze virtual objects in different modalities, such as rendered two-dimensional (2D) images, three-dimensional (3D) meshes, and 3D point clouds. Developments in deep learning-based multimodal feature extraction have enabled the 3D cross-modal retrieval of virtual objects. Three-dimensional cross-modal retrieval is a task aimed at searching for specific objects in different modalities or formats. To obtain accurate search results, it is important to understand and differentiate the context of the 3D information from various modalities. Object retrieval is a search and ranking task that analyzes the feature similarity of a given set of objects. Objects belonging to the same category or class display similar feature distributions. Traditionally, object retrieval has focused on efficiently increasing inter-class variance and decreasing intra-class variance. If inter-class variation is small, the class boundaries become less distinct. If intra-class variation is large, objects with distinct features belonging to the same class may be categorized as belonging to different classes. Therefore, training for object retrieval aims to simultaneously increase inter-class variance and decrease intra-class variance. In the multimodal environment, additional considerations need to be made for object retrieval. In cross-modal object retrieval, objects in different modalities with the same class or category should exhibit similar feature distributions. Therefore, inter-modal variance has to be decreased to extract similar distributed features between modalities. Several studies were carried out to address the issues in cross-modal retrieval. Center features are representative features for each class in the embedding space. There have been attempts to standardize center features to encourage feature convergence [2; 3]. However, the approaches relying on center features have several issues. First, center features are highly sensitive to hyperparameters, e.g., batch sizes, so it is important to select suitable hyperparameters. Second, there is a lack of contrastive characteristics between classes and modalities. Cross-modal center loss performs convergence only by classes and does not conduct comparisons with other modalities of classes [4]. This reduces the robustness of data variations during evaluation. As a result, the weakness of center features may lead to performance degradation in the evaluation stage where the feature distributions diverge. The Robust Discriminative Learning with Noisy Labels for 2D-3D Cross-Modal Retrieval framework (RONO) [2] introduced noisy labels to extract robust features for variations during the training process, but the fundamental problems behind the center features remained unresolved. Other cross-modal methods aimed to represent feature distributions within a comprehensive context through label prediction via a projection head [5; 6; 7]. The projection head is only used during training and does not influence the object retrieval process. In this paper, we propose a novel 3D cross-modal retrieval method based on cross-modal supervised contrastive learning (CSupCon) and the fixed projection head (FPH) strategy. CSupCon employs contrastive learning with positive and negative samples per class using data augmentation to learn to discriminate among the different classes, and it mitigates sensitivity issues in contrast to center loss.