See It All: Contextualized Late Aggregation for 3D Dense Captioning (2024)

Abstract: 3D dense captioning is a task to localize objects in a 3D scene and generate descriptive sentences for each object. Recent approaches in 3D dense captioning have adopted transformer encoder-decoder frameworks from object detection to build an end-to-end pipeline without hand-crafted components. However, these approaches struggle with contradicting objectives where a single query attention has to simultaneously view both the tightly localized object regions and contextual environment. To overcome this challenge, we introduce SIA (See-ItAll), a transformer pipeline that engages in 3D dense captioning with a novel paradigm called late aggregation. SIA simultaneously decodes two sets of queries—context query and instance query. The instance query focuses on localization and object attribute descriptions, while the context query versatilely captures the region-ofinterest of relationships between multiple objects or with the global scene, then aggregated afterwards (i.e., late aggregation) via simple distance-based measures. To further enhance the quality of contextualized caption generation, we design a novel aggregator to generate a fully informed caption based on the surrounding context, the global environment, and object instances. Extensive experiments on two of the most widely-used 3D dense captioning datasets demonstrate that our proposed method achieves a significant improvement over prior methods.

heading: 1 Introduction
text: 3D dense captioning has been defined in former works (Chen et al., 2021, 2022; Yuan et al., 2022; Wang et al., 2022; Jiao et al., 2022; Cai et al., 2022; Chen et al., 2023a) as the task of localizing all the objects in a 3D scene (i.e., object detection) and generating descriptive sentences for each object (i.e., object caption generation). Early works incorporated a two-stage “detect-then-describe" pipeline,
∗ Work done during internship at LG AI Research † Corresponding authors
where we first detect all the object proposals then generate the captions for each object (Chen et al., 2021; Jiao et al., 2022; Wang et al., 2022; Zhong et al., 2022; Cai et al., 2022; Chen et al., 2022; Yuan et al., 2022). However, the sequential design, lacking sufficient integration of contextual information in these endeavors, has been limited in performance and efficiency.
Vote2Cap-DETR (Chen et al., 2023a) has emulated the transformer encoder-decoder pipeline from object detection (Carion et al., 2020) to alleviate these issues and fashioned an end-to-end pipeline for 3D dense captioning. Powered by transformer attentions, this method contextualizes individual objects (i.e., self-attention with other proposals throughout the global scene) to generate dense captions. Nevertheless, compared to the notable advancements that object detection has experienced, the direct application of this architecture has failed to fully leverage the contextual information required for 3D dense captioning.
Dense captioning has to perform precise object localization while generating captions that either independently describe an object’s attributes (e.g., a wooden chair) or describe the object within its contextual environment (e.g., a chair in front of the TV). This presents a challenging scenario where the feature representation for a single query must encompass both accurate local features for localization or attribute-based caption generation, alongside incorporating contextual features that dynamically span neighboring regions or the broader global scene. Focusing attention on local features can enhance localization and detailed attribute description but reduce sensitivity to the surrounding context. Conversely, spreading attention to include the context can improve understanding of the environmental description but at the cost of localization accuracy.
In this paper, we propose a pipeline engaging a novel late aggregation paradigm called SIA (i.e.,
3395
See It All). Rather than assigning a query per object and training them to dynamically incorporate local features, contextual information, and the global scene, SIA allocates a query per caption. To elaborate, SIA identifies a distinct region to focus on when generating each caption and then consolidates the outcomes concerning identical objects. Figure 1 contrasts our proposed late aggregation approach with existing dense captioning paradigms. While previous works have either (a) extracted the features from localized object areas Cai et al. (2022) or (b) generated captions from features that have to perform both localization and proper caption prediction Chen et al. (2023a), SIA (c) focuses on each unique ROIs for each caption then aggregates the captions that include identical objects afterwards. This architecture (i.e., late aggregation) enables SIA to produce attribute captions and localization with features concentrated exclusively on specific local areas, while captions necessitating a broad range of contextual information can be crafted using features gathered without the constraints of localization boundaries.
To further refine the features for contextual captions, we design a unique aggregator that generates captions based on the conTextual surroundings, Global descriptor, and Instance features (i.e., TGIAggregator, see Figure 1-(d)). Our TGI-Aggregator generates contextual caption based on the fully informed feature that can dynamically capture the
area of interest within the scene without the constraints of localization objectives. Extensive experiments on two widely used benchmarks in 3D dense captioning (i.e., ScanRefer (Chen et al., 2020a) and Nr3D (Achlioptas et al., 2020)) show that our proposed SIA surpasses prior approaches by a large margin. The contribution of our paper can be summarized as:
• We propose a new paradigm for 3D dense captioning (i.e., late aggregation). While previous works aggregate the instance and context features first and then generate captions, SIA generates local and contextual captions separately and then aggregates the captions involving identical objects.
• To further improve the quality of features used for contextualized caption generation, we propose a novel aggregator named TGIAggregator.
• Our SIA achieves state-of-the-art performances across multiple evaluation metrics on the ScanRefer and Nr3D datasets.

heading: 2 Preliminary
text: In this preliminary, we start with a basic transformer-based end-to-end 3D dense captioning pipeline (Chen et al., 2023a). The caption head is attached to the top of the existing 3D object detection pipeline (Qi et al., 2019; Misra et al., 2021)
with vote queries that establish captions for each pinpointed object throughout the scene in an objectby-object manner. Afterward, we discuss why this object-centric application of transformer attention is unsuitable for 3D dense captioning.

heading: 2.1 End-to-End 3D Object Detection
text: 3D object detection aims to identify and localize objects in 3D scenes. VoteNet (Qi et al., 2019) incorporates an encoder-decoder architecture where the bounding boxes are predicted by aggregating the votes for the center coordinates. 3DETR (Misra et al., 2021) generates object queries by uniformly sampling seed points from a 3D scene. Vote2CapDETR (Chen et al., 2023a) uses vote queries that connect the object queries in 3DETR to VoteNet, resulting in better localization and improved training efficiencies.

heading: 2.2 Extension to 3D Dense Captioning
text: The goal of 3D dense captioning is to localize objects in a 3D scene and generate informative natural language descriptions per object. An intuitive extension from object detection to 3D dense captioning is simply applying a captioning head for each object proposals (Chen et al., 2023a). Given an input indoor 3D scene as a point cloud PC = [pin; fin] ∈ RN×(3+F ), where pin ∈ RN×3 is the absolute locations for each point and fin ∈ RN×F is additional input features for each point (Chen et al., 2020a, 2021), the objective of 3D dense captioning is to generate a set of box-caption pairs (B̂, Ĉ) = {(b̂1, ĉ1), ..., (b̂K , ĉK)}, representing an estimation of K distinctive objects in this 3D scene. Captions are generated in parallel with bounding box prediction using a caption head. Since the aforementioned vote queries (i.e., pvote) fail to provide adequate attributes and spatial relations for informative caption generation, the contextual information is leveraged through a separate lightweight transformer (Chen et al., 2023a).

heading: 2.3 Retrospect on Object-Centric Captioning
text: Current 3D dense captioning benchmarks require the model to generate multiple captions for each detected object. Therefore, it seems natural to approach this task in an object-centric manner (Wang et al., 2022; Jiao et al., 2022; Achlioptas et al., 2020), where we generate captions per each object proposal. However, unlike object detection, dense captioning requires an extensive understanding of the scene, including the attributes of each object
and the relative information between objects and the global scene. Therefore, designating the queries per object requires a single query attention to versatilely encompass the individual object and its surrounding elements, failing to concentrate on the local element it should describe effectively. We propose a novel late aggregation approach for 3D dense captioning to address this issue and incorporate contextual scene information.

heading: 3 Method
text: In this section, we introduce a transformer encoderdecoder pipeline that engages our novel late aggregation paradigm for 3D dense captioning. In previous methods, the transformer attention aggregates contextual information per-object, where a single feature is used to perform localization, generate localized attributes and simultaneously capture the surrounding context area. SIA is designed to capture the unique region of interest for each caption. Local attribute descriptions are generated with localized features. In contrast, contextualized captions that include relationships with other objects or the entire scene are generated with a separately decoded feature irrelevant to localization objectives. Then, captions involving identical objects are aggregated via distance (i.e., late aggregation) to consist of the final caption. The overall pipeline is illustrated in Figure 2.

heading: 3.1 Encoder
text: Given the input point cloud PC = [pin; fin] ∈ RN×(3+F ), the input point cloud is first tokenized by a set-abstraction layer of PointNet++ (Qi et al., 2017). The tokenized output is inputted into a masked transformer encoder with the setabstraction layer, followed by two additional encoder layers. The final encoded scene tokens are denoted as penc ∈ R1,024×3 and fenc ∈ R1,024×256.

heading: 3.2 Context Query and Instance Query
text: To disentangle the captions that are bound to a single object and captions that include relative information with other objects or the global scene, we designate two separate instance query and context query to each capture a unique region per caption within the 3D scene. While the context query captures the local-global regions capable of captioning, the instance query generates standard object localization and attribute-related caption prediction for each object. The two queries are decoded in parallel and later aggregated to consist of the final caption.
Context Query Generator. Given the encoded scene tokens (penc, fenc), we sample 512 context points pcseed with farthest point sampling (FPS) on penc. Then, the context query (pc, f c) is represented as:
(pc, f c) = SAc(penc, fenc), (1)
where SAc denotes the set-abstraction layer (Qi et al., 2017) with a radius of 1.2 and samples 64 points for pc.
Instance Query Generator. The instance query is decoded to perform standard 3D object detection and generate captions for the individual attributes of each object. Likewise, the instance query (po, fo) is written as:
[∆pvote; ∆fvote] = FFNo(fenc), (2)
(po, fo) = SAo ( penc+∆pvote, fenc+∆fvote ) , (3)
where [∆pvote; ∆fvote] ∈ R1,024×(3+256) is an offset that learns to shift the encoded points to object’s centers spatially by a feed-forward network FFNo, following (Chen et al., 2023a). SAo denotes the set-abstraction layer with a radius of 0.3 and samples 16 points for po. All hyper-parameters are set experimentally.

heading: 3.3 Decoding
text: Given the context and instance queries, we build a parallel decoding pipeline where the Context Decoder describes contextual information between
objects, and the Instance Decoder performs the localization and attribute description. We then feed the decoded context query V c and instance query V o to our TGI-Aggregator.
TGI-Aggregator. Figure 3 shows a conceptual illustration of our TGI-Aggregator. To encompass the understanding of the entire scene for each caption, we generate a global feature using all decoded context queries V c and instance queries V o; related experiments can be found in Section 4.4. We deploy a clustering-based aggregator (Arandjelovic et al., 2016) G(·). As a result, we obtain a global descriptor V g ∈ R256 by V g = G(V c, V o). Then, we concatenate this global descriptor V g to each
decoded instance query V oi and K nearest features within V c in terms of spatial proximity to V oi , resulting in an aggregated feature V a that contains a comprehensive information of conText, Global, and Instance. We set K to 16.
Contextual Caption Generation. For caption generation, we adopt a transformer decoder-based caption head based on GPT-2 (Radford et al., 2019), following Vote2Cap-DETR (Chen et al., 2023a) and SpaCap3D (Wang et al., 2022). The output V a of our TGI-Aggregator contains features about contextual surroundings, global context, and instance information. Based on this feature, SIA can generate descriptions for relationships between multiple objects (e.g., the chair is next to a bookshelf) and global relationships (e.g., the table is in the middle of the room).
Localization & Attribute Caption Generation. To predict localization and attribute descriptions for the participating instances, we feed the decoded instance query V o and parallelly feed it into the detection head and (shared) caption head. For localization, we follow 3DETR (Misra et al., 2021), reformulating the box corner estimation as offset estimation from a query point to an object’s center and box size regression. All subtasks are implemented by FFNs. The object localization head is shared throughout the decoder layers.

heading: 3.4 Training SIA
text: We construct the final caption to be object-centric to compare with previous object-centric methods using benchmark datasets. The final caption for the i-th object is obtained by simply concatenating the captions generated from V oi and V a i . Our SIA is trained and evaluated by locating all objects within a scene and comparing the final caption centered on each object with the ground-truth.
Instance Query Loss. To train the Instance Query Generator to find an object’s center by shifting points penc, we adopt the vote loss from VoteNet (Qi et al., 2019). Given the generated instance query (po, fo) and the encoded scene tokens (penc, fenc), the vote loss Lo is written as:
Lo = 1 M
M∑
i=1
Ngt∑
j=1
∥poi − cntj∥1 · I(pienc), (4)
where I(x) is an indicator function that equals 1 when x ∈ Ij and 0 otherwise, Ngt is the number
of instances in a 3D scene, M is the number of po, and cntj is the center of j-th instance Ij .
Detection Loss. We use Hungarian matching (Kuhn, 1955) to assign each proposal with the ground-truth, following DETR (Carion et al., 2020). The detection loss Ldet is written as:
Ldet = α1Lgiou +α2Lcls +α3Lcnt +α4Lsize, (5)
where α1 = 10, α2 = 1, α3 = 5, α4 = 1 are set heuristically. The detection loss is applied across all decoder layers for better convergence.
Caption Loss. Following the standard protocol for image captioning, we first train caption heads with standard cross-entropy loss for Maximum Likelihood Estimation (MLE). In the MLE training, the model learns to predict the (t+ 1)-th word ct+1n based on the first t words c [1:t] n and the visual features V . The loss function is established for the final caption with length T is defined as follows:
Lcn = T∑
t=1
Lcn(t) = − T∑
t=1
log P̂ ( ct+1n |V, c[1:t]n ) ,
(6) Once the caption head is trained with word-level supervision, it is refined using Self-Critical Sequence Training (SCST) (Rennie et al., 2017). In this phase, the model produces multiple captions ĉ1,...,k with a beam size of k and an additional ĝ using greedy search as a baseline. The loss function for SCST is formulated as follows:
Lcn = − k∑
i=1
(R(ĉi)−R(ĝ)) · 1
|ĉi| log P̂ (ĉi|V).
(7) The reward function R(·) is based on the CIDEr (Vedantam et al., 2015) metric for caption evaluation, and the logarithmic probability of the caption ĉi is normalized by caption length |ĉi|, promoting equal importance to captions of varying lengths by the model.
Final Loss for SIA. Given the instance query Loss Lo, the detection loss for the i-th decoder layer as Lidet, and the average of the caption loss Lcn within a batch denoted as Lcap, the final loss L for SIA is written as:
L = β1Lo + β2 ndec-layer∑
i=1
Lidet + β3Lcap , (8)
where β1 = 10, β2 = 1, and β3 = 10.

heading: 4 Experiments

heading: 4.1 Datasets and Metrics
text: Datasets. In our studies, we focus on 3D dense captioning and employ two established datasets: ScanRefer (Chen et al., 2020a) and Nr3D (Achlioptas et al., 2020). These datasets are rich in humangenerated descriptions, with ScanRefer providing 36, 665 descriptions for 7, 875 objects across 562 scenes, and Nr3D offering 32, 919 descriptions for 4, 664 objects in 511 scenes. For training, these descriptions and objects are derived from the ScanNet (Dai et al.) database, which comprises 1, 201 3D scenes. For evaluation, we use 9, 508 descriptions from ScanRefer and 8, 584 from Nr3D, corresponding to 2, 068 and 1, 214 objects across 141 and 130 scenes, respectively, from the 312 3D scenes in the ScanNet validation set.
Metrics. We evaluate the model using four types of performance metrics: CIDEr (Vedantam et al., 2015), BLEU-4 (Papineni et al., 2002), METEOR (Banerjee and Lavie, 2005), and ROUGE-L (ChinYew, 2004), denoted as C, B-4, M, and R, respectively. Following the previous studies (Chen et al., 2021; Cai et al., 2022; Jiao et al., 2022; Wang et al., 2022; Chen et al., 2023a), we first employ NonMaximum Suppression (NMS) to eliminate redundant object predictions from the object proposals. Each proposal is represented as a pair consisting of a predicted bounding box b̂i and a generated caption ĉi. To assess both the model’s ability to locate objects and generate captions accurately, we
employ the m@k, setting the IoU threshold k at 0.25 and 0.5 for our experiments, following (Chen et al., 2021):
m@k = 1
N
N∑
i=1
m (ĉi, Ci)·I { IoU ( b̂i, bi ) ≥ k } ,
(9) where N is the number of all annotated instances in the evaluation set, and m represents the captioning metrics C, B-4, M, and R.

heading: 4.2 Implementation Details
text: Our training phase is structured into three stages, following the approach of (Chen et al., 2023a). Initially, we pre-train our network excluding the caption head on the ScanNet (Dai et al.) dataset for 1, 080 epochs. The batch size is 8. The loss function is minimized using AdamW optimizer (Loshchilov and Hutter, 2017), for which the initial learning rate is 5× 10−4 and decreases to 10−6 according to a cosine annealing schedule. We also apply a weight decay of 0.1 and a gradient clipping of 0.1, as suggested by (Misra et al., 2021). Afterward, starting from the pre-trained weights, we jointly train the entire model with the standard cross-entropy loss for an additional 720 epochs on the ScanRefer (Chen et al., 2020a) and Nr3D (Achlioptas et al., 2020) datasets, fixing the detector’s learning rate at 10−6 and reducing the caption head’s from 10−4 to 10−6 to prevent over-fitting (about 20/17 hours for ScanRefer/Nr3D). In the SCST (Rennie et al., 2017) phase, we adjust the caption head using a batch size of 2 while keeping the detector fixed
over a span of 180 epochs and maintain a constant learning rate of 10−6 (about 22/18 hours for ScanRefer/Nr3D). In the experimental setup that uses additional 2D data, as shown in Table 1, we employ the pre-trained ENet (Chen et al., 2020b) to extract 128-dimensional multiview features from 2D ScanNet images, as in the (Chen et al., 2021). The parameter size of our model is 21M, and the average inference time on the evaluation set of the ScanRefer is 1.8ms. All experiments of our SIA are conducted with one Titan RTX GPU on PyTorch (Paszke et al., 2019).

heading: 4.3 Comparison with Existing Methods
text: In this section, we benchmark our performance against eleven state-of-the-art methods: Scan2Cap (Chen et al., 2021), D3Net (Chen et al., 2022), SpaCap3D (Wang et al., 2022), MORE (Jiao et al., 2022), 3DJCG (Cai et al., 2022), Contextual (Zhong et al., 2022), REMAN (Mao et al., 2023), 3D-VLP (Jin et al., 2023), χ-Tran2Cap (Yuan et al., 2022), Vote2CapDETR (Chen et al., 2023a), and Unit3D (Chen et al., 2023b). We apply IoU thresholds of 0.25 and 0.5 for ScanRefer (Chen et al., 2020a) as shown in Table 1 and an IoU threshold of 0.5 for Nr3D (Achlioptas et al., 2020) indicated in Table 2. For the baselines, we present the evaluation results reported in the original papers, and "-" in Table 1 and Table 2 means that such results have not reported in the original paper or follow-up study.
ScanRefer. The descriptions in the ScanRefer include depictions of the target object’s attributes and information about the spatial relationships between this target object and other surrounding objects. Table 1 summarizes the results on the ScanRefer dataset. Our method significantly surpasses current methods in 3D dense captioning across all in-
put data settings and IoU threshold configurations. We attribute this enhancement to our contextualized late aggregation mechanism. Unlike previous object-centric methods where surrounding information is bound to the center of object proposals, SIA directly targets the region-of-interest for contextual captions with the context query while simultaneously addressing object localization and its attribute captions with the instance query.
Nr3D. The Nr3D dataset is designed to evaluate the model’s performance in interpreting free-form natural language descriptions of objects as spoken by humans. Our SIA quantitatively demonstrates its ability to generate various descriptions for an object by showing state-of-the-art performance across all evaluation metrics, as shown in Table 2.

heading: 4.4 Ablation Study and Discussion
text: The core components of SIA consist of three factors: i) decomposing the query set into instance query V o and context query V c, ii) generating the global descriptor V g, and iii) composing the fully informed contextualized feature V a using our TGIAggregator. In our ablation study, we validate that every component of our proposed SIA positively contributes to the final performance.
Instance Query Generator. We define SIA using only the Instance Query Generator (i.e., SIA using only V o in Table 3) as our baseline and compare it with Vote2Cap-DETR (Chen et al., 2023a), an object-centric transformer encoder-decoder ar-
chitecture. The major difference between our baseline and Vote2Cap-DETR is how we generate the query set for instances. Vote2Cap-DETR uses farthest point sampling (FPS) to generate queries before the query coordinates are adjusted through voting. Therefore, if the coordinates are mistakenly focused on a specific object after voting, features will be extracted from the same object. On the other hand, our baseline extracts the features from the candidate coordinates after the voting. This enhancement boosts localization performance in terms of mean Average Precision (mAP) and Average Recall (AR), naturally leading to improvements in dense captioning performance.
Context Query Generator. SIA decomposes the role of queries into instance query V o that focuses on the object itself and context query V c that designates the contextual region. SIA w/o V g in Table 3 shows the result of generating a caption using only context query V c and instance query V o in the TGI-aggregator, excluding the global descriptor. By showing high performance improvement compared to the results of SIA using only V o, we demonstrate the effectiveness of query set separation.
TGI-Aggregator. As shown in Table 3, utilizing a global descriptor V g in the TGI-Aggregator results in performance improvements across all aspects. Table 4 shows our ablation study on how we aggregate the decoded context query V c and instance query V o to construct the global feature V g that is afterward fed to our TGI-Aggregator (recall Figure 3). The scenarios include i) aggregating all context features V c, ii) gathering one instance feature V oi with all context features V
c, and iii) aggregating all instance features V o and context features V c to extract a global feature. Aggregating all instance and context features to create a global feature results in the best performance. This implies that reflecting all instances and contexts is better when representing the entire scene.
We also compare two of the most widely used aggregation frameworks for whole-scene representation: GeM pooling (Radenović et al., 2019) and NetVLAD (Arandjelovic et al., 2016), as shown in
Table 5. We empirically adopt NetVLAD for our Global Aggregator. In our TGI-Aggregator, we concatenate the global descriptor V g to each decoded instance query V oi and K context features that are locationally closest to V oi within V c.
Analysis of hyper-parameter K. In the TGIAggregator, we concatenate the global descriptor V g to each decoded instance query V oi and K nearest features within V c in terms of spatial proximity to V oi . Based on the experimental results of Table 6, we set K = 16. A performance decrease is observed with K = 8, likely due to insufficient contextual information, while K = 32 shows little performance change despite significantly increasing memory and execution time costs.

heading: 4.5 Qualitative Analysis
text: We qualitatively present the captions generated from i) using only instance features V o, ii) using only the fully informed contextualized feature V a from the TGI-Aggregator, as shown in Table 7. It can be seen that while the captions generated from V o include descriptions for attributes, the captions generated from V a include contextual information such as relations with other objects and the global scene.
We also provide a qualitative comparison
with the state-of-the-art models: Scan2Cap (Chen et al., 2021), SpaCap3D (Wang et al., 2022), D3Net (Chen et al., 2022), 3DJCG (Cai et al., 2022), and Vote2Cap-DETR (Chen et al., 2023a). The ground-truth includes descriptions of the intrinsic properties of the object (e.g., the monitor is black and square.), explanations using the relationships between objects (e.g., a picture is hanging on the back wall with a couch below it.), and descriptions of the object in the context of the entire space (e.g., at the far end of the room.). Existing models, focusing on objects, generate captions limited to the object and its immediate relations in a fixed format (e.g., this is a black suitcase.). Our model can handle not only object queries but also context queries, allowing it to generate sentences in various formats (e.g., the vending machine is a white rectangle.) and create descriptions of the entire space (e.g., the vending machine is in the corner of the room.). These results emphasize the
importance of an integrated understanding of the object, its surroundings, and the overall space in captioning.

heading: 5 Conclusion
text: In the 3D dense captioning task, the description of an object within a 3D scene encompasses not only the intrinsic characteristics of the object but also the relationship with surrounding objects and the spatial relationship of the object with respect to the overall space. In this work, we propose a novel approach that independently generates captions with different region of interests and aggregates them afterwards to enhance local-global sensitivity of descriptions. Through extensive experiments on benchmark datasets, our method significantly improves 3D dense captioning over previous approaches, demonstrating the importance of an integrated understanding of objects, surroundings, and overall space for caption generation.
Limitations
SIA still faces the limitations of prior set prediction architectures, where the number of instance and context queries must be heuristically determined. Future work could explore methods that allow for the dynamic adjustment of instances and context query numbers based on the complexity of the scene.
Ethical Considerations
3D dense captioning is the task of locating objects in a 3D scene and generating captions for each object. Our proposed method, specifically designed for this task, is ethically sound, employing only publicly available datasets throughout our research. These benchmark datasets feature 3D indoor environments exclusively populated with objects.

heading: Acknowledgement
text: We sincerely thank Junhyug Noh, Seokhee Hong, Jaewoo Ahn, and Dayoon Ko for their constructive comments. This work was supported by LG AI Research and Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (No. RS-2019-II191082, No. RS2022-II220156) funded by the Korea government (MSIT).

heading: Future Work
text: Future work could explore methods that allow for the dynamic adjustment of instances and context query numbers based on the complexity of the scene. Ethical Considerations
3D dense captioning is the task of locating objects in a 3D scene and generating captions for each object. Our proposed method, specifically designed for this task, is ethically sound, employing only publicly available datasets throughout our research. These benchmark datasets feature 3D indoor environments exclusively populated with objects