MTChat: A Multimodal Time-Aware Dataset and Framework for Conversation
Authors: Wanqi Yang, Yanda Li, Meng Fang, Ling Chen


Reviewer paper summaries: This paper introduces MTChat, a multimodal time-aware dialogue dataset that integrates linguistic, visual, and temporal elements in both dialogues and persona memories. The authors propose two time-sensitive tasks: Temporal Next Response Prediction (TNRP) and Temporal Grounding Memory Prediction (TGMP). Additionally, they present a framework with an adaptive temporal module designed to integrate multimodal streams effectively. The paper claims that experimental results demonstrate the novel challenges posed by MTChat and the effectiveness of their proposed framework in multimodal time-sensitive scenarios.
Summary of strengths: - The paper is generally clear and easy to follow, making the proposed ideas accessible to readers.
- MTChat appears to be the first multimodal time-aware dialogue dataset that incorporates temporal dynamics in both responses and grounding memories.
- The introduction of TNRP and TGMP tasks potentially broadens the research landscape for time-sensitive domains in conversational AI.
Summary of weaknesses: - Limited temporal modeling: Despite emphasizing the importance of temporal information, the method of incorporating time data (concatenating date information with text) seems simplistic and raises concerns about its effectiveness in modeling temporal dynamics.
- Questionable "adaptive temporal module" (ATM): The design of the ATM appears to be a simple integration of textual and visual modalities without significant consideration of temporal information. This raises questions about the appropriateness of labeling it as a "temporal" module. Also, the comparison in Table 3 doesn't include baselines that consider temporal features, which weakens the claimed advantages of the ATM in terms of temporal information incorporation.
- Insufficient analysis: The paper lacks in-depth analysis of how the temporal aspect affects model performance and how it compares to non-temporal alternatives
- Overall, The proposed method doesn't present significant technical novelty to multimodal conversation.
Comments, suggestions and typos: Please refer to the comments and questions raised in Weakness section.
Overall assessment scores: 3