pwc-archive/papers-with-abstracts huggingface dataset:
See How You Read? Multi-Reading Habits Fusion Reasoning for Multi-modal Fake News Detection
Authors: Lianwei Wu，Pusheng Liu，Yanning Zhang
Date: 2023-06-06 00:00:00
Proceeding: journal 2023 6
Tasks: Fake News Detection

The existing approaches based on different neural networks automatically capture and fuse the multimodal semantics of news, which have achieved great success for fake news detection. However, they still suffer from the limitations of both shallow fusion of multimodal features and less attention to the inconsistency between different modalities. To overcome them, we propose multi-reading habits fusion reasoning networks (MRHFR) for multi-modal fake news detection. In MRHFR, inspired by people's different reading habits for multimodal news, we summarize three basic cognitive reading habits and put forward cognition-aware fusion layer to learn the dependencies between multimodal features of news, so as to deepen their semantic-level integration. To explore the inconsistency of different modalities of news, we develop coherence constraint reasoning layer from two perspectives, which first measures the semantic consistency between the comments and different modal features of the news, and then probes the semantic deviation caused by unimodal features to the multimodal news content through constraint strategy. Experiments on two public datasets not only demonstrate that MRHFR not only achieves the excellent performance but also provides a new paradigm for capturing inconsistencies between multi-modal news.
Context-aware Style Learning and Content Recovery Networks for Neural Style Transfer
Authors: Lianwei Wu, Pusheng Liu, Yuheng Yuan, Siying Liu, Yanning Zhang
Date: 2023-05-01 00:00:00
Proceeding: journal 2023 5
Tasks: Style Transfer

Neural text transfer aims to change the style of a text sequence while keeping its original content. Due to the lack of parallel data, unsupervised learning-based approaches have gained considerable development. However, there are still several problems in these approaches: (1) The generated transferred sequences sometimes have inconsistencies between the transferred style and content, and (2) It is difficult to ensure sufficient preservation of the core semantics of original sequences in the transferred sequences. To address these defects, we propose Context-aware Style Learning and Content Recovery networks (CSLCR) for neural text transfer. Specifically, to improve the consistency between the transferred style and content, the designed context-aware style learning layer (CSL) retrieves target style samples with similar semantics to the original sequence, and promotes deep interactive fusion with the original sequence, so as to generate transferred sequence with context-aware style. To tackle the second problem, we explore content constraint recovery layer (CCR) from an indirect perspective, which decodes and recovers the core content semantics of the original sequence and the transferred sequence by both recovery decoding layers, respectively, and intensifies the preservation of the core semantics of both the sequences by a multi-level constraint mechanism. Experiments on two public datasets demonstrate the superiority of our proposed method.
Heuristic Heterogeneous Graph Reasoning Networks for Fact Verification
Authors: Lianwei Wu; Dengxiu Yu; Pusheng Liu; Chao Gao; Zhen Wang
Date: 2023-06-13 00:00:00
Proceeding: journal 2023 6
Tasks: Fact Verification, graph construction, Table-based Fact Verification

Existing studies on table-based fact verification generally capture linguistic evidence from claim-table subgraphs or logical evidence from program-table subgraphs independently. However, there is insufficient association interaction between the two types of evidence, which makes it difficult to obtain valuable consistency features between them. In this work, we propose heuristic heterogeneous graph reasoning networks (H2GRN) to capture the shared consistent evidence by strengthening associations between linguistic and logical evidence from two perspectives of graph construction and reasoning mechanism. Specifically, 1) to enhance the close connectivity of the two subgraphs, rather than simply connecting two subgraphs by the nodes with the same content (the constructed graph in this way has severe sparsity), we construct a heuristic heterogeneous graph , which relies on claim semantics as heuristic knowledge to guide the connections of the program-table subgraph, and in turn expands the connectivity of the claim-table subgraph through logical information of programs as heuristic knowledge; and 2) to establish adequate association interaction between linguistic evidence and logical evidence, we design multiview reasoning networks . In detail, we propose local-view multihop knowledge reasoning (MKR) networks to enable the current node to establish association not only with one-hop neighbors, but also with multihop neighbors, to capture context-richer evidence information. We execute MKR on heuristic claim-table and program-table subgraphs to learn context-richer linguistic evidence and logical evidence, respectively. Meanwhile, we develop global-view graph dual-attention networks (DAN) that execute on the entire heuristic heterogeneous graph, reinforcing global-level significant consistency evidence. Finally, the consistency fusion layer is devised to weaken the disagreement between the three types of evidence to assist in capturing consistent shared evidence for verifying claims. Experiments on TABFACT and FEVEROUS demonstrate the effectiveness of H2GRN.
Recurrent Meta-Learning against Generalized Cold-start Problem in CTR Prediction
Authors: Junyu Chen, Qianqian Xu, Zhiyong Yang, Ke Ma, Xiaochun Cao, Qingming Huang
Date: 2022-10-01 00:00:00
Proceeding: Conference 2022 10
Tasks: Click-Through Rate Prediction, Meta-Learning

During the last decades, great success has been witnessed along the course of accurate Click-Through-Rate (CTR) prediction models for online advertising. However, the cold-start problem, which refers to the issue that the standard models can hardly draw accurate inferences for unseen users/ads, is still yet to be fully understood. Most recently, some related studies have been proposed to tackle this problem with only the new users/ads being considered. We argue that such new users/ads are not the only sources for cold-start. From another perspective, since users might shift their interests over time, one's recent behaviors might vary greatly from the records long ago. In this sense, we believe that the cold-start problem should also exist along the temporal dimension. Motivated by this, a generalized definition of the cold-start problem is provided where both new users/ads and recent behavioral data from known users are considered. To attack this problem, we propose a recursive meta-learning model with the user's behavior sequence prediction as a separate training task. Specifically, a time-series CTR model with the MAML (Model-Agnostic Meta-Learning)-like meta-learning method is proposed to make our model adapt to new tasks rapidly. Besides, we propose a parallel structure for extracting the feature interactions to efficiently fuse attention mechanisms and the RNN layer. Finally, experiments on three public datasets demonstrate the effectiveness of the proposed approaches.
ConCur: Self-supervised graph representation based on contrastive learning with curriculum negative sampling
Authors: Rong Yan, Peng Bao
Date: 2023-09-01 00:00:00
Proceeding: journal 2023 9
Tasks: Contrastive Learning, Graph Representation Learning, Node Classification, Representation Learning, Triplet

Contrastive learning has made breakthrough advancements in graph representation learning, which encourages the representation of positive samples to be close and those of negative samples to be far away. However, existing graph contrastive learning (GCL) frameworks have made great efforts toward designing different augmentation strategies for positive samples, while randomly utilizing all other nodes as negative samples and treating them equally, completely ignoring the differences between negative samples. Moreover, almost every GCL framework replaces original graph with different augmented views, which may lead to unexpected information missing caused by randomly perturbing edges and features. To address these issues, we propose a self-supervised graph Contrastive learning framework with Curriculum negative sampling, called ConCur, which feeds negative samples in an easy-to-hard fashion for contrastive learning by performing our proposed curriculum negative sampling strategy. More specifically, ConCur consists of two phases: Graph Augmentations and Curriculum Contrastive Training. Graph Augmentations aim at constructing positive and negative samples through different graph augmentation strategies. In the Curriculum Contrastive Training, we first utilize a triplet network to learn node representations by receiving original graph and different augmented views as input. Then, we propose a curriculum negative sampling strategy to enumerate negative samples from easy to hard for contrastive training. Finally, we utilize a unified contrastive loss to optimize node representations. Comprehensive experiments on five real-world datasets reveal that ConCur yields substantial relative encouraging results on the node classification task.
Learning node representation via Motif Coarsening
Authors: Junyu Chen, Qianqian Xu, Zhiyong Yang, Ke Ma, Xiaochun Cao, Qingming Huang
Date: 2023-10-01 00:00:00
Proceeding: journal 2023 10
Tasks: Graph Representation Learning, Representation Learning

Motifs, as fundamental units of the graph, play a significant role in modeling complex systems in a variety of domains, including social networks, as well as biology and neuroscience. Motif preservation is a widely studied problem that provides new avenues for structure preservation. This paper is dedicated to exploring the significance of motifs with various patterns and effectively incorporating different motif patterns into node-level graph representation learning. We propose a novel node Representation learning framework via Motif Coarsening (RMC), which effectively incorporates different granularity structural information into node representation learning. RMC consists of two parallel components, the node representation learning aggregator and the motif-based node representation learning aggregator. In the node representation learning process, RMC directly encodes lower-order structures into node representation by a one-layer graph convolution network. For the motif-based node representation learning process, we propose a Motif Coarsening strategy for incorporating motif structure into the graph representation learning process. Furthermore, the MotifRe-Weighting strategy is proposed to biased convert motif representation into motif-based node representation. We verify the effectiveness of RMC by several node-related tasks on a series of widely used real-world datasets. Experimental results demonstrate that our proposed framework delivers superior promising representation performance to existing benchmarks. Ablation experiments proved that RMC has potential as an auxiliary framework, which indicates the excellent quality of Motif Coarsening and MotifRe-Weighting strategies over existing benchmarks from several evaluation metrics, involving mean classification accuracy, Micro-F1, and Macro-F1.
DyTSCL: Dynamic graph representation via tempo-structural contrastive learning
Authors: Jianian Li, Peng Bao, Rong Yan, HuaWei Shen
Date: 2023-11-01 00:00:00
Proceeding: journal 2023 11
Tasks: Contrastive Learning, Graph Learning, Graph Representation Learning, Representation Learning

With the massive growth of graph-structured data, extensive research has focused on graph representation learning. Recently, graph representation learning frameworks have made great efforts toward dynamic graph learning. Although dynamic graph methods have achieved impressive results, they require labeled data for model training. The contrastive learning does not require human annotation to complete model training and has been shown to be extremely competitive in visual representation learning and natural language processing. In this paper, we propose a novel Dynamic graph representation framework via Tempo-Structural Contrastive Learning, DyTSCL, which trains the model by identifying three different subgraphs as a task, named Tempo-Structural subgraph, Non-Temporal subgraph and Non-Structural subgraph. Moreover, we propose a Tempo-Structural encoder, which aggregates the temporal and structural information. Finally, a Tempo-Structural contrastive learning module is proposed to maximize the consistency between node and subgraph in temporal and structural perspectives, respectively. To demonstrate the effectiveness of DyTSCL, we validate DyTSCL by applying it on the Wikipedia, Reddit and Mooc datasets, which show that DyTSCL can significantly outperform the existing approaches.
Multi-Source Templates Learning for Real-Time Aerial Tracking
Authors: Yiming Sun; Yang Li; Changbo Wang
Date: 2023-10-01 00:00:00
Proceeding: Conference 2023 10
Tasks: CPU, GPU, Object

Aerial tracking aims at tracking an arbitrary visual object in a video captured by Unmanned Aerial Vehicles (UAV). Due to the scarce computation resources, the deployment of high-consuming state-of-the-art trackers on UAV becomes impractical. On the other hand, lightweight trackers suffer from inferior performance caused by the low sampling frequency and resolution of UAV videos. In this paper, we propose a novel multi-source templates learning method to alleviate the paradox of efficiency and effectiveness for aerial tracking. Besides conventional static and dynamic templates, our work introduces an additional general-object template to learn common feature properties of a general object during training time. To exploit all templates information, a multi-source templates fusion scheme is proposed to capture characteristics of object in low quality UAV video streams. Furthermore, a joint optimization process is employed to enforce the lightness of model while achieving comparable tracking performance. Our experimental results demonstrate an appealing performance trade-off between accuracy and speed. The proposed tracker achieves 200 FPS on GPU, 100 FPS on CPU, and 12 FPS on Nvidia Jetson Xavier NX, respectively. Our code will be released at https://github.com/vpx-ecnu/MSTL.
Contact-conditioned hand-held object reconstruction from single-view images
Authors: Xiaoyuan Wang, Yang Li, Adnane Boukhayma, Changbo Wang, Marc Christie
Date: 2023-08-01 00:00:00
Proceeding: journal 2023 8
Tasks: Object, Object Reconstruction

Reconstructing the shape of hand-held objects from single-view color images is a long-standing problem in computer vision and computer graphics. The task is complicated by the ill-posed nature of single-view reconstruction, as well as potential occlusions due to both the hand and the object. Previous works mostly handled the problem by utilizing known object templates as priors to reduce the complexity. In contrast, our paper proposes a novel approach without knowing the object templates beforehand but by exploiting prior knowledge of contacts in hand-object interactions to train an attention-based network that can perform precise hand-held object reconstructions with only a single forward pass in inference. The network we propose encodes visual features together with contact features using a multi-head attention module as a way to condition the training of a neural field representation. This neural field representation outputs a Signed Distance Field representing the reconstructed object and extensive experiments on three well-known datasets demonstrate that our method achieves superior reconstruction results even under severe occlusion compared to the state-of-the-art techniques.
AdaptMVSNet: Efficient Multi-View Stereo with adaptive convolution and attention fusion
Authors: Pengfei Jiang, Xiaoyan Yang, Yuanjie Chen, Wenjie Song, Yang Li
Date: 2023-11-01 00:00:00
Proceeding: journal 2023 11

Multi-View Stereo (MVS) is a crucial technique for reconstructing the geometric structure of a scene, given the known camera parameters. Previous deep learning-based MVS methods have mainly focused on improving the reconstruction quality but overlooked the running efficiency during the actual algorithm deployment. For example, deformable convolutions have been introduced to improve the accuracy of the reconstruction results further, however, its inability for parallel optimization caused low inference speed. In this paper, we propose AdaptMVSNet which is device-friendly and reconstruction-efficient, while preserving the original results. To this end, adaptive convolution is introduced to significantly improve the efficiency in speed and metrics compared to current methods. In addition, an attention fusion module is proposed to blend features from adaptive convolution and the feature pyramid network. Our experiments demonstrate that our proposed approach achieves state-of-the-art performance and is almost 2
 faster than the recent fastest MVS method. We will release our source code.
A Sequential Flow Control Framework for Multi-hop Knowledge Base Question Answering
Authors: Minghui Xie, Chuzhan Hao, Peng Zhang
Date: 2022-12-01 00:00:00
Proceeding: Conference 2022 12
Tasks: Knowledge Base Question Answering, Question Answering

One of the key challenges of knowledge base question answering (KBQA) is the multi-hop reasoning. Since in different hops, one attends to different parts of question, it is important to dynamically represent the question semantics for each hop. Existing methods, however, (i) infer the dynamic question representation only through coarse-grained attention mechanisms, which may bring information loss, (ii) and have not effectively modeled the sequential logic, which is crucial for the multi-hop reasoning process in KBQA.To address these issues, we propose a sequential reasoning self-attention mechanism to capture the crucial reasoning information of each single hop in a more fine-grained way. Based on Gated Recurrent Unit (GRU) which is good at modeling sequential process, we propose a simple but effective GRU-inspired Flow Control (GFC) framework to model sequential logic in the whole multi-hop process. Extensive experiments on three popular benchmark datasets have demonstrated the superior effectiveness of our model. In particular, GFC achieves new state-of-the-art Hits@1 of 76.8% on WebQSP and is also effective when KB is incomplete. Our code and data are available at https://github.com/Xie-Minghui/GFC.
A Measurement-Based Quantum-Like Language Model for Text Matching
Authors: Wantong Zhang, Guobing Gan, Hui Gao, Peng Zhang, Wenjie Hui & Zipeng Fan
Date: 2023-04-01 00:00:00
Proceeding: Conference 2023 4
Tasks: Language Modeling, Language Modelling, Sentence, Text Matching

Quantum language models have attracted extensive attention in natural language processing tasks. Nevertheless, in this field measurement operators are mostly generated by randomly initialized parameter matrices, which cannot well explain the role of measurement operators in quantum theory. In this paper, we propose a Measurement-Based Quantum-like Language Model (MBQLM). Specifically, each word is considered a fundamental event in quantum probability space, which is a quantum state represented by a density matrix. We take the word density matrix in one sentence as a set of measurement operators to measure another sentence, which is consistent with the definition of measurement operators in quantum theory and has a specific semantic interpretation. The measured sentence state matrix representation effectively interact the feature information between sentences. To evaluate the performance of our model, we conduct experiments on WikiQA and TREC-QA datasets. The results show that our model achieves better performance than all benchmarks.
TIDE: Affective Time-aware Representations for Fine-grained Depression Identification on Social Media
Authors: Zhuanzhuan Liu; Xing Ma; Peng Zhang; Chuzhan Hao; Shuo Zhang; Lin Wang
Date: 2023-06-01 00:00:00
Proceeding: Conference 2023 6

The growing availability of the Internet provides opportunities for depression screening. In recent years, depression analysis based on social media texts has shown great promise, yet most works have focused on treating it as a binary problem. Meanwhile, existing methods need the perception of fine-grained emotions in historical contexts. Moreover, they do not consider the time interval between posts and thus ignore the decay of emotions over time. In this paper, we propose a Time-aware Depression identification model based on Emotion capturing (TIDE), a novel method to identify the severity of depression in social media users. We define the assessment as an ordinal regression problem to distinguish differences in depression levels. Specifically, TIDE uses Plutchik's wheel of emotions to characterize the emotional historical spectrum, then models the emotion decay process of historical context using a Time-aware LSTM (T-LSTM). We experiment on two Reddit public datasets to demonstrate that our approach outperforms state-of-the-art models. Then, ablation studies and qualitative analysis further demonstrate the validity of the proposed modules. Overall, our work can facilitate social media-based analysis of depression and shows potential for application to mental health-related issues.
Proceedings of the 40th International Conference on Machine Learning
Authors: Huan Ma, Qingyang Zhang, Changqing Zhang, Bingzhe Wu, Huazhu Fu, Joey Tianyi Zhou, QinGhua Hu
Date: 2023-07-01 00:00:00
Proceeding: journal 2023 7

Multimodal machine learning has achieved remarkable progress in a wide range of scenarios. However, the reliability of multimodal learning remains largely unexplored. In this paper, through extensive empirical studies, we identify current multimodal classification methods suffer from unreliable predictive confidence that tend to rely on partial modalities when estimating confidence. Specifically, we find that the confidence estimated by current models could even increase when some modalities are corrupted. To address the issue, we introduce an intuitive principle for multimodal learning, i.e., the confidence should not increase when one modality is removed. Accordingly, we propose a novel regularization technique, i.e., Calibrating Multimodal Learning (CML) regularization, to calibrate the predictive confidence of previous methods. This technique could be flexibly equipped by existing models and improve the performance in terms of confidence calibration, classification accuracy, and model robustness.
PacketGame: Multi-Stream Packet Gating for Concurrent Video Inference at Scale
Authors: Mu Yuan, Lan Zhang, Xuanke You, Xiang-Yang Li
Date: 2023-09-01 00:00:00
Proceeding: journal 2023 9
Tasks: Decoder, Video Compression

The resource efficiency of video analytics workloads is critical for large-scale deployments on edge nodes and cloud clusters. Recent advanced systems have benefited from techniques including video compression, frame filtering, and deep model acceleration. However, based on our year-long experience of operating a real-time video analytics system on more than 1000 cameras, we identified a previously overlooked bottleneck of end-to-end concurrency: video decoding. To support concurrent video inference at scale, in this work, we investigate a new task, named video packet gating, which selectively filters packets before running a decoder. We propose a novel multi-view embedding approach for video packets and present PacketGame that has both theoretical performance guarantee and practical system designs. Experiments on both public datasets and a real system show PacketGame saves 52.0--79.3% decoding costs and achieves 2.1--4.8× concurrency compared to original workloads. Comparisons with four state-of-the-art complementary methods show the superiority of PacketGame in end-to-end concurrency.