GenAI4ELab/papercli-papers huggingface dataset:
Evaluating Robustness of Vision Transformers on Imbalanced Datasets (Student Abstract)
Authors: Kevin Li; Rahul Duggal; Duen Horng Chau
Venue: AAAI 2023 (main)

Abstract: Data in the real world is commonly imbalanced across classes. Training neural networks on imbalanced datasets often leads to poor performance on rare classes. Existing work in this area has primarily focused on Convolution Neural Networks (CNN), which are increasingly being replaced by Self-Attention-based Vision Transformers (ViT). Fundamentally, ViTs differ from CNNs in that they offer the flexibility in learning the appropriate inductive bias conducive to improving performance. This work is among the first to evaluate the performance of ViTs under class imbalance. We find that accuracy degradation in the presence of class imbalance is much more prominent in ViTs compared to CNNs. This degradation can be partially mitigated through loss reweighting - a popular strategy that increases the loss contributed by rare classes. We investigate the impact of loss reweighting on different components of a ViT, namely, the patch embedding, self-attention backbone, and linear classifier. Our ongoing investigations reveal that loss reweighting impacts mostly the linear classifier and self-attention backbone while having a small and negligible effect on the embedding layer.
On Analyzing the Role of Image for Visual-Enhanced Relation Extraction (Student Abstract)
Authors: Lei Li; Xiang Chen; Shuofei Qiao; Feiyu Xiong; Huajun Chen; Ningyu Zhang
Venue: AAAI 2023 (main)

Abstract: Multimodal relation extraction is an essential task for knowledge graph construction. In this paper, we take an in-depth empirical analysis that indicates the inaccurate information in the visual scene graph leads to poor modal alignment weights, further degrading performance. Moreover, the visual shuffle experiments illustrate that the current approaches may not take full advantage of visual information. Based on the above observation, we further propose a strong baseline with an implicit fine-grained multimodal alignment based on Transformer for multimodal relation extraction. Experimental results demonstrate the better performance of our method. Codes are available at https://github.com/zjunlp/DeepKE/tree/main/example/re/multimodal.
Double Policy Network for Aspect Sentiment Triplet Extraction (Student Abstract)
Authors: Xuting Li; Daifeng Li; Ruo Du; Dingquan Chen; Andrew Madden
Venue: AAAI 2023 (main)

Abstract: Aspect Sentiment Triplet Extraction (ASTE) is the task to extract aspects, opinions and associated sentiments from sentences. Previous studies do not adequately consider the complicated interactions between aspect and opinion terms in both extraction logic and strategy. We present a novel Double Policy Network with Multi-Tag based Reward model (DPN-MTR), which adopts two networks ATE, TSOTE and a Trigger Mechanism to execute ASTE task following a more logical framework. A Multi-Tag based reward is also proposed to solve the limitations of existing studies for identifying aspect/opinion terms with multiple tokens (one term may consist of two or more tokens) to a certain extent. Extensive experiments are conducted on four widely-used benchmark datasets, and demonstrate the effectiveness of our model in generally improving the performance on ASTE significantly.
Learning Generalizable Batch Active Learning Strategies via Deep Q-networks (Student Abstract)
Authors: Yi-Chen Li; Wen-Jie Shen; Boyu Zhang; Feng Mao; Zongzhang Zhang; Yang Yu
Venue: AAAI 2023 (main)

Abstract: To handle a large amount of unlabeled data, batch active learning (BAL) queries humans for the labels of a batch of the most valuable data points at every round. Most current BAL strategies are based on human-designed heuristics, such as uncertainty sampling or mutual information maximization. However, there exists a disagreement between these heuristics and the ultimate goal of BAL, i.e., optimizing the model's final performance within the query budgets. This disagreement leads to a limited generality of these heuristics. To this end, we formulate BAL as an MDP and propose a data-driven approach based on deep reinforcement learning. Our method learns the BAL strategy by maximizing the model's final performance. Experiments on the UCI benchmark show that our method can achieve competitive performance compared to existing heuristics-based approaches.
Cross-Regional Fraud Detection via Continual Learning (Student Abstract)
Authors: Yujie Li; Yuxuan Yang; Qiang Gao; Xin Yang
Venue: AAAI 2023 (main)

Abstract: Detecting fraud is an urgent task to avoid transaction risks. Especially when expanding a business to new cities or new countries, developing a totally new model will bring the cost issue and result in forgetting previous knowledge. This study proposes a novel solution based on heterogeneous trade graphs, namely HTG-CFD, to prevent knowledge forgetting of cross-regional fraud detection. Specifically, a novel heterogeneous trade graph is meticulously constructed from original transactions to explore the complex semantics among different types of entities and relationships. Motivated by continual learning, we present a practical and task-oriented forgetting prevention method to alleviate knowledge forgetting in the context of cross-regional detection. Extensive experiments demonstrate that HTG-CFD promotes performance in both cross-regional and single-regional scenarios.
Category-Guided Visual Question Generation (Student Abstract)
Authors: Hongfei Liu; Jiali Chen; Wenhao Fang; Jiayuan Xie; Yi Cai
Venue: AAAI 2023 (main)

Abstract: Visual question generation aims to generate high-quality questions related to images. Generating questions based only on images can better reduce labor costs and thus be easily applied. However, their methods tend to generate similar general questions that fail to ask questions about the specific content of each image scene. In this paper, we propose a category-guided visual question generation model that can generate questions with multiple categories that focus on different objects in an image. Specifically, our model first selects the appropriate question category based on the objects in the image and the relationships among objects. Then, we generate corresponding questions based on the selected question categories. Experiments conducted on the TDIUC dataset show that our proposed model outperforms existing models in terms of diversity and quality.
Can Graph Neural Networks Learn to Solve the MaxSAT Problem? (Student Abstract)
Authors: Minghao Liu; Pei Huang; Fuqi Jia; Fan Zhang; Yuchen Sun; Shaowei Cai; Feifei Ma; Jian Zhang
Venue: AAAI 2023 (main)

Abstract: The paper presents an attempt to bridge the gap between machine learning and symbolic reasoning. We build graph neural networks (GNNs) to predict the solution of the Maximum Satisfiability (MaxSAT) problem, an optimization variant of SAT. Two closely related graph representations are adopted, and we prove their theoretical equivalence. We also show that GNNs can achieve attractive performance to solve hard MaxSAT problems in certain distributions even compared with state-of-the-art solvers through experimental evaluation.
Flaky Performances When Pretraining on Relational Databases (Student Abstract)
Authors: Shengchao Liu; David Vazquez; Jian Tang; Pierre-André Noël
Venue: AAAI 2023 (main)

Abstract: We explore the downstream task performances for graph neural network (GNN) self-supervised learning (SSL) methods trained on subgraphs extracted from relational databases (RDBs). Intuitively, this joint use of SSL and GNNs should allow to leverage more of the available data, which could translate to better results. However, we found that naively porting contrastive SSL techniques can cause ``negative transfer'': linear evaluation on fixed representation from a pretrained model performs worse than on representations from the randomly-initialized model. Based on the conjecture that contrastive SSL conflicts with the message passing layers of the GNN, we propose InfoNode: a contrastive loss aiming to maximize the mutual information between a node's initial- and final-layer representation. The primary empirical results support our conjecture and the effectiveness of InfoNode.
A Highly Efficient Marine Mammals Classifier Based on a Cross-Covariance Attended Compact Feed-Forward Sequential Memory Network (Student Abstract)
Authors: Xiangrui Liu; Julian Cheng
Venue: AAAI 2023 (main)

Abstract: Military active sonar and marine transportation are detrimental to the livelihood of marine mammals and the ecosystem. Early detection and classification of marine mammals using machine learning can help humans to mitigate the harm to marine mammals. This paper proposes a cross-covariance attended compact Feed-Forward Sequential Memory Network (CC-FSMN). The proposed framework shows improved efficiency over multiple convolutional neural network (CNN) backbones. It also maintains a relatively decent performance.
MGIA: Mutual Gradient Inversion Attack in Multi-Modal Federated Learning (Student Abstract)
Authors: Xuan Liu; Siqi Cai; Lin Li; Rui Zhang; Song Guo
Venue: AAAI 2023 (main)

Abstract: Recent studies have demonstrated that local training data in Federated Learning can be recovered from gradients, which are called gradient inversion attacks. These attacks display powerful effects on either computer vision or natural language processing tasks. As it is known that there are certain correlations between multi-modality data, we argue that the threat of such attacks combined with Multi-modal Learning may cause more severe effects. Different modalities may communicate through gradients to provide richer information for the attackers, thus improving the strength and efficiency of the gradient inversion attacks. In this paper, we propose the Mutual Gradient Inversion Attack (MGIA), by utilizing the shared labels between image and text modalities combined with the idea of knowledge distillation. Our experimental results show that MGIA achieves the best quality of both modality data and label recoveries in comparison with other methods. In the meanwhile, MGIA verifies that multi-modality gradient inversion attacks are more likely to disclose private information than the existing single-modality attacks.
Semi-supervised Review-Aware Rating Regression (Student Abstract)
Authors: Xiangkui Lu; Jun Wu
Venue: AAAI 2023 (main)

Abstract: Semi-supervised learning is a promising solution to mitigate data sparsity in review-aware rating regression (RaRR), but it bears the risk of learning with noisy pseudo-labelled data. In this paper, we propose a paradigm called co-training-teaching (CoT2), which integrates the merits of both co-training and co-teaching towards the robust semi-supervised RaRR. Concretely, CoT2 employs two predictors and each of them alternately plays the roles of  "labeler" and "validator" to generate and validate pseudo-labelled instances. Extensive experiments show that CoT2 considerably outperforms state-of-the-art RaRR techniques, especially when training data is severely insufficient.
Toplogical Data Analysis Detects and Classifies Sunspots (Student Abstract)
Authors: Aidan Lytle; Neil Pritchard; Alicia Aarnio; Thomas Weighill
Venue: AAAI 2023 (main)

Abstract: In our technology-dependent modern world, it is imperative to monitor the Sun for space weather threats to critical infrastructure. Topological data analysis (TDA) is a new set of mathematical techniques used in data analysis and machine learning. We demonstrate that TDA can robustly detect and classify solar surface and coronal activity. This technique is a promising step toward future application in predictive space weather modeling.
Risk-Aware Decentralized Safe Control via Dynamic Responsibility Allocation (Student Abstract)
Authors: Yiwei Lyu; Wenhao Luo; John M. Dolan
Venue: AAAI 2023 (main)

Abstract: In this work, we present a novel risk-aware decentralized Control Barrier Function (CBF)-based controller for multi-agent systems. The proposed decentralized controller is composed based on pairwise agent responsibility shares (a percentage), calculated from the risk evaluation of each individual agent faces in a multi-agent interaction environment. With our proposed CBF-inspired risk evaluation framework, the responsibility portions between pairwise agents are dynamically updated based on the relative risk they face. Our method allows agents with lower risk to enjoy a higher level of freedom in terms of a wider action space, and the agents exposed to higher risk are constrained more tightly on action spaces, and are therefore forced to proceed with caution.
A Mutually Enhanced Bidirectional Approach for Jointly Mining User Demand and Sentiment (Student Abstract)
Authors: Xue Mao; Haoda Qian; Minjie Yuan; Qiudan Li
Venue: AAAI 2023 (main)

Abstract: User demand mining aims to identify the implicit demand from the e-commerce reviews, which are always irregular, vague and diverse. Existing sentiment analysis research mainly focuses on aspect-opinion-sentiment triplet extraction, while the deeper user demands remain unexplored. In this paper, we formulate a novel research question of jointly mining aspect-opinion-sentiment-demand, and propose a Mutually Enhanced Bidirectional Extraction (MEMB) framework for capturing the dynamic interaction among different types of information. Finally, experiments on Chinese e-commerce data demonstrate the efficacy of the proposed model.
Debiasing Intrinsic Bias and Application Bias Jointly via Invariant Risk Minimization (Student Abstract)
Authors: Yuzhou Mao; Liu Yu; Yi Yang; Fan Zhou; Ting Zhong
Venue: AAAI 2023 (main)

Abstract: Demographic biases and social stereotypes are common in pretrained language models (PLMs), while the fine-tuning in downstream applications can also produce new biases or amplify the impact of the original biases. Existing works separate the debiasing from the fine-tuning procedure, which results in a gap between intrinsic bias and application bias. In this work, we propose a debiasing framework CauDebias to eliminate both biases, which directly combines debiasing with fine-tuning and can be applied for any PLMs in downstream tasks. We distinguish the bias-relevant (non-causal factors) and label-relevant (causal factors) parts in sentences from a causal invariant perspective. Specifically, we perform intervention on non-causal factors in different demographic groups, and then devise an invariant risk minimization loss to trade-off performance between bias mitigation and task accuracy. Experimental results on three downstream tasks show that our CauDebias can remarkably reduce biases in PLMs while minimizing the impact on downstream tasks.
Label Smoothing for Emotion Detection (Student Abstract)
Authors: George Maratos; Tiberiu Sosea; Cornelia Caragea
Venue: AAAI 2023 (main)

Abstract: Automatically detecting emotions from text has countless
applications, ranging from large scale opinion mining to
social robots in healthcare and education. However, emotions
are subjective in nature and are often expressed in ambiguous
ways. At the same time, detecting emotions can also require
implicit reasoning, which may not be available as surface-
level, lexical information. In this work, we conjecture that
the overconfidence of pre-trained language models such as
BERT is a critical problem in emotion detection and show
that alleviating this problem can considerably improve the
generalization performance. We carry out comprehensive
experiments on four emotion detection benchmark datasets
and show that calibrating our model predictions leads to an
average improvement of 1.35% in weighted F1 score.
Counting Knot Mosaics with ALLSAT (Student Abstract)
Authors: Hannah Miller
Venue: AAAI 2023 (main)

Abstract: Knot mosaics are a model of a quantum knot system.  A knot mosaic is a m-by-n grid where each location on the grid may contain any of 11 possible tiles such that the final layout has closed loops.  Oh et al. proved a recurrence relation of state matrices to count the number of m-by-n knot mosaics.  Our contribution is to use ALLSAT solvers to count knot mosaics and to experimentally try different ways to encode the AT MOST ONE constraint in SAT.  We plan to use our SAT method as a tool to list knot mosaics of interest for specific classes of knots.
Novel Intent Detection and Active Learning Based Classification (Student Abstract)
Authors: Ankan Mullick
Venue: AAAI 2023 (main)

Abstract: Novel intent class detection is an important problem in real world scenario for conversational agents for continuous interaction. Several research works have been done to detect novel intents in a mono-lingual (primarily English) texts and
images. But, current systems lack an end-to-end universal framework to detect novel intents across various different languages with less human annotation effort for mis-classified and system rejected samples. This paper proposes
NIDAL (Novel Intent Detection and Active Learning based
classification), a semi-supervised framework to detect novel
intents while reducing human annotation cost. Empirical results on various benchmark datasets demonstrate that this system outperforms the baseline methods by more than 10%
margin for accuracy and macro-F1. The system achieves this while maintaining overall annotation cost to be just ~6-10% of the unlabeled data available to the system.
Pre-training with Scientific Text Improves Educational Question Generation (Student Abstract)
Authors: Hamze Muse; Sahan Bulathwela; Emine Yilmaz
Venue: AAAI 2023 (main)

Abstract: With the boom of digital educational materials and scalable e-learning systems, the potential for realising AI-assisted personalised learning has skyrocketed. In this landscape, the automatic generation of educational questions will play a key role, enabling scalable self-assessment when a global population is manoeuvring their personalised learning journeys. We develop EduQG, a novel educational question generation model built by adapting a large language model. Our initial experiments demonstrate that EduQG can produce superior educational questions by pre-training on scientific text.
Fraud’s Bargain Attacks to Textual Classifiers via Metropolis-Hasting Sampling (Student Abstract)
Authors: Mingze Ni; Zhensu Sun; Wei Liu
Venue: AAAI 2023 (main)

Abstract: Recent studies on adversarial examples expose vulnerabilities of natural language processing (NLP) models. Existing techniques for generating adversarial examples are typically driven by deterministic heuristic rules that are agnostic to the optimal adversarial examples, a strategy that often results in attack failures. To this end, this research proposes Fraud's Bargain Attack (FBA), which utilizes a novel randomization mechanism to enlarge the searching space and enables high-quality adversarial examples to be generated with high probabilities. FBA applies the Metropolis-Hasting algorithm to enhance the selection of adversarial examples from all candidates proposed by a customized Word Manipulation Process (WMP). WMP perturbs one word at a time via insertion, removal, or substitution in a contextual-aware manner. Extensive experiments demonstrate that FBA outperforms the baselines in terms of attack success rate and imperceptibility.
Improving Adversarial Robustness to Sensitivity and Invariance Attacks with Deep Metric Learning (Student Abstract)
Authors: Anaelia Ovalle; Evan Czyzycki; Cho-Jui Hsieh
Venue: AAAI 2023 (main)

Abstract: Intentionally crafted adversarial samples have effectively exploited weaknesses in deep neural networks. A standard method in adversarial robustness assumes a framework to defend against samples crafted by minimally perturbing a sample such that its corresponding model output changes. These sensitivity attacks exploit the model's sensitivity toward task-irrelevant features. Another form of adversarial sample can be crafted via invariance attacks, which exploit the model underestimating the importance of relevant features. Previous literature has indicated a tradeoff in defending against both attack types within a strictly L-p bounded defense. To promote robustness toward both types of attacks beyond Euclidean distance metrics, we use metric learning to frame adversarial regularization as an optimal transport problem. Our preliminary results indicate that regularizing over invariant perturbations in our framework improves both invariant and sensitivity defense.
LVRNet: Lightweight Image Restoration for Aerial Images under Low Visibility (Student Abstract)
Authors: Esha Pahwa; Achleshwar Luthra; Pratik Narang
Venue: AAAI 2023 (main)

Abstract: Learning to recover clear images from images having a combination of degrading factors is a challenging task. That being said, autonomous surveillance in low visibility conditions caused by high pollution/smoke, poor air quality index, low light, atmospheric scattering, and haze during a blizzard, etc, becomes even more important to prevent accidents. It is thus crucial to form a solution that can not only result in a high-quality image but also which is efficient enough to be deployed for everyday use. However, the lack of proper datasets available to tackle this task limits the performance of the previous methods proposed. To this end, we generate the LowVis-AFO dataset, containing 3647 paired dark-hazy and clear images. We also introduce a new lightweight deep learning model called Low-Visibility Restoration Network (LVRNet). It outperforms previous image restoration methods with low latency, achieving a PSNR value of 25.744 and an SSIM of 0.905, hence making our approach scalable and ready for practical use.
Hardness of Learning AES Key (Student Abstract)
Authors: Artur Pak; Sultan Nurmukhamedov; Rustem Takhanov; Zhenisbek Assylbekov
Venue: AAAI 2023 (main)

Abstract: We show hardness of learning AES key from pairs of ciphertexts under the assumption of computational closeness of AES to pairwise independence. The latter is motivated by a recent result on statistical closeness of AES to pairwise independence.
Generative Pipeline for Data Augmentation of Unconstrained Document Images with Structural and Textural Degradation (Student Abstract)
Authors: Arnab Poddar; Abhishek Kumar Sah; Soumyadeep Dey; Pratik Jawanpuria; Jayanta Mukhopadhyay; Prabir Kumar Biswas
Venue: AAAI 2023 (main)

Abstract: Computer vision applications for document image understanding (DIU) such as optical character recognition, word spotting, enhancement etc. suffer  from structural deformations like strike-outs and unconstrained strokes, to name a few. They also suffer from texture degradation due to blurring, aging, or  blotting-spots etc. 
The DIU applications with deep networks are limited to constrained environment and lack diverse data with text-level and pixel-level annotation simultaneously. In this work, we  propose a generative framework to produce realistic synthetic handwritten document images with simultaneous annotation of text and corresponding pixel-level spatial foreground information. The proposed approach generates realistic backgrounds with artificial handwritten texts which supplements data-augmentation in multiple unconstrained DIU systems.  The proposed framework is an early work to facilitate DIU system-evaluation in both image quality and recognition performance at a go.