bluuebunny/crossref_metadata_2025 huggingface dataset:
Identifying Early Warning Signals from News Using Network Community Detection
Authors: Le Vine Nataliya , Boxer Eric , Dinani Mustafa , Tortora Paolo , Das Subhradeep
Published: July 2022

The paper addresses the challenge of accelerating identification of changes in risk drivers in the insurance industry. Specifically, the work presents a method to identify significant news events ("signals") from batches of news data to inform Life & Health insurance decisions. Signals are defined as events that are relevant to a tracked risk driver, widely discussed in multiple news outlets, contain novel information and affect stakeholders. The method converts unstructured data (news articles) into a sequence of keywords by employing a linguistic knowledge graph-based model. Then, for each time window, the method forms a graph with extracted keywords as nodes and draws weighted edges based on keyword co-occurrences in articles. Lastly, events are derived in an unsupervised way as graph communities and scored for the requirements of a signal: relevance, novelty and virality. The methodology is illustrated for a Life & Health topic using news articles from Dow Jones DNA proprietary data set, and assessed against baselines on a publicly available news data set. The method is implemented as an analytics engine in Early Warning System deployed at Swiss Re for the last 1.5 years to extract relevant events from live news data. We present the system's architectural design in production and discuss its use and impact.
Deploying an Artificial Intelligence Application to Detect Flood from Sentinel 1 Data
Authors: Fraccaro Paolo , Stoyanov Nikola , Gaffoor Zaheed , Cue La Rosa Laura Elena , Singh Jitendra , Ishikawa Tatsuya , Edwards Blair , Jones Anne , Weldermariam Komminist
Published: July 2022

As climate change is increasing the frequency and intensity of climate and weather hazards, improving detection and monitoring of flood events is a priority. Being weather independent and high resolution, Sentinel 1 (S1) radar satellite imagery data has become the go to data source to detect flood events accurately. However, current methods are either based on fixed thresholds to differentiate water from land or train Artificial Intelligence (AI) models based on only S1 data, despite the availability of many other relevant data sources publicly. These models also lack comprehensive validations on out-of-sample data and deployment at scale. In this study, we investigated whether adding extra input layers could increase the performance of AI models in detecting floods from S1 data. We also provide performance across a range of 11 historical events, with results ranging between 0.93 and 0.97 accuracy, 0.53 and 0.81 IoU, and 0.68 and 0.89 F1 scores. Finally, we show the infrastructure we developed to deploy our AI models at scale to satisfy a range of use cases and user requests.
PaintTeR: Automatic Extraction of Text Spans for Generating Art-Centered Questions
Authors: Gollapalli Sujatha Das , Ng See-Kiong , Tham Ying Kiat , Chow Shan Shan , Wong Jia Min , Lim Kevin
Published: July 2022

We propose PaintTeR, our Paintings TextRank algorithm for extracting art-related text spans from passages on paintings. PaintTeR combines a lexicon of painting words curated automatically through distant supervision with random walks on a large-scale word co-occurrence graph for ranking passage spans for artistic characteristics. The spans extracted with PaintTeR are used in state-of-the-art Question Generation and Reading Comprehension models for designing an interactive aid that enables gallery and museum visitors focus on the artistic elements of paintings. We provide experiments on two datasets of expert-written passages on paintings to showcase the effectiveness of PaintTeR. Evaluations by both gallery experts as well as crowdworkers indicate that our proposed algorithm can be used to
select relevant and interesting art-centered questions. To the best of our knowledge, ours is the first work to effectively fine-tune question generation models using minimal supervision for a low-resource, specialized context such as gallery visits.
Predictive Maintenance for General Aviation Using Convolutional Transformers
Authors: Yang Hong , LaBella Aidan , Desell Travis
Published: July 2022

Predictive maintenance systems have the potential to significantly reduce costs for maintaining aircraft fleets as well as provide improved safety by detecting maintenance issues before they come severe. However, the development of such systems has been limited due to a lack of publicly labeled multivariate time series (MTS) sensor data. MTS classification has advanced greatly over the past decade, but there is a lack of sufficiently challenging benchmarks for new methods. This work introduces the NGAFID Maintenance Classification (NGAFID-MC) dataset as a novel benchmark in terms of difficulty, number of samples, and sequence length. NGAFID-MC consists of over 7,500 labeled flights, representing over 11,500 hours of per second flight data recorder readings of 23 sensor parameters. Using this benchmark, we demonstrate that Recurrent Neural Network (RNN) methods are not well suited for capturing temporally distant relationships and propose a new architecture called Convolutional Multiheaded Self Attention (Conv-MHSA) that achieves greater classification performance at greater computational efficiency. We also demonstrate that image inspired augmentations of cutout, mixup, and cutmix, can be used to reduce overfitting and improve generalization in MTS classification. Our best trained models have been incorporated back into the NGAFID to allow users to potentially detect flights that require maintenance as well as provide feedback to further expand and refine the NGAFID-MC dataset.
Game Design for Better Security of Combination Locks
Authors: Astudillo Guerra Jean Pierre , Ahmed Karim , Maher Ryan , Ubri Eddie , Blum Jeremy
Published: July 2022

Dial locks are commonly used to secure a person’s items. Commercially available dial locks often use four or five wheels of letters, allowing a user to select a word as a combination. In order to evaluate the security of these locks, we create a game, with an instance created by the lock designer, and played by a lock owner and a thief. In the game, the lock owner chooses a word as a combination, and the thief creates a brute force strategy to try all possible combinations that yield words until the combination is found. To accomplish the task, the thief will solve a version of the Probabilistic Travelling Salesman Problem (PTSP) by creating an a priori tour through all the words a lock can create. The goal for the game designer, then, is to create a lock configuration that maximizes the expected length of the best possible PTSP tour. This paper describes a Genetic Algorithm (GA) approach to design a near-optimal game, i.e. a lock configuration that makes it as difficult for the thief to crack. An analysis of the output of the GA shows that the locks that the system creates are significantly more secure than both commercial locks, in the context of this game.
Game Balancing in Dominion: An Approach to Identifying Problematic Game Elements
Authors: Ford Cassandra , Ohata Merrick
Published: July 2022

In the popular card game Dominion, the configuration of game elements greatly affects the experience for players. If one were redesigning Dominion, therefore, it may be useful to identify game elements that reduce the number of viable strategies in any given game configuration - i.e. elements that are unbalanced. In this paper, we propose an approach that assigns credit to the outcome of an episode to individual elements. Our approach uses statistical analysis to learn the interactions and dependencies between game elements. This learned knowledge is used to recommend elements to game designers for further consideration. Designers may then choose to modify the recommended elements with the goal of increasing the number of viable strategies.
Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees
Authors: Singla Yaman Kumar , Krishna Sriram , Shah Rajiv Ratn , Chen Changyou
Published: July 2022

Automated Scoring (AS), the natural language processing task of scoring essays and speeches in an educational testing setting, is growing in popularity and being deployed across contexts from government examinations to companies providing language proficiency services. However, existing systems either forgo human raters entirely, thus harming the reliability of the test, or score every response by both human and machine thereby increasing costs. We target the spectrum of possible solutions in between, making use of both humans and machines to provide a higher quality test while keeping costs reasonable to democratize access to AS. In this work, we propose a combination of the existing paradigms, sampling responses to be scored by humans intelligently. We propose reward sampling and observe significant gains in accuracy (19.80% increase on average) and quadratic weighted kappa (QWK) (25.60% on average) with a relatively small human budget (30% samples) using our proposed sampling. The accuracy increase observed using standard random and importance sampling baselines are 8.6% and 12.2% respectively. Furthermore, we demonstrate the system's model agnostic nature by measuring its performance on a variety of models currently deployed in an AS setting as well as pseudo models. Finally, we propose an algorithm to estimate the accuracy/QWK with statistical guarantees (Our code is available at https://git.io/J1IOy).
Artificial Intelligence Approaches to Build Ticket to Ride Maps
Authors: Smith Iain , Anton Calin
Published: July 2022

Fun, as a game trait, is challenging to evaluate. Previous research explores game arc and game refinement to improve the quality of games. Fun, for some players, is having an even chance to win while executing their strategy. To explore this, we build boards for the game Ticket to Ride while optimizing for a given win rate between four AI agents. These agents execute popular strategies human players use: one-step thinking, long route exploitation, route focus, and destination hungry strategies. We create the underlying graph of a map by connecting several planar bipartite graphs. To build the map, we use a multiple phase design, with each phase implementing several simplified Monte Carlo Tree Search components. Within a phase, the components communicate with each other passively. The experiments show that the proposed approach results in improvements over randomly generated graphs and maps.
Teaching AI with the Hands-On AI Projects for the Classroom Series
Authors: Black Nancye Blair
Published: July 2022

The Hands-On AI Projects for the Classroom series, a collection of five guides, includes interactive projects that can be used by teachers across grade levels and subject areas to teach K-12 students about artificial intelligence (AI).
Towards Automating the Generation of Human-Robot Interaction Scenarios
Authors: Fontaine Matthew C.
Published: July 2022

My work studies the problem of generating scenarios to evaluate interaction between humans and robots. I expect these interactions to grow in complexity as robots become more intelligent and enter our daily lives. However, evaluating such interactions only through user studies, which are the de facto evaluation method in human-robot interaction, will quickly become infeasible as the number of possible scenarios grows exponentially with scenario complexity. Therefore, I propose automatically generating scenarios in simulation to explore the diverse possibility space of scenarios to better understand interaction and avoid costly failures in real world settings.
On the Practical Robustness of the Nesterov’s Accelerated Quasi-Newton Method
Authors: Indrapriyadarsini S. , Ninomiya Hiroshi , Kamio Takeshi , Asai Hideki
Published: July 2022

This study focuses on the Nesterov's accelerated quasi-Newton (NAQ) method in the context of deep neural networks (DNN) and its applications. The thesis objective is to confirm the robustness and efficiency of Nesterov's acceleration to quasi-Netwon (QN) methods by developing practical algorithms for different fields of optimization problems.
On Semantic Cognition, Inductive Generalization, and Language Models
Authors: Misra Kanishka
Published: July 2022

My doctoral research focuses on understanding semantic knowledge in neural network models trained solely to predict natural language (referred to as language models, or LMs), by drawing on insights from the study of concepts and categories grounded in cognitive science. I propose a framework inspired by 'inductive reasoning,' a phenomenon that sheds light on how humans utilize background knowledge to make inductive leaps and generalize from new pieces of information about concepts and their properties. Drawing from experiments that study inductive reasoning, I propose to analyze semantic inductive generalization in LMs using phenomena observed in human-induction literature, investigate inductive behavior on tasks such as implicit reasoning and emergent feature recognition, and analyze and relate induction dynamics to the learned conceptual representation space.
Learning Modular Structures That Generalize Out-of-Distribution (Student Abstract)
Authors: Ashok Arjun , Devaguptapu Chaitanya , Balasubramanian Vineeth N
Published: July 2022

Out-of-distribution (O.O.D.) generalization remains to be a key challenge for real-world machine learning systems. We describe a method for O.O.D. generalization that, through training, encourages models to only preserve features in the network that are well reused across multiple training domains. Our method combines two complementary neuron-level regularizers with a probabilistic differentiable binary mask over the network, to extract a modular sub-network that achieves better O.O.D. performance than the original network. Preliminary evaluation on two benchmark datasets corroborates the promise of our method.
INDEPROP: Information-Preserving De-propagandization of News Articles (Student Abstract)
Authors: Bhagat Aaryan , Mallick Faraaz , Karia Neel , Kaushal Ayush
Published: July 2022

We propose INDEPROP, a novel Natural Language Processing (NLP) application for combating online disinformation by mitigating propaganda from news articles. INDEPROP (Information-Preserving De-propagandization) involves fine-grained propaganda detection and its removal while maintaining document level coherence, grammatical correctness and most importantly, preserving the news articles’ information content. We curate the first large-scale dataset of its kind consisting of around 1M tokens. We also propose a set of automatic evaluation metrics for the same and observe its high correlation with human judgment. Furthermore, we show that fine-tuning the existing propaganda detection systems on our dataset considerably improves their generalization to the test set.
Controlling the Spread of Two Secrets in Diverse Social Networks (Student Abstract)
Authors: Blažej Václav , Knop Dušan , Schierreich Šimon
Published: July 2022

Information diffusion in social networks is a well-studied concept in social choice theory. We propose the study of the diffusion of two secrets in a heterogeneous environment from the complexity perspective, that is, there are two different networks with the same set of agents (e.g., the structure of the set of followers might be different in two distinct social networks).

Formally, our model combines two group identification processes for which we do have independent desiderata---either constructive, where we would like a given group of agents to be exposed to a secret, or destructive, where a given group of agents should not be exposed to a secret. To be able to reach these targets, we can either delete an agent or introduce a previously latent agent.

Our results are mostly negative---all of the problems are NP-hard. Therefore, we propose a parameterized study with respect to the natural parameters, the number of influenced agents, the size of the required/protected agent sets, and the duration of the diffusion process. Most of the studied problems remain W[1]-hard even for a combination of these parameters. We complement these results with nearly optimal XP algorithms.
Visual Explanations for Convolutional Neural Networks via Latent Traversal of Generative Adversarial Networks (Student Abstract)
Authors: Dravid Amil , Katsaggelos Aggelos K.
Published: July 2022

Lack of explainability in artificial intelligence, specifically deep neural networks, remains a bottleneck for implementing models in practice. Popular techniques such as Gradient-weighted Class Activation Mapping (Grad-CAM) provide a coarse map of salient features in an image, which rarely tells the whole story of what a convolutional neural network(CNN) learned. Using COVID-19 chest X-rays, we present a method for interpreting what a CNN has learned by utilizing Generative Adversarial Networks (GANs). Our GAN framework disentangles lung structure from COVID-19 features. Using this GAN, we can visualize the transition of a pair of COVID negative lungs in a chest radiograph to a COVID positive pair by interpolating in the latent space of the GAN, which provides fine-grained visualization of how the CNN responds to varying features within the lungs.
Wind Prediction under Random Data Corruption (Student Abstract)
Authors: Flansburg Conner , Diochnos Dimitrios I.
Published: July 2022

We study the robustness of ridge regression, lasso regression, and of a neural network, when the training set has been randomly corrupted and in response to this corruption the training-size is reduced in order to remove the corrupted data. While the neural network appears to be the most robust method among these three, nevertheless lasso regression appears to be the method of choice since it suffers less loss both when the full information is available to the learner, as well as when a significant amount of the original training set has been rendered useless because of random data corruption.
From Video to Images: Contrastive Pretraining for Emotion Recognition from Single Image (Student Abstract)
Authors: Garg Bhanu , Kim Kijun , Ranjan Sudhanshu
Published: July 2022

Emotion detection from face is an important problem and has received attention from industry and academia. Although emotion recognition from videos has a very high performance, emotion recognition from a single image stays a challenging task. In this paper, we try to use information from videos to do emotion recognition on a single image. More specifically, we leverage contrastive loss for pretraining the network on the videos and experiment with different sampling methods to select consistently hard triplets for continual learning of the network. Once the embeddings have been trained, we test them on a standard emotion classification task. Our method significantly improves the performance of the models and shows the efficacy of self-supervision in emotion recognition.
Deep Representation Debiasing via Mutual Information Minimization and Maximization (Student Abstract)
Authors: Han Ruijiang , Wang Wei , Long Yuxi , Peng Jiajie
Published: July 2022

Deep representation learning has succeeded in several fields. However, pre-trained deep representations are usually biased and make downstream models sensitive to different attributes. In this work, we propose a post-processing unsupervised deep representation debiasing algorithm, DeepMinMax, which can obtain unbiased representations directly from pre-trained representations without re-training or fine-tuning the entire model. The experimental results on synthetic and real-world datasets indicate that DeepMinMax outperforms the existing state-of-the-art algorithms on downstream tasks.
Class-Wise Adaptive Self Distillation for Federated Learning on Non-IID Data (Student Abstract)
Authors: He Yuting , Chen Yiqiang , Yang Xiaodong , Zhang Yingwei , Zeng Bixiao
Published: July 2022

Federated learning (FL) enables multiple clients to collaboratively train a globally generalized model while keeping local data decentralized. A key challenge in FL is to handle the heterogeneity of data distributions among clients. The local model will shift the global feature when fitting local data, which results in forgetting the global knowledge. Following the idea of knowledge distillation, the global model's prediction can be utilized to help local models preserve the global knowledge in FL. However, when the global model hasn't converged completely, its predictions tend to be less reliable on certain classes, which may results in distillation's misleading of local models. In this paper, we propose a class-wise adaptive self distillation (FedCAD) mechanism to ameliorate this problem. We design class-wise adaptive terms to soften the influence of distillation loss according to the global model's performance on each class and therefore avoid the misleading. Experiments show that our method outperforms other state-of-the-art FL algorithms on benchmark datasets.
A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract)
Authors: Indrapriyadarsini S. , Mahboubi Shahrzad , Ninomiya Hiroshi , Kamio Takeshi , Asai Hideki
Published: July 2022

Incorporating curvature information in stochastic methods has been a challenging task. This paper proposes a momentum accelerated BFGS quasi-Newton method in both its full and limited memory forms, for solving stochastic large scale non-convex optimization problems in neural networks (NN).
Switch-GPT: An Effective Method for Constrained Text Generation under Few-Shot Settings (Student Abstract)
Authors: Ma Chang , Zhang Song , Shen Gehui , Deng Zhihong
Published: July 2022

In real-world applications of natural language generation, target sentences are often required to satisfy some lexical constraints. However, the success of most neural-based models relies heavily on data, which is infeasible for data-scarce new domains. In this work, we present FewShotAmazon, the first benchmark for the task of Constrained Text Generation under few-shot settings on multiple domains. Further, we propose the Switch-GPT model, in which we utilize the strong language modeling capacity of GPT-2 to generate fluent and well-formulated sentences, while using a light attention module to decide which constraint to attend to at each step. Experiments show that the proposed Switch-GPT model is effective and remarkably outperforms the baselines. Codes will be available at https://github.com/chang-github-00/Switch-GPT.
Early Forecast of Traffic Accident Impact Based on a Single-Snapshot Observation (Student Abstract)
Authors: Meng Guangyu , Jiang Qisheng , Fu Kaiqun , Lin Beiyu , Lu Chang-Tien , Chen Zhiqian
Published: July 2022

Predicting and quantifying the impact of traffic accidents is necessary and critical to Intelligent Transport Systems (ITS). As a state-of-the-art technique in graph learning, current graph neural networks heavily rely on graph Fourier transform, assuming homophily among the neighborhood. However, the homophily assumption makes it challenging to characterize abrupt signals such as traffic accidents. Our paper proposes an abrupt graph wavelet network (AGWN) to model traffic accidents and predict their time durations using only one single snapshot.
Aspect-Opinion Sentiment Alignment for Cross-Domain Sentiment Analysis (Student Abstract)
Authors: Ren Haopeng , Cai Yi , Zeng Yushi
Published: July 2022

Cross-domain sentiment analysis (SA) has recently attracted significant attention, which can effectively alleviate the problem of lacking large-scale labeled data for deep neural network based methods. However, exiting unsupervised cross-domain SA models ignore the relation between the aspect and opinion, which suffer from the sentiment transfer error problem. To solve this problem, we propose an aspect-opinion sentiment alignment SA model and extensive experiments are conducted to evaluate the effectiveness of our model.
Do We Need a New Large-Scale Quality Assessment Database for Generative Inpainting Based 3D View Synthesis? (Student Abstract)
Authors: Sadbhawna  , Jakhetiya Vinit , Subudhi Badri N. , Shakya Harshit , Mumtaz Deebha
Published: July 2022

The advancement in Image-to-Image translation techniques using generative Deep Learning-based approaches has shown promising results for the challenging task of inpainting-based 3D view synthesis. At the same time, even the current 3D view synthesis methods often create distorted structures or blurry textures inconsistent with surrounding areas. We analyzed the recently proposed algorithms for inpainting-based 3D view synthesis and observed that these algorithms no longer produce stretching and black holes. However, the existing databases such as IETR, IRCCyN, and IVY have 3D-generated views with these artifacts. This observation suggests that the existing 3D view synthesis quality assessment algorithms can not judge the quality of most recent 3D synthesized views. With this view, through this abstract, we analyze the need for a new large-scale database and a new perceptual quality metric oriented for 3D views using a test dataset.
C3D and Localization Model for Locating and Recognizing the Actions from Untrimmed Videos (Student Abstract)
Authors: Singh Himanshu , Pallewad Tirupati , Subudhi Badri N , Jakhetiya Vinit
Published: July 2022

In this article, we proposed a technique for action localization and recognition from long untrimmed videos. It consists of C3D CNN model followed by the action mining using the localization model, where the KNN classifier is used. We segment the video into expressible sub-action known as action-bytes. The pseudo labels have been used to train the localization model, which makes the trimmed videos untrimmed for action-bytes. We present experimental results on the recent benchmark trimmed video dataset “Thumos14”.
Training Up to 50 Class ML Models on 3 $ IoT Hardware via Optimizing One-vs-One Algorithm (Student Abstract)
Authors: Sudharsan Bharath
Published: July 2022

Multi-class classifier training using traditional meta-algorithms such as the popular One-vs-One (OvO) method may not always work well under cost-sensitive setups. Also, during inference, OvO becomes computationally challenging for higher class counts K as O(K^2) is its time complexity. In this paper, we present Opt-OvO, an optimized (resource-friendly) version of the One-vs-One algorithm to enable high-performance multi-class ML classifier training and inference directly on microcontroller units (MCUs). Opt-OvO enables billions of tiny IoT devices to self learn/train (offline) after their deployment, using live data from a wide range of IoT use-cases. We demonstrate Opt-OvO by performing live ML model training on 4 popular MCU boards using datasets of varying class counts, sizes, and feature dimensions. The most exciting finding was, on the 3 $ ESP32 chip, Opt-OvO trained a multi-class ML classifier using a dataset of class count 50 and performed unit inference in super real-time of 6.2 ms.
Augmentation of Chinese Character Representations with Compositional Graph Learning (Student Abstract)
Authors: Wang Jason , Fu Kaiqun , Chen Zhiqian , Lu Chang-Tien
Published: July 2022

Chinese characters have semantic-rich compositional information in radical form. While almost all previous research has applied CNNs to extract this compositional information, our work utilizes deep graph learning on a compact, graph-based representation of Chinese characters. This allows us to exploit temporal information within the strict stroke order used in writing characters. Our results show that our stroke-based model has potential for helping large-scale language models on some Chinese natural language understanding tasks. In particular, we demonstrate that our graph model produces more interpretable embeddings shown through word subtraction analogies and character embedding visualizations.
A Hybrid Evolutionary Algorithm for the Diversified Top-k Weight Clique Search Problem (Student Abstract)
Authors: Wu Jun , Yin Minghao
Published: July 2022

The diversified top-k weight clique (DTKWC) search problem is an important generalization of the diversified top-k clique search problem, which extends the DTKC search problem by taking into account the weight of vertices. This problem involves finding at most k maximal weighted cliques that cover maximum weight of vertices with low overlapping in a given graph. In this study, a mixed integer linear program constraint formulation is proposed to model DTKWC search problem and an efficient hybrid evolutionary algorithm (HEA-D) based on some heuristic strategies is proposed to tackle it. Experiments on two sets of 110 graphs show that HEA-D outperforms the state-of-art methods.
Learning to Evolve on Dynamic Graphs (Student Abstract)
Authors: Xiang Xintao , Huang Tiancheng , Wang Donglin
Published: July 2022

Representation learning in dynamic graphs is a challenging problem because the topology of graph and node features vary at different time. This requires the model to be able to effectively capture both graph topology information and temporal information. Most existing works are built on recurrent neural networks (RNNs), which are used to exact temporal information of dynamic graphs, and thus they inherit the same drawbacks of RNNs. In this paper, we propose Learning to Evolve on Dynamic Graphs (LEDG) - a novel algorithm that jointly learns graph information and time information. Specifically, our approach utilizes gradient-based meta-learning to learn updating strategies that have better generalization ability than RNN on snapshots. It is model-agnostic and thus can train any message passing based graph neural network (GNN) on dynamic graphs. To enhance the representation power, we disentangle the embeddings into time embeddings and graph intrinsic embeddings. We conduct experiments on various datasets and down-stream tasks, and the experimental results validate the effectiveness of our method.
Multi-View Adjacency-Constrained Nearest Neighbor Clustering (Student Abstract)
Authors: Yang Jie , Lin Chin-Teng
Published: July 2022

Most existing multi-view clustering methods have problems with parameter selection and high computational complexity, and there have been very few works based on hierarchical clustering to learn the complementary information of multiple views. In this paper, we propose a Multi-view Adjacency-constrained Nearest Neighbor Clustering (MANNC) and its parameter-free version (MANNC-PF) to overcome these limitations. Experiments tested on eight real-world datasets validate the superiority of the proposed methods compared with the 13 current state-of-the-art methods.
Fine-Grained Urban Flow Inference via Normalizing Flow (Student Abstract)
Authors: Yu Haoyang , Xu Xovee , Zhong Ting , Zhou Fan
Published: July 2022

Fine-grained urban flow inference (FUFI) aims to infer the coarse-grained (CG) urban flow map to the corresponding fine-grained (FG) one, which plays an important role in efficient traffic monitoring and management in smart cities. In FUFI, the CG map can be obtained with only a small number of monitoring devices, greatly reducing the overhead of deploying devices and the costs of maintenance, labor, and electricity. Existing FUFI methods are mainly based on techniques from image super-resolution (SR) models, which cannot fully consider the influence of external factors and face the ill-posed problem in SR tasks. In this paper, we propose UFI-Flow, a novel approach for addressing the FUFI problem by learning the conditional distributions of CG and FG map pairs. Given the CG map and the latent variables, the corresponding FG map is inferred by invertible transformations. In addition, an augmented distribution fusion mechanism is further proposed to constrain the urban flow distribution within the influence of external factors. We provide a new large-scale real-world FUFI dataset and show that UFI-Flow significantly outperforms the strong baselines.
Enhance Weakly-Supervised Aspect Detection with External Knowledge (Student Abstract)
Authors: Zheng Zhuoming , Cai Yi , Li Liuwu
Published: July 2022

Aspect detection aims to identify aspects of reviews and is an essential up-stream task of opinion mining and so on. However, existing weakly-supervised methods suffer from lacking the ability of identifying implicit aspects with infrequent aspect terms and "Misc" aspects. To tackle these problems, we propose to enhance the representation of segment with external knowledge by a weakly-supervised method. Experiments demonstrate the effectiveness of our model and the improvement by incorporating external knowledge.
Towards Multimodal Vision-Language Models Generating Non-generic Text
Authors: Robbins Wes
Published: July 2022

While text generated by current vision-language models may be accurate and syntactically correct, it is often general. Recent work has used optical character recognition to supplement visual information with text extracted from an image. In many cases, using text in the image improves the specificity and usefulness of generated text. We contend that vision-language models can benefit from additional information extracted from an image. We modify previous multimodal frameworks to accept relevant information from a number of auxiliary classifiers. In particular, we focus on person names as an additional set of tokens and create a novel image-caption dataset to facilitate captioning with person names. The dataset, Politicians and Athletes in Captions (PAC), consists of captioned images of well-known people in context. By fine-tuning pretrained models with this dataset, we demonstrate a model that can naturally integrate facial recognition tokens into generated text by training on limited data.
Building Goal-Oriented Dialogue Systems with Situated Visual Context
Authors: Agarwal Sanchit , Jezabek Jan , Biswas Arijit , Barut Emre , Gao Bill , Chung Tagyoung
Published: July 2022

Goal-oriented dialogue agents can comfortably utilize the conversational context and understand its users' goals. However, in visually driven user experiences, these conversational agents are also required to make sense of the screen context in order to provide a proper interactive experience. In this paper, we propose a novel multimodal conversational framework where the dialogue agent's next action and their arguments are derived jointly conditioned both on the conversational and the visual context. We demonstrate the proposed approach via a prototypical furniture shopping experience for a multimodal virtual assistant.
UCSM-DNN: User and Card Style Modeling with Deep Neural Networks for Personalized Game AI
Authors: Choe Daegeun , Jo Youngbak , Kang Shindong , An Shounan , Oh Insoo
Published: July 2022

This paper tries to resolve long waiting time to find a matching person in player versus player mode of online sports games, such as baseball, soccer and basketball. In player versus player mode, game playing AI which is instead of player needs to be not just smart as human but also show variety to improve user experience against AI. Therefore a need to design game playing AI agents with diverse personalized styles rises. To this end, we propose a personalized game AI which encodes user style vectors and card style vectors with a general DNN, named UCSM-DNN. Extensive experiments show that UCSM-DNN shows improved performance in terms of personalized styles, which enrich user experiences. UCSM-DNN has already been integrated into popular mobile baseball game: MaguMagu 2021 as personalized game AI.
Cross-Modal Object Tracking: Modality-Aware Representations and a Unified Benchmark
Authors: Li Chenglong , Zhu Tianhao , Liu Lei , Si Xiaonan , Fan Zilin , Zhai Sulan
Published: July 2022

In many visual systems, visual tracking often bases on RGB image sequences, in which some targets are invalid in low-light conditions, and tracking performance is thus affected significantly. Introducing other modalities such as depth and infrared data is an effective way to handle imaging limitations of individual sources, but multi-modal imaging platforms usually require elaborate designs and cannot be applied in many real-world applications at present. Near-infrared (NIR) imaging becomes an essential part of many surveillance cameras, whose imaging is switchable between RGB and NIR based on the light intensity. These two modalities are heterogeneous with very different visual properties and thus bring big challenges for visual tracking. However, existing works have not studied this challenging problem. In this work, we address the cross-modal object tracking problem and contribute a new video dataset, including 654 cross-modal image sequences with over 481K frames in total, and the average video length is more than 735 frames. To promote the research and development of cross-modal object tracking, we propose a new algorithm, which learns the modality-aware target representation to mitigate the appearance gap between RGB and NIR modalities in the tracking process. It is plug-and-play and could thus be flexibly embedded into different tracking frameworks. Extensive experiments on the dataset are conducted, and we demonstrate the effectiveness of the proposed algorithm in two representative tracking frameworks against 19 state-of-the-art tracking methods. Dataset, code, model and results are available at https://github.com/mmic-lcl/source-code.
EditVAE: Unsupervised Parts-Aware Controllable 3D Point Cloud Shape Generation
Authors: Li Shidi , Liu Miaomiao , Walder Christian
Published: July 2022

This paper tackles the problem of parts-aware point cloud generation. Unlike existing works which require the point cloud to be segmented into parts a priori, our parts-aware editing and generation are performed in an unsupervised manner. We achieve this with a simple modification of the Variational Auto-Encoder which yields a joint model of the point cloud itself along with a schematic representation of it as a combination of shape primitives. In particular, we introduce a latent representation of the point cloud which can be decomposed into a disentangled representation for each part of the shape. These parts are in turn disentangled into both a shape primitive and a point cloud representation, along with a standardising transformation to a canonical coordinate system. The dependencies between our standardising transformations preserve the spatial dependencies between the parts in a manner that allows meaningful parts-aware point cloud generation and shape editing. In addition to the flexibility afforded by our disentangled representation, the inductive bias introduced by our joint modeling approach yields state-of-the-art experimental results on the ShapeNet dataset.
FedFR: Joint Optimization Federated Framework for Generic and Personalized Face Recognition
Authors: Liu Chih-Ting , Wang Chien-Yi , Chien Shao-Yi , Lai Shang-Hong
Published: July 2022

Current state-of-the-art deep learning based face recognition (FR) models require a large number of face identities for central training. However, due to the growing privacy awareness, it is prohibited to access the face images on user devices to continually improve face recognition models. Federated Learning (FL) is a technique to address the privacy issue, which can collaboratively optimize the model without sharing the data between clients. In this work, we propose a FL based framework called FedFR to improve the generic face representation in a privacy-aware manner. Besides, the framework jointly optimizes personalized models for the corresponding clients via the proposed Decoupled Feature Customization module. The client-specific personalized model can serve the need of optimized face recognition experience for registered identities at the local device. To the best of our knowledge, we are the first to explore the personalized face recognition in FL setup. The proposed framework is validated to be superior to previous approaches on several generic and personalized face recognition benchmarks with diverse FL scenarios. The source codes and our proposed personalized FR benchmark under FL setup are available at https://github.com/jackie840129/FedFR.
Information-Theoretic Bias Reduction via Causal View of Spurious Correlation
Authors: Seo Seonguk , Lee Joon-Young , Han Bohyung
Published: July 2022

We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information. Although several bias measurement methods have been proposed and widely investigated to achieve algorithmic fairness in various tasks such as face recognition, their accuracy- or logit-based metrics are susceptible to leading to trivial prediction score adjustment rather than fundamental bias reduction. Hence, we design a novel debiasing framework against the algorithmic bias, which incorporates a bias regularization loss derived by the proposed information-theoretic bias measurement approach. In addition, we present a simple yet effective unsupervised debiasing technique based on stochastic label noise, which does not require the explicit supervision of bias information. The proposed bias measurement and debiasing approaches are validated in diverse realistic scenarios through extensive experiments on multiple standard benchmarks.
P^3-Net: Part Mobility Parsing from Point Cloud Sequences via Learning Explicit Point Correspondence
Authors: Shi Yahao , Cao Xinyu , Lu Feixiang , Zhou Bin
Published: July 2022

Understanding an articulated 3D object with its movable parts is an essential skill for an intelligent agent. This paper presents a novel approach to parse 3D part mobility from point cloud sequences. The key innovation is learning explicit point correspondence from a raw unordered point cloud sequence. We propose a novel deep network called P^3-Net to parallelize trajectory feature extraction and point correspondence establishment, performing joint optimization between them. Specifically, we design a Match-LSTM module to reaggregate point features among different frames by a point correspondence matrix, a.k.a. the matching matrix. To obtain this matrix, an attention module is proposed to calculate the point correspondence. Moreover, we implement a Gumbel-Sinkhorn module to reduce the many-to-one relationship for better point correspondence. We conduct comprehensive evaluations on public benchmarks, including the motion dataset and the PartNet dataset. Results demonstrate that our approach outperforms SOTA methods on various 3D parsing tasks of part mobility, including motion flow prediction, motion part segmentation, and motion attribute (i.e. axis & range) estimation. Moreover, we integrate our approach into a robot perception module to validate its robustness.
JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection
Authors: Song Nan , Jiang Tianyuan , Yao Jian
Published: July 2022

Voxel and point representations are widely applied in recent 3D object detection tasks from LiDAR point clouds. Voxel representations contribute to efficiently and rapidly locating objects, whereas point representations are capable of describing intra-object spatial relationship for detection refinement. In this work, we aim to exploit the strengths of both two representations, and present a novel two-stage detector, named Joint Point-Voxel Network (JPV-Net). Specifically, our framework is equipped with a Dual Encoders-Fusion Decoder, which consists of the dual encoders to extract voxel features of sketchy 3D scenes and point features rich in geometric context, respectively, and the Feature Propagation Fusion (FP-Fusion) decoder to attentively fuse them from coarse to fine. By making use of the advantages of these features, the refinement network can effectively eliminate false detection and provide better accuracy. Besides, to further develop the perception characteristics of voxel CNN and point backbone, we design two novel intersection-over-union (IoU) estimation modules for proposal generation and refinement, both of which can alleviate the misalignment between the localization and the classification confidence. Extensive experiments on the KITTI dataset and ONCE dataset demonstrate that our proposed JPV-Net outperforms other state-of-the-art methods with remarkable margins.
Boost Supervised Pretraining for Visual Transfer Learning: Implications of Self-Supervised Contrastive Representation Learning
Authors: Sun Jinghan , Wei Dong , Ma Kai , Wang Liansheng , Zheng Yefeng
Published: July 2022

Unsupervised pretraining based on contrastive learning has made significant progress recently and showed comparable or even superior transfer learning performance to traditional supervised pretraining on various tasks. In this work, we first empirically investigate when and why unsupervised pretraining surpasses supervised counterparts for image classification tasks with a series of control experiments. Besides the commonly used accuracy, we further analyze the results qualitatively with the class activation maps and assess the learned representations quantitatively with the representation entropy and uniformity. Our core finding is that it is the amount of information effectively perceived by the learning model that is crucial to transfer learning, instead of absolute size of the dataset. Based on this finding, we propose Classification Activation Map guided contrastive (CAMtrast) learning which better utilizes the label supervsion to strengthen supervised pretraining, by making the networks perceive more information from the training images. CAMtrast is evaluated with three fundamental visual learning tasks: image recognition, object detection, and semantic segmentation, on various public datasets. Experimental results show that our CAMtrast effectively improves the performance of supervised pretraining, and that its performance is superior to both unsupervised counterparts and a recent related work which similarly attempted improving supervised pretraining.
Learning Temporally and Semantically Consistent Unpaired Video-to-Video Translation through Pseudo-Supervision from Synthetic Optical Flow
Authors: Wang Kaihong , Akash Kumar , Misu Teruhisa
Published: July 2022

Unpaired video-to-video translation aims to translate videos between a source and a target domain without the need of paired training data, making it more feasible for real applications. Unfortunately, the translated videos generally suffer from temporal and semantic inconsistency. To address this, many existing works adopt spatiotemporal consistency constraints incorporating temporal information based on motion estimation. However, the inaccuracies in the estimation of motion deteriorate the quality of the guidance towards spatiotemporal consistency, which leads to unstable translation. In this work, we propose a novel paradigm that regularizes the spatiotemporal consistency by synthesizing motions in input videos with the generated optical flow instead of estimating them. Therefore, the synthetic motion can be applied in the regularization paradigm to keep motions consistent across domains without the risk of errors in motion estimation. Thereafter, we utilize our unsupervised recycle and unsupervised spatial loss, guided by the pseudo-supervision provided by the synthetic optical flow, to accurately enforce spatiotemporal consistency in both domains. Experiments show that our method is versatile in various scenarios and achieves state-of-the-art performance in generating temporally and semantically consistent videos. Code is available at: https://github.com/wangkaihong/Unsup_Recycle_GAN/.