bluuebunny/crossref_metadata_2025 huggingface dataset:
Is Active Learning Always Beneficial? (Student Abstract)
Authors: Kravchenko Anna , Cusack Rhodri
Published: September 2022

This study highlights the limitations of automated curriculum learning, which may not be a viable strategy for tasks in which the benefits of the chosen curriculum are not apparent until much later. Using a simple convolutional network and a two-task training regime, we show that in some cases a network is not able to derive an optimal learning strategy using only the data available during a single training run.
Melodic Phrase Attention Network for Symbolic Data-based Music Genre Classification (Student Abstract)
Authors: Li Li , Zhang Rui , Wang Zhenyu
Published: September 2022

Compared with audio data-based music genre classification, researches on symbolic data-based music are scarce. Existing methods generally utilize manually extracted features, which is very time-consuming and laborious, and use traditional classifiers for label prediction without considering specific music features. To tackle this issue, we propose the Melodic Phrase Attention Network (MPAN) for symbolic data-based music genre classification. Our model is trained in three steps: First, we adopt representation learning, instead of the traditional musical feature extraction method, to obtain a vectorized representation of the music pieces. Second, the music pieces are divided into several melodic phrases through melody segmentation. Finally, the Melodic Phrase Attention Network is designed according to music characteristics, to identify the reflection of each melodic phrase on the music genre, thereby generating more accurate predictions. Experimental results show that our proposed method is superior to baseline symbolic data-based music genre classification approaches, and has achieved significant performance improvements on two large datasets.
Information Block Detection in Infographic Based on Spatial Proximity and Structural Similarity (Student Abstract)
Authors: Lin Jie , Wu Xin , Lu Jianwei , Cai Yi
Published: September 2022

The infographic is a type of visualization chart used to display information. Existing infographic understanding works utilize spatial proximity to group elements into information blocks. However, these works ignore structural features such as background color and boundary, which results in poor performance towards complex infographic. We propose Spatial and Structural Feature Extraction model to group elements based on spatial proximity and structural similarity. We introduce a new dataset towards information block detection. Experiments show that our model can effectively identify the information blocks in the infographic.
A Quantum-inspired Complex-valued Representation for Encoding Sentiment Information (Student Abstract)
Authors: Liu Guangcheng , Hou Yuexian , Song Shikai
Published: September 2022

Recently, a Quantum Probability Drive Network (QPDN) is proposed to model different levels of semantic units by extending word embedding to complex-valued representation (CR). The extended complex-valued embeddings are still insensitive to polarity causing that they generalize badly in sentiment analysis (SA). To solve it, we propose a method of encoding sentiment information into sentiment words for SA. Attention mechanism and an auxiliary task are introduced to help learn the CR of sentiment words with the help of the sentiment lexicon. We use the amplitude part to represent the distributional information and the phase part to represent the sentiment information of the language. Experiments on three popular SA datasets show that our method is effective.
Generating Adversarial yet Inconspicuous Patches with a Single Image (Student Abstract)
Authors: Luo Jinqi , Bai Tao , Zhao Jun
Published: September 2022

Deep neural networks have been shown vulnerable to adversarial patches, where exotic patterns can result in model’s wrong prediction. Nevertheless, existing approaches to adversarial patch generation hardly consider the contextual consistency between patches and the image background, causing such patches to be easily detected by human observation. Additionally, these methods require a large amount of data for training, which is computationally expensive. To overcome these challenges, we propose an approach to generate adversarial yet inconspicuous patches with one single image. In our approach, adversarial patches are produced in a coarse-to-fine way with multiple scales of generators and discriminators. The selection of patch location is based on the perceptual sensitivity of victim models. Contextual information is encoded during the Min-Max training to make patches consistent with surroundings.
A Context Aware Approach for Generating Natural Language Attacks
Authors: Maheshwary Rishabh , Maheshwary Saket , Pudi Vikram
Published: September 2022

We study an important task of attacking natural language processing models in a black box setting. We propose an attack strategy that crafts semantically similar adversarial examples on text classification and entailment tasks. Our proposed attack finds candidate words by considering the information of both the original word and its surrounding context. It jointly leverages masked language modelling and next sentence prediction for context understanding. In comparison to attacks proposed in prior literature, we are able to generate high quality adversarial examples that do significantly better both in terms of success rate and word perturbation percentage.
A Method for Taxonomy-Aware Embeddings Evaluation (Student Abstract)
Authors: Nobani Navid , Malandri Lorenzo , Mercorio Fabio , Mezzanzanica Mario
Published: September 2022

While word embeddings have been showing their effectiveness in capturing semantic and lexical similarities in a large number of domains, in case the corpus used to generate embeddings is associated with a taxonomy (i.e., classification tasks over standard de-jure taxonomies) the common intrinsic and extrinsic evaluation tasks cannot guarantee that the generated embeddings are consistent with the taxonomy. This, as a consequence sharply limits the use of distributional semantics in those domains. To address this issue, we design and implement MEET, which proposes a new measure -HSS- that allows evaluating embeddings from a text corpus preserving the semantic similarity relations of the taxonomy.
Solving JumpIN’ Using Zero-Dependency Reinforcement Learning (Student Abstract)
Authors: Ostic Rachel , Benning Oliver , Boily Patrick
Published: September 2022

Reinforcement learning seeks to teach agents to solve problems using numerical rewards as feedback. This makes it possible to incentivize actions that maximize returns despite having no initial strategy or knowledge of their environment. We implement a zero-external-dependency Q-learning algorithm using Python to optimally solve the single-player game JumpIn’ from SmartGames. We focus on interpretability of the model using Q-table parsing, and transferability to other games through a modular code structure. We observe rapid performance gains using our backtracking update algorithm.
Preventing Overfitting via Sample Reweighting for Recommender System Incremental Update (Student Abstract)
Authors: Peng Danni , Hu Xiaobo , Zeng Anxiang , Zhang Jie
Published: September 2022

Incremental update of recommender system models using only newly arrived data may easily cause the model to overfit to the current data. To address this issue without relying on historical data, we propose a sample reweighting method based on prediction performance of previous model on current data. The proposed method effectively alleviates the problem of overfitting and improves the performance of incremental update.
Context-Enhanced Entity and Relation Embedding for Knowledge Graph Completion (Student Abstract)
Authors: Qiao Ziyue , Ning Zhiyuan , Du Yi , Zhou Yuanchun
Published: September 2022

Most researches for knowledge graph completion learn representations of entities and relations to predict missing links in incomplete knowledge graphs. However, these methods fail to take full advantage of both the contextual information of entity and relation. Here, we extract contexts of entities and relations from the triplets which they compose. We propose a model named AggrE, which conducts efficient aggregations respectively on entity context and relation context in multi-hops, and learns context-enhanced entity and relation embeddings for knowledge graph completion. The experiment results show that AggrE is competitive to existing models.
Automatic Optimal Multi-Agent Path Finding Algorithm Selector (Student Abstract)
Authors: Ren Jingyao , Sathiyanarayanan Vikraman , Ewing Eric , Senbaslar Baskin , Ayanian Nora
Published: September 2022

Solving Multi-Agent Path Finding (MAPF) problems optimally is known to be NP-Hard for both make-span and total arrival time minimization. Many algorithms have been developed to solve MAPF problems optimally and they all have different strengths and weaknesses. There is no dominating MAPF algorithm that works well in all types of problems and no standard guidelines for when to use which algorithm. Therefore, there is a need for developing an automatic algorithm selector that suggests the best optimal algorithm to use given a MAPF problem instance. We propose a model based on convolutions and inception modules by treating the input MAPF instance as an image. We further show that techniques such as single-agent shortest path annotation and graph embedding are very effective for improving training quality. We evaluate our model and show that it outperforms all individual algorithms in its portfolio, as well as an existing state-of-the-art MAPF algorithm selector.
Generating Long Financial Report using Conditional Variational Autoencoders with Knowledge Distillation
Authors: Ren Yunpeng , Wang Ziao , Wang Yiyuan , Zhang Xiaofeng
Published: September 2022

Automatically generating financial report from a piece of news is quite a challenging task. Apparently, the difficulty of this task lies in the lack of sufficient background knowledge to effectively generate long financial report. To address this issue, this paper proposes the conditional variational autoencoders (CVAE) based approach which distills external knowledge from a corpus of news-report data. Experimental results demonstrate that the proposed approach could achieve the SOTA performance.
Are Chess Discussions Racist? An Adversarial Hate Speech Data Set (Student Abstract)
Authors: Sarkar Rupak , KhudaBukhsh Ashiqur R.
Published: September 2022

On June 28, 2020, while presenting a chess podcast on Grandmaster Hikaru Nakamura, Antonio Radic's YouTube handle got blocked because it contained ``harmful and dangerous'' content. YouTube did not give further specific reason, and the channel got reinstated within 24 hours. However, Radic speculated that given the current political situation, a referral to ``black against white'', albeit in the context of chess, earned him this temporary ban. In this paper, via a substantial corpus of 681,995 comments, on 8,818 YouTube videos hosted by five highly popular chess-focused YouTube channels, we ask the following research question: \emph{how robust are off-the-shelf hate-speech classifiers to out-of-domain adversarial examples?} We release a data set of 1,000 annotated comments where existing hate speech classifiers misclassified benign chess discussions as hate speech. We conclude with an intriguing analogy result on racial bias with our findings pointing out to the broader challenge of color polysemy.
Skills2Job: A Recommender System that Encodes Job Offer Embeddings on Graph Databases (Student Abstract)
Authors: Seveso Andrea , Giabelli Anna , Malandri Lorenzo , Mercorio Fabio , Mezzanzanica Mario
Published: September 2022

We propose a recommender system that, starting from a set of users skills, identifies the most suitable jobs as they emerge from a large text of Online Job Vacancies (OJVs). To this aim, we process 2.5M+ OJVs posted in three different countries (United Kingdom, France and Germany), generating several embeddings and performing an intrinsic evaluation of their quality. Besides, we compute a measure of skill importance for each occupation in each country, the Revealed Comparative Advantage (rca). The best vector models, together with the rca, are used to feed a graph database, which will serve as the keystone for the recommender system.
Finally, a user study of 10 validates the effectiveness of Skills2Job, both in terms of precision and nDGC.
Mental Actions and Explainability in Kripkean Semantics: What Else do I Know? (Student Abstract)
Authors: Singh Shikha , Khemani Deepak
Published: September 2022

The ability of an agent to distinguish the ramification effects of an action from its direct effects adds value to the explainability of its decisions. In this work, we propose to encode the ramification effects of ontic and epistemic actions as single-point update models in an epistemic planning domain modeled with Kripkean semantics of Knowledge and Belief. We call them “mental actions”. We discuss a preliminary approach to realize our idea, and we conclude by pointing out some optimizations as our ongoing pursuit.
Sampling Partial Acyclic Orientations in Chordal Graphs by the Lovasz Local Lemma (Student Abstract)
Authors: Sun Wenbo , Bezáková Ivona
Published: September 2022

Sampling of various types of acyclic orientations of chordal graphs plays a central role in several AI applications. In this work we investigate the use of the recently proposed general partial rejection sampling technique of Guo, Jerrum, and Liu, based on the Lovasz Local Lemma, for sampling partial acyclic orientations. For a given undirected graph, an acyclic orientation is an assignment of directions to all of its edges so that there is no directed cycle. In partial orientations some edges are allowed to be undirected. We show how the technique can be used to sample partial acyclic orientations of chordal graphs fast and with a clearly specified underlying distribution. This is in contrast to other samplers of various acyclic orientations with running times exponentially dependent on the maximum degree of the graph.
Semi-Supervised Learning via Triplet Network Based Active Learning (Student Abstract)
Authors: Sundriyal Divyanshu , Ghosh Soumyadeep , Vatsa Mayank , Singh Richa
Published: September 2022

In recent years, deep learning models have pushed state-of-the-art accuracies for several machine learning tasks. However, such models require a large amount of (supervised) data for training. While unlabelled data is available in abundance, manually labeling them is very costly. Active learning techniques helps in utilizing unlabelled data which may result in an improved classification model. In this research, we present an active learning algorithm which can help in increasing performance of deep learning models by using large amount of unlabelled data. A novel active learning algorithm, Triplet AL is proposed which uses a triplet network to select samples from an unlabelled data set. Previous active learning methods rely on classification model's final prediction scores as a measure of confidence for an unlabelled sample. We propose a more reliable confidence measure, termed as Top-Two-Margin which is given by the Triplet Network. The proposed algorithm shows improved performance compared to other active learning approaches.
FACS: Fast Code-based Algorithm for Coalition Structure Generation (Student Abstract)
Authors: Taguelmimt Redha , Aknine Samir , Boukredera Djamila , Changder Narayan
Published: September 2022

In this paper, we propose a new algorithm for the Coalition Structure Generation (CSG) problem that can be run with more than 28 agents while using a complete set of coalitions as input. The current state-of-the-art limit for exact algorithms to solve the CSG problem within a reasonable time is 27 agents. Our algorithm uses a novel representation of the search space and a new code-based search technique. We propose an effective heuristic search method to efficiently explore the space of coalition structures using our code based technique and show that our method outperforms existing state-of-the-art algorithms by multiple orders of magnitude.
Multi-label Few-shot Learning with Semantic Inference (Student Abstract)
Authors: Wang Zhen , Duan Yiqun , Liu Liu , Tao Dacheng
Published: September 2022

Few-shot learning can adapt the classification model to new labels with only a few labeled examples. Previous studies mainly focus on the scenario of a single category label per example but have not solved the more challenging multi-label scenario with exponential-sized output space and low-data effectively. In this paper, we propose a semantic-aware meta-learning model for multi-label few-shot learning. Our approach can learn and infer the semantic correlation between unseen labels and historical labels to quickly adapt multi-label tasks from only a few examples. Specifically, features can be mapped into the semantic embedding space via label word vectors to explore and exploit the label correlation, and thus cope with the challenge on the overwhelming size of the output space. Then a novel semantic inference mechanism is designed for leveraging prior knowledge learned from historical labels, which will produce good generalization performance on new labels to alleviate the low-data problem. Finally, extensive empirical results show that the proposed method significantly outperforms the existing state-of-the-art methods on the multi-label few-shot learning tasks.
Extending Policy Shaping to Continuous State Spaces (Student Abstract)
Authors: Wei Thomas , Faulkner Taylor A. Kessler , Thomaz Andrea L.
Published: September 2022

Policy Shaping is a Human-in-the-loop Reinforcement Learning (HRL) algorithm. We extend this work to continuous states with our algorithm, Deep Policy Shaping (DPS). DPS uses a feedback neural network that learns the optimality of actions from noisy feedback combined with an RL algorithm. In simulation, we find that DPS outperforms or matches baselines averaged over multiple hyperparameter settings and varying feedback correctness.
Measuring Vegetation Density in Marsh Grass Photographs Using Deep Neural Networks (Student Abstract)
Authors: Welch Lucas , Liu Xudong
Published: September 2022

To protect the world’s marshlands, it is of utmost importance to be able to monitor their vegetation composition and coverage. This currently is accomplished by large teams of researchers and volunteers manually looking at the marsh images and labeling randomly selected pixels by what species (or lack thereof) is present at the pixel. This task, however, is extremely labor intensive, limiting the amount of quality environmental monitoring that can be done in the field. If the task was automated, teams would be able to monitor larger swaths of land. In this paper, we propose a novel framework for such automation using deep neural networks. Then, we focus on the key component of this framework: a binary classifier to decide whether a pixel is vegetated or not. To this end, we create a dataset of labeled snippet images out of publicly available photoquadrats of the marshlands in Florida. Finally, we construct LeNet-5 and AlexNet, adjusted to our input snippets, faster training time, networks and experiment to learn them on our dataset for the binary classification task. Our results show that the AlexNet model achieves higher accuracy on the test set than the LeNet-5 model, with 92.41% for AlexNet and 91.34% for LeNet-5.
Towards Extracting Graph Neural Network Models via Prediction Queries (Student Abstract)
Authors: Wu Bang , Pan Shirui , Yuan Xingliang
Published: September 2022

Graph data has been widely used to represent data from various domain, e.g., social networks, recommendation system. With great power, the GNN models, usually as valuable properties of their owners, also become attractive targets of the adversary who covets to steal them. While existing works show that simple deep neural networks can be reproduced by so-called Model Extraction Attacks, how to extract a GNN model has not been explored. In this paper, we exploit the threat of model extraction attacks against GNN models. Unlike ordinary attacks which obtain model information via only the input-output query pairs, we utilize both the node queries and the graph structure to extract the GNNs. Furthermore, we consider the stealthiness of the attack and propose to generate legitimate queries so the extraction can be applied discreetly. We implement our attack by leveraging the responses of these queries, as well as other accessible knowledge, e.g., neighbor connectives of the queried nodes. By evaluating over three real-world datasets, our attack is shown to effectively produce a surrogate model with more than 80% equivalent predictions as the target model.
MMIM: An Interpretable Regularization Method for Neural Networks (Student Abstract)
Authors: Xie Nan , Hou Yuexian
Published: September 2022

In deep learning models, most of network architectures are designed artificially and empirically. Although adding new structures such as convolution kernels in CNN is widely used, there are few methods to design new structures and mathematical tools to evaluate feature representation capabilities of new structures. Inspired by ensemble learning, we propose an interpretable regularization method named Minimize Mutual Information Method(MMIM), which minimize the generalization error by minimizing the mutual information of hidden neurons. The experimental results also verify the effectiveness of our proposed MMIM.
Enhancing Context-Based Meta-Reinforcement Learning Algorithms via An Efficient Task Encoder (Student Abstract)
Authors: Xu Feng , Jiang Shengyi , Yin Hao , Zhang Zongzhang , Yu Yang , Li Ming , Li Dong , Liu Wulong
Published: September 2022

Meta-Reinforcement Learning (meta-RL) algorithms enable agents to adapt to new tasks from small amounts of exploration, based on the experience of similar tasks. Recent studies have pointed out that a good representation of a task is key to the success of off-policy context-based meta-RL. Inspired by contrastive methods in unsupervised representation learning, we propose a new method to learn the task representation based on the mutual information between transition tuples in a trajectory and the task embedding. We also propose a new estimation for task similarity based on Q-function, which can be used to form a constraint on the distribution of the encoded task variables, making the task encoder encode the task variables more effective on new tasks. Experiments on meta-RL tasks show that the newly proposed method outperforms existing meta-RL algorithms.
Improving the Morphology and Control Policy of Self-reconfiguring Modular Robots in Dynamic Environment (Student Abstract)
Authors: Xue Dong , Lu Qiang , Li Jian
Published: September 2022

Ideally, self-reconfiguration modular robots (SMR) can change their morphology and perform actions related to a specific task in any scene. However, most SMRs only adapt to several specific scenes because their morphology and control policies are designed or trained based on these scenes. Once SMRs meet an unknown scene, especially multiply unknown scenes (called dynamic environment), these policies will be useless.
Although some of these policies import evolutionary algorithms to enhance the ability of SMR to explore unknown scenes, they are very time-consuming. The reason is that individual fitness depends on the interaction between SMR and these scenes. We propose a two-stage reconfiguration algorithm (TSRA) without any prior knowledge to address the time-consuming problem. In the two stages, the reconfiguration methods use the evolutionary algorithm (GA) to simultaneously generate scene-fitted morphology and actions. The first stage method uses the estimation neural network to evaluate the individual fitness to run faster and can recommend better policies to the second stage. The second stage method obtains this fitness from scenes and updates the neural network to approximate these scenes. Through experiments, TSRA can find better morphology and control policies than the other two canonical algorithms --- GA and GEM-RL.
Text Analysis for Understanding Symptoms of Social Anxiety in Student Veterans
Authors: Byers Morgan , Metsis Vangelis
Published: September 2022

A significant portion of the veteran population suffers from PTSD, a mental illness that is often accompanied by social anxiety disorder. Student veterans are especially vulnerable as they struggle to adapt to a new, less structured college lifestyle. In order to assist psychologists and social workers in the treatment of social anxiety disorder we use machine learning to analyze transcribed interview text and apply topic modelling to highlight common stress factors for student veterans. The results detailed in this paper also have broader impacts in fields such as pedagogy and public health.
Probabilistic Robustness Quantification of Neural Networks
Authors: Kishan Gopi
Published: September 2022

Safety properties of neural networks are critical to their application in safety-critical domains. Quantification of their robustness against uncertainties is an upcoming area of research. In this work, we propose an approach for providing probabilistic guarantees on the performance of a trained neural network.

We present two novel metrics for probabilistic verification on training data distribution and test dataset. Given a trained neural network, we quantify the probability of the model to make errors on a random sample drawn from the training data distribution. Second, from the output logits of a sample test point, we measure its p-value on the learned logit distribution to quantify the confidence of the model at this test point. We compare our results with softmax based metric using the black-box adversarial attacks on a simple CNN architecture trained for MNIST digit classification.
Investigating Methods of Balancing Inequality and Efficiency in Ride Pooling
Authors: Raman Naveen
Published: September 2022

Our research focuses on developing matching policies that match drivers and riders for ride-pooling services. We aim to develop policies that balance efficiency and various forms of fairness. We did this through two methods: new matching algorithms that include a fairness term in the objective function, and income redistribution methods based on the Shapley value of a driver. I tested these methods on New York City Taxicab data to evaluate their performance and found that they succeed in reducing certain forms of fairness.
A Health-friendly Speaker Verification System Supporting Mask Wearing
Authors: Chen Chaotao , Jiang Di , Peng Jinhua , Lian Rongzhong , Zhang Chen Jason , Xu Qian , Fan Lixin , Yang Qiang
Published: September 2022

We demonstrate a health-friendly speaker verification system for voice-based identity verification on mobile devices. The system is built upon a speech processing module, a ResNet-based local acoustic feature extractor and a multi-head attention-based embedding layer, and is optimized under an additive margin softmax loss for discriminative speaker verification. It is shown that the system achieves superior performance no matter whether there is mask wearing or not. This characteristic is important for speaker verification services operating in regions affected by the raging coronavirus pneumonia. With this demonstration, the audience will have an in-depth experience of how the accuracy of bio-metric verification and the personal health are simultaneously ensured. We wish that this demonstration would boost the development of next-generation bio-metric verification technologies.
Bootstrapping Dialog Models from Human to Human Conversation Logs
Authors: Dhoolia Pankaj , Kumar Vineet , Contractor Danish , Joshi Sachindra
Published: September 2022

State-of-the-art commercial dialog platforms provide powerful tools to build a conversational agent. These platforms provide complete control to the dialog designer to model user-agent interactions. However, a dialog designer needs to rely on domain experts to manually build the dialog model -- by creating dialog flow nodes and modeling user intents. This process is laborious, time consuming and expensive and does not allow the designer to exploit human to human conversation logs effectively. In this work, we present a research prototype that can ingest human-to-human conversation logs between an end-user and an agent, and suggest user-intents and agent-responses, given a conversation context. We utilize human to human conversation logs to build two emulators: user and agent. An agent emulator models an agent response given the conversation context so far, and a user emulator outputs possible user responses. Our system is able to recommend conversational intents as well as conversation flow using emulators based on real-world data, thus making the process of designing a bot more efficient. To the best our knowledge this is the first system that enables data-driven dialog model creation by emulating users and agents.
KAAPA: Knowledge Aware Answers from PDF Analysis
Authors: Fauceglia Nicolas , Canim Mustafa , Gliozzo Alfio , Liang Jennifer J , Wang Nancy Xin Ru , Burdick Douglas , Mihindukulasooriya Nandana , Castelli Vittorio , Feigenblat Guy , Konopnicki David , Katsis Yannis , Florian Radu , Li Yunyao , Roukos Salim , Sil Avirup
Published: September 2022

We present KaaPa (Knowledge Aware Answers from Pdf Analysis), an integrated solution for machine reading comprehension over both text and tables extracted from PDFs. KaaPa enables interactive question refinement using facets generated from an automatically induced Knowledge Graph. In addition it provides a concise summary of the supporting evidence for the provided answers by aggregating information across multiple sources. KaaPa can be applied consistently to any collection of documents in English with zero domain adaptation effort. We showcase the use of KaaPa for QA on scientific literature using the COVID-19 Open Research Dataset.
Dialog Router: Automated Dialog Transition via Multi-Task Learning
Authors: Huang Ziming , Jiang Zhuoxuan , Chen Hao , Han Xue , Dang Yabin
Published: September 2022

Dialog Router is a general paradigm for human-bot symbiosis
dialog systems to provide friendly customer care service. It is
equipped with a multi-task learning model to automatically
capture the underlying correlation between multiple related
tasks, i.e. dialog classification and regression, and greatly reduce
human labor work for system customization, which improves
the accuracy of dialog transition. In addition, for learning
the multi-task model, the training data and labels are easy
to collect from human-to-human historical dialog logs, and
the Dialog Router can be easily integrated into the majority of
existing dialog systems by calling general APIs. We conduct
experiments on real-world datasets for dialog classification
and regression. The results show that our model achieves improvements
on both tasks, which benefits the dialog transition
application. The demo illustrates our method’s effectiveness
in a real customer care service.
EasyRL: A Simple and Extensible Reinforcement Learning Framework
Authors: Hulbert Neil , Spillers Sam , Francis Brandon , Haines-Temons James , Gil Romero Ken , De Jager Benjamin , Wong Sam , Flora Kevin , Huang Bowei , A. Irissappane Athirai
Published: September 2022

In recent years, Reinforcement Learning (RL), has become a popular field of study as well as a tool for enterprises working on cutting-edge artificial intelligence research. To this end, many researchers have built RL frameworks such as openAI Gym, and KerasRL for ease of use. While these works have made great strides towards bringing down the barrier of entry for those new
to RL, we propose a much simpler framework called EasyRL,
by providing an interactive graphical user interface for users to train and evaluate RL agents. As it is entirely graphical, EasyRL does not require programming knowledge for training and testing simple built-in RL agents. EasyRL also supports custom RL agents and environments, which can be highly beneficial for RL researchers in evaluating and comparing their RL models.
RADAR-X: An Interactive Interface Pairing Contrastive Explanations with Revised Plan Suggestions
Authors: Karthik Valmeekam , Sreedharan Sarath , Sengupta Sailik , Kambhampati Subbarao
Published: September 2022

Automated Planning techniques can be leveraged to build effective decision support systems that assist the human-in-the-loop. Such systems must provide intuitive explanations when the suggestions made by these systems seem inexplicable to the human. In this regard, we consider scenarios where the user questions the system's suggestion by providing alternatives (referred to as foils). In response, we empower existing decision support technologies to engage in an interactive explanatory dialogue with the user and provide contrastive explanations based on user-specified foils to reach a consensus on proposed decisions. To provide contrastive explanations, we adapt existing techniques in Explainable AI Planning (XAIP). Furthermore, we use this dialog to elicit the user's latent preferences and propose three modes of interaction that use these preferences to provide revised plan suggestions. Finally, we showcase a decision support system that provides all these capabilities.
i-Parser: Interactive Parser Development Kit for Natural Language Processing
Authors: Morio Gaku , Ozaki Hiroaki , Koreeda Yuta , Morishita Terufumi , Miyoshi Toshinori
Published: September 2022

This demonstration paper presents i-Parser, a novel development kit that produces high-performance semantic parsers.
i-Parser converts training graphs into sequences written in a context-free language, then our proposed model learns to generate the sequences.
With interactive configuration and visualization, users can easily build their own parsers.
Benchmark results of i-Parser showed high performances of various parsing tasks in natural language processing.
ITRIX - an AI Enabled Solution for Orchestration of Recovery Instructions
Authors: Sherchan Wanita , Vaughn Garfield , Pervin Shaila , Barone Bryan
Published: September 2022

This paper presents ITRIX (IT Recovery Instructions
eXtraction)- a solution that automates the complex manual
process of creating executable recovery plans from textual
documents describing IT environment disaster recovery procedure.
The solution applies natural language processing and
deep learning to identify textual content that contain recovery
instructions and information relevant to disaster recovery.
This enables rapid creation and review of recovery plans resulting
in 80% reduction in time taken to generate them.
The Active Sensing Testbed
Authors: Stearns Lee , Fendley Neil , Llorens Ashley J.
Published: September 2022

The Active Sensing Testbed (AST) is a novel framework for research in machine perception and world-view rea-soning. The AST supports exploratory development of perception systems that can build internal models of the world by combining multi-view and multi-modal analyt-ics, utilize these models to form hypotheses about a sce-ne, and potentially take action to fill in gaps in knowledge or make predictions about future world states. As a modular software framework, the AST is in-tended to lower the barrier to entry for researchers and developers in applying state-of-the-art computer vision techniques to real-world problems.
MMKE: A Multi-Model Knowledge Extraction System from Unstructured Texts
Authors: Zhang Qian-Wen , Yan Zhao , Zhao Tianyang , Zhang Shi-Wei , Yao Meng , Rao Meng-Liang , Cao Yunbo
Published: September 2022

In this work, we present a Multi-Model Knowledge Extraction (MMKE) System which consists of two unstructured text extraction models (RelationSO model and SubjectRO model) based on a multi-task learning framework. Instead of recognizing entity first and then predicting relationships between entity pairs in previous works, MMKE detects subject and corresponding relationships before extracting objects to cope with the diverse object-type problem, overlapping problem and non-predefined relation problem. Our system accepts unstructured text as input, from which it automatically extracts triplets knowledge (subject, relation, object). More importantly, we incorporate a number of user-friendly extraction functionalities, such as multi-format uploading, one-click extractions, knowledge editing and graphical displays. The demonstration video is available at this link: https://youtu.be/HtOPJrGhSxk.
Context-aware Attentional Pooling (CAP) for Fine-grained Visual Classification
Authors: Behera Ardhendu , Wharton Zachary , Hewage Pradeep R P G , Bera Asish
Published: September 2022

Deep convolutional neural networks (CNNs) have shown a strong ability in mining discriminative object pose and parts information for image recognition. For fine-grained recognition, context-aware rich feature representation of object/scene plays a key role since it exhibits a significant variance in the same subcategory and subtle variance among different subcategories. Finding the subtle variance that fully characterizes the object/scene is not straightforward. To address this, we propose a novel context-aware attentional pooling (CAP) that effectively captures subtle changes via sub-pixel gradients, and learns to attend informative integral regions and their importance in discriminating different subcategories without requiring the bounding-box and/or distinguishable part annotations. We also introduce a novel feature encoding by considering the intrinsic consistency between the informativeness of the integral regions and their spatial structures to capture the semantic correlation among them. Our approach is simple yet extremely effective and can be easily applied on top of a standard classification backbone network. We evaluate our approach using six state-of-the-art (SotA) backbone networks and eight benchmark datasets. Our method significantly outperforms the SotA approaches on six datasets and is very competitive with the remaining two.
Commonsense Knowledge Aware Concept Selection For Diverse and Informative Visual Storytelling
Authors: Chen Hong , Huang Yifei , Takamura Hiroya , Nakayama Hideki
Published: September 2022

Visual storytelling is a task of generating relevant and interesting stories for given image sequences. In this work we aim at increasing the diversity of the generated stories while preserving the informative content from the images. We propose to foster the diversity and informativeness of a generated story by using a concept selection module that suggests a set of concept candidates. Then, we utilize a large scale pre-trained model to convert concepts and images into full stories. To enrich the candidate concepts, a commonsense knowledge graph is created for each image sequence from which the concept candidates are proposed. To obtain appropriate concepts from the graph, we propose two novel modules that consider the correlation among candidate concepts and the image-concept correlation. Extensive automatic and human evaluation results demonstrate that our model can produce reasonable concepts. This enables our model to outperform the previous models by a large margin on the diversity and informativeness of the story, while retaining the relevance of the story to the image sequence.
Arbitrary Video Style Transfer via Multi-Channel Correlation
Authors: Deng Yingying , Tang Fan , Dong Weiming , Huang Haibin , Ma Chongyang , Xu Changsheng
Published: September 2022

Video style transfer is attracting increasing attention from the artificial intelligence community because of its numerous applications, such as augmented reality and animation production. Relative to traditional image style transfer, video style transfer presents new challenges, including how to effectively generate satisfactory stylized results for any specified style while maintaining temporal coherence across frames. Towards this end, we propose a Multi-Channel Correlation network (MCCNet), which can be trained to fuse exemplar style features and input content features for efficient style transfer while naturally maintaining the coherence of input videos to output videos. Specifically, MCCNet works directly on the feature space of style and content domain where it learns to rearrange and fuse style features on the basis of their similarity to content features. The outputs generated by MCC are features containing the desired style patterns that can further be decoded into images with vivid style textures. Moreover, MCCNet is also designed to explicitly align the features to input and thereby ensure that the outputs maintain the content structures and the temporal continuity. To further improve the performance of MCCNet under complex light conditions, we also introduce illumination loss during training. Qualitative and quantitative evaluations demonstrate that MCCNet performs well in arbitrary video and image style transfer tasks. Code is available at https://github.com/diyiiyiii/MCCNet.
Semantic-guided Reinforced Region Embedding for Generalized Zero-Shot Learning
Authors: Ge Jiannan , Xie Hongtao , Min Shaobo , Zhang Yongdong
Published: September 2022

Generalized zero-shot Learning (GZSL) aims to recognize images from either seen or unseen domain, mainly by learning a joint embedding space to associate image features with the corresponding category descriptions. Recent methods have proved that localizing important object regions can effectively bridge the semantic-visual gap. However, these are all based on one-off visual localizers, lacking of interpretability and flexibility. In this paper, we propose a novel Semantic-guided Reinforced Region Embedding (SR2E) network that can localize important objects in the long-term interests to construct semantic-visual embedding space. SR2E consists of Reinforced Region Module (R2M) and Semantic Alignment Module (SAM). First, without the annotated bounding box as supervision, R2M encodes the semantic category guidance into the reward and punishment criteria to teach the localizer serialized region searching. Besides, R2M explores different action spaces during the serialized searching path to avoid local optimal localization, which thereby generates discriminative visual features with less redundancy. Second, SAM preserves the semantic relationship into visual features via semantic-visual alignment and designs a domain detector to alleviate the domain confusion. Experiments on four public benchmarks demonstrate that the proposed SR2E is an effective GZSL method with reinforced embedding space, which obtains averaged 6.1% improvements.
Analogical Image Translation for Fog Generation
Authors: Gong Rui , Dai Dengxin , Chen Yuhua , Li Wen , Paudel Danda Pani , Van Gool Luc
Published: September 2022

Image-to-image translation is to map images from a given style to another given style. While exceptionally successful, current methods assume the availability of training images in both source and target domains, which does not always hold in practice. Inspired by humans' reasoning capability of analogy, we propose analogical image translation (AIT) that exploit the concept of gist, for the first time. Given images of two styles in the source domain: A and A', along with images B of the first style in the target domain, learn a model to translate B to B' in the target domain, such that A:A' :: B:B'. AIT is especially useful for translation scenarios in which training data of one style is hard to obtain but training data of the same two styles in another domain is available. For instance, in the case from normal conditions to extreme, rare conditions, obtaining real training images for the latter case is challenging. However, obtaining synthetic data for both cases is relatively easy. In this work, we aim at adding adverse weather effects, more specifically fog, to images taken in clear weather. To circumvent the challenge of collecting real foggy images, AIT learns the gist of translating synthetic clear-weather to foggy images, followed by adding fog effects onto real clear-weather images, without ever seeing any real foggy image. AIT achieves zero-shot image translation capability, whose effectiveness and benefit are demonstrated by the downstream task of semantic foggy scene understanding.
PTN: A Poisson Transfer Network for Semi-supervised Few-shot Learning
Authors: Huang Huaxi , Zhang Junjie , Zhang Jian , Wu Qiang , Xu Chang
Published: September 2022

The predicament in semi-supervised few-shot learning (SSFSL) is to maximize the value of the extra unlabeled data to boost the few-shot learner. In this paper, we propose a Poisson Transfer Network (PTN) to mine the unlabeled information for SSFSL from two aspects. First, the Poisson Merriman–Bence–Osher (MBO) model builds a bridge for the communications between labeled and unlabeled examples. This model serves as a more stable and informative classifier than traditional graph-based SSFSL methods in the message-passing process of the labels. Second, the extra unlabeled samples are employed to transfer the knowledge from base classes to novel classes through contrastive learning. Specifically, we force the augmented positive pairs close while push the negative ones distant. Our contrastive transfer scheme implicitly learns the novel-class embeddings to alleviate the over-fitting problem on the few labeled data. Thus, we can mitigate the degeneration of embedding generality in novel classes. Extensive experiments indicate that PTN outperforms the state-of-the-art few-shot and SSFSL models on miniImageNet and tieredImageNet benchmark datasets.
Augmented Partial Mutual Learning with Frame Masking for Video Captioning
Authors: Lin Ke , Gan Zhuoxin , Wang Liwei
Published: September 2022

Recent video captioning work improves greatly due to the invention of various elaborate model architectures. If multiple captioning models are combined into a unified framework not only by simple more ensemble, and each model can benefit from each other, the final captioning might be boosted further. Jointly training of multiple model have not been explored in previous works. In this paper, we propose a novel Augmented Partial Mutual Learning (APML) training method where multiple decoders are trained jointly with mimicry losses between different decoders and different input variations. Another problem of training captioning model is the "one-to-many" mapping problem which means that one identical video input is mapped to multiple caption annotations. To address this problem, we propose an annotation-wise frame masking approach to convert the "one-to-many" mapping to "one-to-one" mapping. The experiments performed on MSR-VTT and MSVD datasets demonstrate our proposed algorithm achieves the state-of-the-art performance.