pwc-archive/papers-with-abstracts huggingface dataset:
The Spike Gating Flow: A Hierarchical Structure Based Spiking Neural Network for Online Gesture Recognition
Authors: Zihao Zhao, Yanhong Wang, Qiaosha Zou, Tie XU, Fangbo Tao, Jiansong Zhang, Xiaoan Wang, C. -J. Richard Shi, Junwen Luo, Yuan Xie
Date: 2022-06-04 00:00:00
Tasks: Action Recognition, Few-Shot Learning, Gesture Recognition

Action recognition is an exciting research avenue for artificial intelligence since it may be a game changer in the emerging industrial fields such as robotic visions and automobiles. However, current deep learning faces major challenges for such applications because of the huge computational cost and the inefficient learning. Hence, we develop a novel brain-inspired Spiking Neural Network (SNN) based system titled Spiking Gating Flow (SGF) for online action learning. The developed system consists of multiple SGF units which assembled in a hierarchical manner. A single SGF unit involves three layers: a feature extraction layer, an event-driven layer and a histogram-based training layer. To demonstrate the developed system capabilities, we employ a standard Dynamic Vision Sensor (DVS) gesture classification as a benchmark. The results indicate that we can achieve 87.5% accuracy which is comparable with Deep Learning (DL), but at smaller training/inference data number ratio 1.5:1. And only a single training epoch is required during the learning process. Meanwhile, to the best of our knowledge, this is the highest accuracy among the non-backpropagation algorithm based SNNs. At last, we conclude the few-shot learning paradigm of the developed network: 1) a hierarchical structure-based network design involves human prior knowledge; 2) SNNs for content based global dynamic feature detection.
Soft Adversarial Training Can Retain Natural Accuracy
Authors: Abhijith Sharma, Apurva Narayan
Date: 2022-06-04 00:00:00

Adversarial training for neural networks has been in the limelight in recent years. The advancement in neural network architectures over the last decade has led to significant improvement in their performance. It sparked an interest in their deployment for real-time applications. This process initiated the need to understand the vulnerability of these models to adversarial attacks. It is instrumental in designing models that are robust against adversaries. Recent works have proposed novel techniques to counter the adversaries, most often sacrificing natural accuracy. Most suggest training with an adversarial version of the inputs, constantly moving away from the original distribution. The focus of our work is to use abstract certification to extract a subset of inputs for (hence we call it 'soft') adversarial training. We propose a training framework that can retain natural accuracy without sacrificing robustness in a constrained setting. Our framework specifically targets moderately critical applications which require a reasonable balance between robustness and accuracy. The results testify to the idea of soft adversarial training for the defense against adversarial attacks. At last, we propose the scope of future work for further improvement of this framework.
Estimating counterfactual treatment outcomes over time in complex multiagent scenarios
Authors: Keisuke Fujii, Koh Takeuchi, Atsushi Kuribayashi, Naoya Takeishi, Yoshinobu Kawahara, Kazuya Takeda
Date: 2022-06-04 00:00:00
Tasks: Autonomous Driving, counterfactual, Prediction

Evaluation of intervention in a multiagent system, e.g., when humans should intervene in autonomous driving systems and when a player should pass to teammates for a good shot, is challenging in various engineering and scientific fields. Estimating the individual treatment effect (ITE) using counterfactual long-term prediction is practical to evaluate such interventions. However, most of the conventional frameworks did not consider the time-varying complex structure of multiagent relationships and covariate counterfactual prediction. This may lead to erroneous assessments of ITE and difficulty in interpretation. Here we propose an interpretable, counterfactual recurrent network in multiagent systems to estimate the effect of the intervention. Our model leverages graph variational recurrent neural networks and theory-based computation with domain knowledge for the ITE estimation framework based on long-term prediction of multiagent covariates and outcomes, which can confirm the circumstances under which the intervention is effective. On simulated models of an automated vehicle and biological agents with time-varying confounders, we show that our methods achieved lower estimation errors in counterfactual covariates and the most effective treatment timing than the baselines. Furthermore, using real basketball data, our methods performed realistic counterfactual predictions and evaluated the counterfactual passes in shot scenarios.
Evaluation of creating scoring opportunities for teammates in soccer via trajectory prediction
Authors: Masakiyo Teranishi, Kazushi Tsutsui, Kazuya Takeda, Keisuke Fujii
Date: 2022-06-04 00:00:00
Tasks: Trajectory Prediction

Evaluating the individual movements for teammates in soccer players is crucial for assessing teamwork, scouting, and fan engagement. It has been said that players in a 90-min game do not have the ball for about 87 minutes on average. However, it has remained difficult to evaluate an attacking player without receiving the ball, and to reveal how movement contributes to the creation of scoring opportunities for teammates. In this paper, we evaluate players who create off-ball scoring opportunities by comparing actual movements with the reference movements generated via trajectory prediction. First, we predict the trajectories of players using a graph variational recurrent neural network that can accurately model the relationship between players and predict the long-term trajectory. Next, based on the difference in the modified off-ball evaluation index between the actual and the predicted trajectory as a reference, we evaluate how the actual movement contributes to scoring opportunity compared to the predicted movement. For verification, we examined the relationship with the annual salary, the goals, and the rating in the game by experts for all games of a team in a professional soccer league in a year. The results show that the annual salary and the proposed indicator correlated significantly, which could not be explained by the existing indicators and goals. Our results suggest the effectiveness of the proposed method as an indicator for a player without the ball to create a scoring chance for teammates.
Modeling of Textures to Predict Immune Cell Status and Survival of Brain Tumour Patients
Authors: Ahmad Chaddad, Mingli Zhang, Lama Hassan, Tamim Niazi
Date: 2022-06-04 00:00:00

Radiomics has shown a capability for different types of cancers such as glioma to predict the clinical outcome. It can have a non-invasive means of evaluating the immunotherapy response prior to treatment. However, the use of deep convolutional neural networks (CNNs)-based radiomics requires large training image sets. To avoid this problem, we investigate a new imaging features that model distribution with a Gaussian mixture model (GMM) of learned 3D CNN features. Using these deep radiomic features (DRFs), we aim to predict the immune marker status (low versus high) and overall survival for glioma patients. We extract the DRFs by aggregating the activation maps of a pre-trained 3D-CNN within labeled tumor regions of MRI scans that corresponded immune markers of 151 patients. Our experiments are performed to assess the relationship between the proposed DRFs, three immune cell markers (Macrophage M1, Neutrophils and T Cells Follicular Helper), and measure their association with overall survival. Using the random forest (RF) model, DRFs was able to predict the immune marker status with area under the ROC curve (AUC) of 78.67, 83.93 and 75.67\% for Macrophage M1, Neutrophils and T Cells Follicular Helper, respectively. Combined the immune markers with DRFs and clinical variables, Kaplan-Meier estimator and Log-rank test achieved the most significant difference between predicted groups of patients (short-term versus long-term survival) with p\,=\,4.31$\times$10$^{-7}$ compared to p\,=\,0.03 for Immune cell markers, p\,=\,0.07 for clinical variables , and p\,=\,1.45$\times$10$^{-5}$ for DRFs. Our findings indicate that the proposed features (DRFs) used in RF models may significantly consider prognosticating patients with brain tumour prior to surgery through regularly acquired imaging data.
Initial Study into Application of Feature Density and Linguistically-backed Embedding to Improve Machine Learning-based Cyberbullying Detection
Authors: Juuso Eronen, Michal Ptaszynski, Fumito Masui, Gniewosz Leliwa, Michal Wroczynski, Mateusz Piech, Aleksander Smywinski-Pohl
Date: 2022-06-04 00:00:00

In this research, we study the change in the performance of machine learning (ML) classifiers when various linguistic preprocessing methods of a dataset were used, with the specific focus on linguistically-backed embeddings in Convolutional Neural Networks (CNN). Moreover, we study the concept of Feature Density and confirm its potential to comparatively predict the performance of ML classifiers, including CNN. The research was conducted on a Formspring dataset provided in a Kaggle competition on automatic cyberbullying detection. The dataset was re-annotated by objective experts (psychologists), as the importance of professional annotation in cyberbullying research has been indicated multiple times. The study confirmed the effectiveness of Neural Networks in cyberbullying detection and the correlation between classifier performance and Feature Density while also proposing a new approach of training various linguistically-backed embeddings for Convolutional Neural Networks.
Reward Poisoning Attacks on Offline Multi-Agent Reinforcement Learning
Authors: Young Wu, Jeremy McMahan, Xiaojin Zhu, Qiaomin Xie
Date: 2022-06-04 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In offline multi-agent reinforcement learning (MARL), agents estimate policies from a given dataset. We study reward-poisoning attacks in this setting where an exogenous attacker modifies the rewards in the dataset before the agents see the dataset. The attacker wants to guide each agent into a nefarious target policy while minimizing the $L^p$ norm of the reward modification. Unlike attacks on single-agent RL, we show that the attacker can install the target policy as a Markov Perfect Dominant Strategy Equilibrium (MPDSE), which rational agents are guaranteed to follow. This attack can be significantly cheaper than separate single-agent attacks. We show that the attack works on various MARL agents including uncertainty-aware learners, and we exhibit linear programs to efficiently solve the attack problem. We also study the relationship between the structure of the datasets and the minimal attack cost. Our work paves the way for studying defense in offline MARL.
A Superimposed Divide-and-Conquer Image Recognition Method for SEM Images of Nanoparticles on The Surface of Monocrystalline silicon with High Aggregation Degree
Authors: Ruiling Xiao, Jiayang Niu
Date: 2022-06-04 00:00:00

The nanoparticle size and distribution information in the SEM images of silicon crystals are generally counted by manual methods. The realization of automatic machine recognition is significant in materials science. This paper proposed a superposition partitioning image recognition method to realize automatic recognition and information statistics of silicon crystal nanoparticle SEM images. Especially for the complex and highly aggregated characteristics of silicon crystal particle size, an accurate recognition step and contour statistics method based on morphological processing are given. This method has technical reference value for the recognition of Monocrystalline silicon surface nanoparticle images under different SEM shooting conditions. Besides, it outperforms other methods in terms of recognition accuracy and algorithm efficiency.
Estimating the Effect of Team Hitting Strategies Using Counterfactual Virtual Simulation in Baseball
Authors: Hiroshi Nakahara, Kazuya Takeda, Keisuke Fujii
Date: 2022-06-04 00:00:00
Tasks: counterfactual

In baseball, every play on the field is quantitatively evaluated and has an effect on individual and team strategies. The weighted on base average (wOBA) is well known as a measure of an batter's hitting contribution. However, this measure ignores the game situation, such as the runners on base, which coaches and batters are known to consider when employing multiple hitting strategies, yet, the effectiveness of these strategies is unknown. This is probably because (1) we cannot obtain the batter's strategy and (2) it is difficult to estimate the effect of the strategies. Here, we propose a new method for estimating the effect using counterfactual batting simulation. To this end, we propose a deep learning model that transforms batting ability when batting strategy is changed. This method can estimate the effects of various strategies, which has been traditionally difficult with actual game data. We found that, when the switching cost of batting strategies can be ignored, the use of different strategies increased runs. When the switching cost is considered, the conditions for increasing runs were limited. Our validation results suggest that our simulation could clarify the effect of using multiple batting strategies.
Design and Implementation of an Heuristic-Enhanced Branch-and-Bound Solver for MILP
Authors: Warley Almeida Silva, Federico Bobbio, Flore Caye, Defeng Liu, Justine Pepin, Carl Perreault-Lafleur, William St-Arnaud
Date: 2022-06-04 00:00:00

We present a solver for Mixed Integer Programs (MIP) developed for the MIP competition 2022. Given the 10 minutes bound on the computational time established by the rules of the competition, our method focuses on finding a feasible solution and improves it through a Branch-and-Bound algorithm. Another rule of the competition allows the use of up to 8 threads. Each thread is given a different primal heuristic, which has been tuned by hyper-parameters, to find a feasible solution. In every thread, once a feasible solution is found, we stop and we use a Branch-and-Bound method, embedded with local search heuristics, to ameliorate the incumbent solution. The three variants of the Diving heuristic that we implemented manage to find a feasible solution for 10 instances of the training data set. These heuristics are the best performing among the heuristics that we implemented. Our Branch-and-Bound algorithm is effective on a small portion of the training data set, and it manages to find an incumbent feasible solution for an instance that we could not solve with the Diving heuristics. Overall, our combined methods, when implemented with extensive computational power, can solve 11 of the 19 problems of the training data set within the time limit. Our submission to the MIP competition was awarded the "Outstanding Student Submission" honorable mention.
Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning
Authors: Yujia Xie, Luowei Zhou, Xiyang Dai, Lu Yuan, Nguyen Bach, Ce Liu, Michael Zeng
Date: 2022-06-03 00:00:00
Tasks: Image Paragraph Captioning, Language Modeling, Language Modelling, Large Language Model

People say, "A picture is worth a thousand words". Then how can we get the rich information out of the image? We argue that by using visual clues to bridge large pretrained vision foundation models and language models, we can do so without any extra cross-modal training. Thanks to the strong zero-shot capability of foundation models, we start by constructing a rich semantic representation of the image (e.g., image tags, object attributes / locations, captions) as a structured textual prompt, called visual clues, using a vision foundation model. Based on visual clues, we use large language model to produce a series of comprehensive descriptions for the visual content, which is then verified by the vision model again to select the candidate that aligns best with the image. We evaluate the quality of generated descriptions by quantitative and qualitative measurement. The results demonstrate the effectiveness of such a structured semantic representation.
Coffee Roast Intelligence
Authors: Sakdipat Ontoum, Thitaree Khemanantakul, Pornphat Sroison, Tuul Triyason, Bunthit Watanapa
Date: 2022-06-03 00:00:00

As the coffee industry has grown, there would be more demand for roasted coffee beans, as well as increased rivalry for selling coffee and attracting customers. As the flavor of each variety of coffee is dependent on the degree of roasting of the coffee beans, it is vital to maintain a consistent quality related to the degree of roasting. Each barista has their own method for determining the degree of roasting. However, extrinsic circumstances such as light, fatigue, and other factors may alter their judgment. As a result, the quality of the coffee cannot be controlled. The Coffee Roast Intelligence application is a machine learning-based study of roasted coffee bean degrees classification produced as an Android application platform that identifies the color of coffee beans by photographing or uploading them while roasting. This application displays the text showing at what level the coffee beans have been roasted, as well as informs the percent chance of class prediction to the consumers. Users may also keep track of the result of the predictions related to the roasting level of coffee beans.
Drawing out of Distribution with Neuro-Symbolic Generative Models
Authors: Yichao Liang, Joshua B. Tenenbaum, Tuan Anh Le, N. Siddharth
Date: 2022-06-03 00:00:00

Learning general-purpose representations from perceptual inputs is a hallmark of human intelligence. For example, people can write out numbers or characters, or even draw doodles, by characterizing these tasks as different instantiations of the same generic underlying process -- compositional arrangements of different forms of pen strokes. Crucially, learning to do one task, say writing, implies reasonable competence at another, say drawing, on account of this shared process. We present Drawing out of Distribution (DooD), a neuro-symbolic generative model of stroke-based drawing that can learn such general-purpose representations. In contrast to prior work, DooD operates directly on images, requires no supervision or expensive test-time inference, and performs unsupervised amortised inference with a symbolic stroke model that better enables both interpretability and generalization. We evaluate DooD on its ability to generalise across both data and tasks. We first perform zero-shot transfer from one dataset (e.g. MNIST) to another (e.g. Quickdraw), across five different datasets, and show that DooD clearly outperforms different baselines. An analysis of the learnt representations further highlights the benefits of adopting a symbolic stroke model. We then adopt a subset of the Omniglot challenge tasks, and evaluate its ability to generate new exemplars (both unconditionally and conditionally), and perform one-shot classification, showing that DooD matches the state of the art. Taken together, we demonstrate that DooD does indeed capture general-purpose representations across both data and task, and takes a further step towards building general and robust concept-learning systems.
HDDL 2.1: Towards Defining an HTN Formalism with Time
Authors: D. Pellier, H. Fiorino, M. Grand, A. Albore, R. Bailon-Ruiz
Date: 2022-06-03 00:00:00

Real world applications of planning, like in industry and robotics, require modelling rich and diverse scenarios. Their resolution usually requires coordinated and concurrent action executions. In several cases, such planning problems are naturally decomposed in a hierarchical way and expressed by a Hierarchical Task Network (HTN) formalism. The PDDL language used to specify planning domains has evolved to cover the different planning paradigms. However, formulating real and complex scenarios where numerical and temporal constraints concur in defining a solution is still a challenge. Our proposition aims at filling the gap between existing planning languages and operational needs. To do so, we propose to extend HDDL taking inspiration from PDDL 2.1 and ANML to express temporal and numerical expressions. This paper opens discussions on the semantics and the syntax needed to extend HDDL, and illustrate these needs with the modelling of an Earth Observing Satellite planning problem.
QAGCN: Answering Multi-Relation Questions via Single-Step Implicit Reasoning over Knowledge Graphs
Authors: Ruijie Wang, Luca Rossetto, Michael Cochez, Abraham Bernstein
Date: 2022-06-03 00:00:00
Tasks: Decision Making, Knowledge Graphs, Navigate, Question Answering, Relation

Multi-relation question answering (QA) is a challenging task, where given questions usually require long reasoning chains in KGs that consist of multiple relations. Recently, methods with explicit multi-step reasoning over KGs have been prominently used in this task and have demonstrated promising performance. Examples include methods that perform stepwise label propagation through KG triples and methods that navigate over KG triples based on reinforcement learning. A main weakness of these methods is that their reasoning mechanisms are usually complex and difficult to implement or train. In this paper, we argue that multi-relation QA can be achieved via end-to-end single-step implicit reasoning, which is simpler, more efficient, and easier to adopt. We propose QAGCN -- a Question-Aware Graph Convolutional Network (GCN)-based method that includes a novel GCN architecture with controlled question-dependent message propagation for the implicit reasoning. Extensive experiments have been conducted, where QAGCN achieved competitive and even superior performance compared to state-of-the-art explicit-reasoning methods. Our code and pre-trained models are available in the repository: https://github.com/ruijie-wang-uzh/QAGCN
Option Discovery for Autonomous Generation of Symbolic Knowledge
Authors: Gabriele Sartor, Davide Zollo, Marta Cialdea Mayer, Angelo Oddi, Riccardo Rasconi, Vieri Giuliano Santucci
Date: 2022-06-03 00:00:00

In this work we present an empirical study where we demonstrate the possibility of developing an artificial agent that is capable to autonomously explore an experimental scenario. During the exploration, the agent is able to discover and learn interesting options allowing to interact with the environment without any pre-assigned goal, then abstract and re-use the acquired knowledge to solve possible tasks assigned ex-post. We test the system in the so-called Treasure Game domain described in the recent literature and we empirically demonstrate that the discovered options can be abstracted in an probabilistic symbolic planning model (using the PPDDL language), which allowed the agent to generate symbolic plans to achieve extrinsic goals.