pwc-archive/papers-with-abstracts huggingface dataset:
HMSG: Heterogeneous Graph Neural Network based on Metapath Subgraph Learning
Authors: Xinjun Cai, Jiaxing Shang, Fei Hao, Dajiang Liu, Linjiang Zheng
Date: 2021-09-07 00:00:00
Tasks: Attribute, Graph Neural Network, Link Prediction, Node Classification, Node Clustering

Many real-world data can be represented as heterogeneous graphs with different types of nodes and connections. Heterogeneous graph neural network model aims to embed nodes or subgraphs into low-dimensional vector space for various downstream tasks such as node classification, link prediction, etc. Although several models were proposed recently, they either only aggregate information from the same type of neighbors, or just indiscriminately treat homogeneous and heterogeneous neighbors in the same way. Based on these observations, we propose a new heterogeneous graph neural network model named HMSG to comprehensively capture structural, semantic and attribute information from both homogeneous and heterogeneous neighbors. Specifically, we first decompose the heterogeneous graph into multiple metapath-based homogeneous and heterogeneous subgraphs, and each subgraph associates specific semantic and structural information. Then message aggregation methods are applied to each subgraph independently, so that information can be learned in a more targeted and efficient manner. Through a type-specific attribute transformation, node attributes can also be transferred among different types of nodes. Finally, we fuse information from subgraphs together to get the complete representation. Extensive experiments on several datasets for node classification, node clustering and link prediction tasks show that HMSG achieves the best performance in all evaluation metrics than state-of-the-art baselines.
Dutch Comfort: The limits of AI governance through municipal registers
Authors: Corinne Cath, Fieke Jansen
Date: 2021-09-07 00:00:00
Tasks: Ethics, Management

In this commentary, we respond to a recent editorial letter by Professor Luciano Floridi entitled 'AI as a public service: Learning from Amsterdam and Helsinki'. Here, Floridi considers the positive impact of these municipal AI registers, which collect a limited number of algorithmic systems used by the city of Amsterdam and Helsinki. There are a number of assumptions about AI registers as a governance model for automated systems that we seek to question. Starting with recent attempts to normalize AI by decontextualizing and depoliticizing it, which is a fraught political project that encourages what we call 'ethics theater' given the proven dangers of using these systems in the context of the digital welfare state. We agree with Floridi that much can be learned from these registers about the role of AI systems in municipal city management. Yet, the lessons we draw, on the basis of our extensive ethnographic engagement with digital well-fare states are distinctly less optimistic.
Smart Automotive Technology Adherence to the Law: (De)Constructing Road Rules for Autonomous System Development, Verification and Safety
Authors: Scott McLachlan, Martin Neil, Kudakwashe Dube, Ronny Bogani, Norman Fenton, Burkhard Schaffer
Date: 2021-09-07 00:00:00
Tasks: Autonomous Driving

Driving is an intuitive task that requires skills, constant alertness and vigilance for unexpected events. The driving task also requires long concentration spans focusing on the entire task for prolonged periods, and sophisticated negotiation skills with other road users, including wild animals. These requirements are particularly important when approaching intersections, overtaking, giving way, merging, turning and while adhering to the vast body of road rules. Modern motor vehicles now include an array of smart assistive and autonomous driving systems capable of subsuming some, most, or in limited cases, all of the driving task. The UK Department of Transport's response to the Safe Use of Automated Lane Keeping System consultation proposes that these systems are tested for compliance with relevant traffic rules. Building these smart automotive systems requires software developers with highly technical software engineering skills, and now a lawyer's in-depth knowledge of traffic legislation as well. These skills are required to ensure the systems are able to safely perform their tasks while being observant of the law. This paper presents an approach for deconstructing the complicated legalese of traffic law and representing its requirements and flow. The approach (de)constructs road rules in legal terminology and specifies them in structured English logic that is expressed as Boolean logic for automation and Lawmaps for visualisation. We demonstrate an example using these tools leading to the construction and validation of a Bayesian Network model. We strongly believe these tools to be approachable by programmers and the general public, and capable of use in developing Artificial Intelligence to underpin motor vehicle smart systems, and in validation to ensure these systems are considerate of the law when making decisions.
Sequential Attention Module for Natural Language Processing
Authors: Mengyuan Zhou, Jian Ma, Haiqin Yang, Lianxin Jiang, Yang Mo
Date: 2021-09-07 00:00:00
Tasks: Language Modelling, Sentiment Analysis

Recently, large pre-trained neural language models have attained remarkable performance on many downstream natural language processing (NLP) applications via fine-tuning. In this paper, we target at how to further improve the token representations on the language models. We, therefore, propose a simple yet effective plug-and-play module, Sequential Attention Module (SAM), on the token embeddings learned from a pre-trained language model. Our proposed SAM consists of two main attention modules deployed sequentially: Feature-wise Attention Module (FAM) and Token-wise Attention Module (TAM). More specifically, FAM can effectively identify the importance of features at each dimension and promote the effect via dot-product on the original token embeddings for downstream NLP applications. Meanwhile, TAM can further re-weight the features at the token-wise level. Moreover, we propose an adaptive filter on FAM to prevent noise impact and increase information absorption. Finally, we conduct extensive experiments to demonstrate the advantages and properties of our proposed SAM. We first show how SAM plays a primary role in the champion solution of two subtasks of SemEval'21 Task 7. After that, we apply SAM on sentiment analysis and three popular NLP tasks and demonstrate that SAM consistently outperforms the state-of-the-art baselines.
Fudge: A light-weight solver for abstract argumentation based on SAT reductions
Authors: Matthias Thimm, Federico Cerutti, Mauro Vallati
Date: 2021-09-07 00:00:00
Tasks: Abstract Argumentation, Translation

We present Fudge, an abstract argumentation solver that tightly integrates satisfiability solving technology to solve a series of abstract argumentation problems. While most of the encodings used by Fudge derive from standard translation approaches, Fudge makes use of completely novel encodings to solve the skeptical reasoning problem wrt. preferred semantics and problems wrt. ideal semantics.
The pyglaf argumentation reasoner (ICCMA2021)
Authors: Mario Alviano
Date: 2021-09-07 00:00:00
Tasks: Abstract Argumentation

The pyglaf reasoner takes advantage of circumscription to solve computational problems of abstract argumentation frameworks. In fact, many of these problems are reduced to circumscription by means of linear encodings, and a few others are solved by means of a sequence of calls to an oracle for circumscription. Within pyglaf, Python is used to build the encodings and to control the execution of the external circumscription solver, which extends the SAT solver glucose and implements algorithms taking advantage of unsatisfiable core analysis and incremental computation.
Aspartix-V21
Authors: Wolfgang Dvořák, Matthias König, Johannes P. Wallner, Stefan Woltran
Date: 2021-09-07 00:00:00

In this solver description we present ASPARTIX-V, in its 2021 edition, which participates in the International Competition on Computational Models of Argumentation (ICCMA) 2021. ASPARTIX-V is capable of solving all classical (static) reasoning tasks part of ICCMA'21 and extends the ASPARTIX system suite by incorporation of recent ASP language constructs (e.g. conditional literals), domain heuristics within ASP, and multi-shot methods. In this light ASPARTIX-V deviates from the traditional focus of ASPARTIX on monolithic approaches (i.e., one-shot solving via a single ASP encoding) to further enhance performance.
ExCode-Mixed: Explainable Approaches towards Sentiment Analysis on Code-Mixed Data using BERT models
Authors: Aman Priyanshu, Aleti Vardhan, Sudarshan Sivakumar, Supriti Vijay, Nipuna Chhabra
Date: 2021-09-07 00:00:00
Tasks: Sentiment Analysis

The increasing use of social media sites in countries like India has given rise to large volumes of code-mixed data. Sentiment analysis of this data can provide integral insights into people's perspectives and opinions. Developing robust explainability techniques which explain why models make their predictions becomes essential. In this paper, we propose an adequate methodology to integrate explainable approaches into code-mixed sentiment analysis.
On the impact of MDP design for Reinforcement Learning agents in Resource Management
Authors: Renato Luiz de Freitas Cunha, Luiz Chaimowicz
Date: 2021-09-07 00:00:00
Tasks: Management, reinforcement-learning, Reinforcement Learning (RL)

The recent progress in Reinforcement Learning applications to Resource Management presents MDPs without a deeper analysis of the impacts of design decisions on agent performance. In this paper, we compare and contrast four different MDP variations, discussing their computational requirements and impacts on agent performance by means of an empirical analysis. We conclude by showing that, in our experiments, when using Multi-Layer Perceptrons as approximation function, a compact state representation allows transfer of agents between environments, and that transferred agents have good performance and outperform specialized agents in 80\% of the tested scenarios, even without retraining.
Graph Attention Layer Evolves Semantic Segmentation for Road Pothole Detection: A Benchmark and Algorithms
Authors: Rui Fan, Hengli Wang, YuAn Wang, Ming Liu, Ioannis Pitas
Date: 2021-09-06 00:00:00
Tasks: Graph Attention, Graph Neural Network, Segmentation, Semantic Segmentation

Existing road pothole detection approaches can be classified as computer vision-based or machine learning-based. The former approaches typically employ 2-D image analysis/understanding or 3-D point cloud modeling and segmentation algorithms to detect road potholes from vision sensor data. The latter approaches generally address road pothole detection using convolutional neural networks (CNNs) in an end-to-end manner. However, road potholes are not necessarily ubiquitous and it is challenging to prepare a large well-annotated dataset for CNN training. In this regard, while computer vision-based methods were the mainstream research trend in the past decade, machine learning-based methods were merely discussed. Recently, we published the first stereo vision-based road pothole detection dataset and a novel disparity transformation algorithm, whereby the damaged and undamaged road areas can be highly distinguished. However, there are no benchmarks currently available for state-of-the-art (SoTA) CNNs trained using either disparity images or transformed disparity images. Therefore, in this paper, we first discuss the SoTA CNNs designed for semantic segmentation and evaluate their performance for road pothole detection with extensive experiments. Additionally, inspired by graph neural network (GNN), we propose a novel CNN layer, referred to as graph attention layer (GAL), which can be easily deployed in any existing CNN to optimize image feature representations for semantic segmentation. Our experiments compare GAL-DeepLabv3+, our best-performing implementation, with nine SoTA CNNs on three modalities of training data: RGB images, disparity images, and transformed disparity images. The experimental results suggest that our proposed GAL-DeepLabv3+ achieves the best overall pothole detection accuracy on all training data modalities.
Puzzle Solving without Search or Human Knowledge: An Unnatural Language Approach
Authors: David Noever, Ryerson Burdick
Date: 2021-09-07 00:00:00
Tasks: Rubik's Cube

The application of Generative Pre-trained Transformer (GPT-2) to learn text-archived game notation provides a model environment for exploring sparse reward gameplay. The transformer architecture proves amenable to training on solved text archives describing mazes, Rubik's Cube, and Sudoku solvers. The method benefits from fine-tuning the transformer architecture to visualize plausible strategies derived outside any guidance from human heuristics or domain expertise. The large search space ($>10^{19}$) for the games provides a puzzle environment in which the solution has few intermediate rewards and a final move that solves the challenge.
Symbolic Computation in Software Science: My Personal View
Authors: Bruno Buchberger
Date: 2021-09-07 00:00:00
Tasks: Management

In this note, I develop my personal view on the scope and relevance of symbolic computation in software science. For this, I discuss the interaction and differences between symbolic computation, software science, automatic programming, mathematical knowledge management, artificial intelligence, algorithmic intelligence, numerical computation, and machine learning. In the discussion of these notions, I allow myself to refer also to papers (1982, 1985, 2001, 2003, 2013) of mine in which I expressed my views on these areas at early stages of some of these fields.
Few-shot Learning via Dependency Maximization and Instance Discriminant Analysis
Authors: Zejiang Hou, Sun-Yuan Kung
Date: 2021-09-07 00:00:00
Tasks: Few-Shot Learning, Inductive Bias, Meta-Learning

We study the few-shot learning (FSL) problem, where a model learns to recognize new objects with extremely few labeled training data per category. Most of previous FSL approaches resort to the meta-learning paradigm, where the model accumulates inductive bias through learning many training tasks so as to solve a new unseen few-shot task. In contrast, we propose a simple approach to exploit unlabeled data accompanying the few-shot task for improving few-shot performance. Firstly, we propose a Dependency Maximization method based on the Hilbert-Schmidt norm of the cross-covariance operator, which maximizes the statistical dependency between the embedded feature of those unlabeled data and their label predictions, together with the supervised loss over the support set. We then use the obtained model to infer the pseudo-labels for those unlabeled data. Furthermore, we propose anInstance Discriminant Analysis to evaluate the credibility of each pseudo-labeled example and select the most faithful ones into an augmented support set to retrain the model as in the first step. We iterate the above process until the pseudo-labels for the unlabeled data becomes stable. Following the standard transductive and semi-supervised FSL setting, our experiments show that the proposed method out-performs previous state-of-the-art methods on four widely used benchmarks, including mini-ImageNet, tiered-ImageNet, CUB, and CIFARFS.
Learning Visual-Audio Representations for Voice-Controlled Robots
Authors: Peixin Chang, Shuijing Liu, D. Livingston McPherson, Katherine Driggs-Campbell
Date: 2021-09-07 00:00:00
Tasks: Representation Learning

Based on the recent advancements in representation learning, we propose a novel pipeline for task-oriented voice-controlled robots with raw sensor inputs. Previous methods rely on a large number of labels and task-specific reward functions. Not only can such an approach hardly be improved after the deployment, but also has limited generalization across robotic platforms and tasks. To address these problems, our pipeline first learns a visual-audio representation (VAR) that associates images and sound commands. Then the robot learns to fulfill the sound command via reinforcement learning using the reward generated by the VAR. We demonstrate our approach with various sound types, robots, and tasks. We show that our method outperforms previous work with much fewer labels. We show in both the simulated and real-world experiments that the system can self-improve in previously unseen scenarios given a reasonable number of newly labeled data.
Hierarchical Graph Convolutional Skeleton Transformer for Action Recognition
Authors: Ruwen Bai, Min Li, Bo Meng, Fengfa Li, Miao Jiang, Junxing Ren, Degang Sun
Date: 2021-09-07 00:00:00
Tasks: Action Recognition, Disentanglement, Skeleton Based Action Recognition

Graph convolutional networks (GCNs) have emerged as dominant methods for skeleton-based action recognition. However, they still suffer from two problems, namely, neighborhood constraints and entangled spatiotemporal feature representations. Most studies have focused on improving the design of graph topology to solve the first problem but they have yet to fully explore the latter. In this work, we design a disentangled spatiotemporal transformer (DSTT) block to overcome the above limitations of GCNs in three steps: (i) feature disentanglement for spatiotemporal decomposition;(ii) global spatiotemporal attention for capturing correlations in the global context; and (iii) local information enhancement for utilizing more local information. Thereon, we propose a novel architecture, named Hierarchical Graph Convolutional skeleton Transformer (HGCT), to employ the complementary advantages of GCN (i.e., local topology, temporal dynamics and hierarchy) and Transformer (i.e., global context and dynamic attention). HGCT is lightweight and computationally efficient. Quantitative analysis demonstrates the superiority and good interpretability of HGCT.
Prescriptive Process Monitoring Under Resource Constraints: A Causal Inference Approach
Authors: Mahmoud Shoush, Marlon Dumas
Date: 2021-09-07 00:00:00
Tasks: Causal Inference

Prescriptive process monitoring is a family of techniques to optimize the performance of a business process by triggering interventions at runtime. Existing prescriptive process monitoring techniques assume that the number of interventions that may be triggered is unbounded. In practice, though, specific interventions consume resources with finite capacity. For example, in a loan origination process, an intervention may consist of preparing an alternative loan offer to increase the applicant's chances of taking a loan. This intervention requires a certain amount of time from a credit officer, and thus, it is not possible to trigger this intervention in all cases. This paper proposes a prescriptive process monitoring technique that triggers interventions to optimize a cost function under fixed resource constraints. The proposed technique relies on predictive modeling to identify cases that are likely to lead to a negative outcome, in combination with causal inference to estimate the effect of an intervention on the outcome of the case. These outputs are then used to allocate resources to interventions to maximize a cost function. A preliminary empirical evaluation suggests that the proposed approach produces a higher net gain than a purely predictive (non-causal) baseline.
Blockchains through ontologies: the case study of the Ethereum ERC721 standard in OASIS (Extended Version)
Authors: Giampaolo Bella, Domenico Cantone, Cristiano Longo, Marianna Nicolosi-Asmundo, Daniele Francesco Santamaria
Date: 2021-09-07 00:00:00
Tasks: Management

Blockchains are gaining momentum due to the interest of industries and people in \emph{decentralized applications} (Dapps), particularly in those for trading assets through digital certificates secured on blockchain, called tokens. As a consequence, providing a clear unambiguous description of any activities carried out on blockchains has become crucial, and we feel the urgency to achieve that description at least for trading. This paper reports on how to leverage the \emph{Ontology for Agents, Systems, and Integration of Services} ("\ONT{}") as a general means for the semantic representation of smart contracts stored on blockchain as software agents. Special attention is paid to non-fungible tokens (NFTs), whose management through the ERC721 standard is presented as a case study.
IndicBART: A Pre-trained Model for Indic Natural Language Generation
Authors: Raj Dabre, Himani Shrotriya, Anoop Kunchukuttan, Ratish Puduppully, Mitesh M. Khapra, Pratyush Kumar
Date: 2021-09-07 00:00:00
Proceeding: Findings (ACL) 2022 5
Tasks: Extreme Summarization, Machine Translation, NMT, Text Generation, Transfer Learning, Translation

In this paper, we study pre-trained sequence-to-sequence models for a group of related languages, with a focus on Indic languages. We present IndicBART, a multilingual, sequence-to-sequence pre-trained model focusing on 11 Indic languages and English. IndicBART utilizes the orthographic similarity between Indic scripts to improve transfer learning between similar Indic languages. We evaluate IndicBART on two NLG tasks: Neural Machine Translation (NMT) and extreme summarization. Our experiments on NMT and extreme summarization show that a model specific to related languages like IndicBART is competitive with large pre-trained models like mBART50 despite being significantly smaller. It also performs well on very low-resource translation scenarios where languages are not included in pre-training or fine-tuning. Script sharing, multilingual training, and better utilization of limited model capacity contribute to the good performance of the compact IndicBART model.
Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization
Authors: Alexandre Rame, Corentin Dancette, Matthieu Cord
Date: 2021-09-07 00:00:00
Tasks: Domain Generalization, Out-of-Distribution Generalization

Learning robust models that generalize well under changes in the data distribution is critical for real-world applications. To this end, there has been a growing surge of interest to learn simultaneously from multiple training domains - while enforcing different types of invariance across those domains. Yet, all existing approaches fail to show systematic benefits under controlled evaluation protocols. In this paper, we introduce a new regularization - named Fishr - that enforces domain invariance in the space of the gradients of the loss: specifically, the domain-level variances of gradients are matched across training domains. Our approach is based on the close relations between the gradient covariance, the Fisher Information and the Hessian of the loss: in particular, we show that Fishr eventually aligns the domain-level loss landscapes locally around the final weights. Extensive experiments demonstrate the effectiveness of Fishr for out-of-distribution generalization. Notably, Fishr improves the state of the art on the DomainBed benchmark and performs consistently better than Empirical Risk Minimization. Our code is available at https://github.com/alexrame/fishr.
Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT
Authors: Ye Liu, Wolfgang Maier, Wolfgang Minker, Stefan Ultes
Date: 2021-09-07 00:00:00
Proceeding: RANLP 2021 9
Tasks: Informativeness, Text Generation, Transfer Learning

This paper presents an automatic method to evaluate the naturalness of natural language generation in dialogue systems. While this task was previously rendered through expensive and time-consuming human labor, we present this novel task of automatic naturalness evaluation of generated language. By fine-tuning the BERT model, our proposed naturalness evaluation method shows robust results and outperforms the baselines: support vector machines, bi-directional LSTMs, and BLEURT. In addition, the training speed and evaluation performance of naturalness model are improved by transfer learning from quality and informativeness linguistic knowledge.