pwc-archive/papers-with-abstracts huggingface dataset:
Evaluating Generalization and Transfer Capacity of Multi-Agent Reinforcement Learning Across Variable Number of Agents
Authors: Bengisu Guresti, Nazim Kemal Ure
Date: 2021-11-28 00:00:00
Tasks: Multi-agent Reinforcement Learning

Multi-agent Reinforcement Learning (MARL) problems often require cooperation among agents in order to solve a task. Centralization and decentralization are two approaches used for cooperation in MARL. While fully decentralized methods are prone to converge to suboptimal solutions due to partial observability and nonstationarity, the methods involving centralization suffer from scalability limitations and lazy agent problem. Centralized training decentralized execution paradigm brings out the best of these two approaches; however, centralized training still has an upper limit of scalability not only for acquired coordination performance but also for model size and training time. In this work, we adopt the centralized training with decentralized execution paradigm and investigate the generalization and transfer capacity of the trained models across variable number of agents. This capacity is assessed by training variable number of agents in a specific MARL problem and then performing greedy evaluations with variable number of agents for each training configuration. Thus, we analyze the evaluation performance for each combination of agent count for training versus evaluation. We perform experimental evaluations on predator prey and traffic junction environments and demonstrate that it is possible to obtain similar or higher evaluation performance by training with less agents. We conclude that optimal number of agents to perform training may differ from the target number of agents and argue that transfer across large number of agents can be a more efficient solution to scaling up than directly increasing number of agents during training.
Learning Physical Concepts in Cyber-Physical Systems: A Case Study
Authors: Henrik S. Steude, Alexander Windmann, Oliver Niggemann
Date: 2021-11-28 00:00:00
Tasks: Representation Learning, Time Series Analysis, Transfer Learning

Machine Learning (ML) has achieved great successes in recent decades, both in research and in practice. In Cyber-Physical Systems (CPS), ML can for example be used to optimize systems, to detect anomalies or to identify root causes of system failures. However, existing algorithms suffer from two major drawbacks: (i) They are hard to interpret by human experts. (ii) Transferring results from one systems to another (similar) system is often a challenge. Concept learning, or Representation Learning (RepL), is a solution to both of these drawbacks; mimicking the human solution approach to explain-ability and transfer-ability: By learning general concepts such as physical quantities or system states, the model becomes interpretable by humans. Furthermore concepts on this abstract level can normally be applied to a wide range of different systems. Modern ML methods are already widely used in CPS, but concept learning and transfer learning are hardly used so far. In this paper, we provide an overview of the current state of research regarding methods for learning physical concepts in time series data, which is the primary form of sensor data of CPS. We also analyze the most important methods from the current state of the art using the example of a three-tank system. Based on these concrete implementations1, we discuss the advantages and disadvantages of the methods and show for which purpose and under which conditions they can be used.
FashionSearchNet-v2: Learning Attribute Representations with Localization for Image Retrieval with Attribute Manipulation
Authors: Kenan E. Ak, Joo Hwee Lim, Ying Sun, Jo Yew Tham, Ashraf A. Kassim
Date: 2021-11-28 00:00:00
Tasks: Attribute, Image Retrieval, Retrieval, Triplet

The focus of this paper is on the problem of image retrieval with attribute manipulation. Our proposed work is able to manipulate the desired attributes of the query image while maintaining its other attributes. For example, the collar attribute of the query image can be changed from round to v-neck to retrieve similar images from a large dataset. A key challenge in e-commerce is that images have multiple attributes where users would like to manipulate and it is important to estimate discriminative feature representations for each of these attributes. The proposed FashionSearchNet-v2 architecture is able to learn attribute specific representations by leveraging on its weakly-supervised localization module, which ignores the unrelated features of attributes in the feature space, thus improving the similarity learning. The network is jointly trained with the combination of attribute classification and triplet ranking loss to estimate local representations. These local representations are then merged into a single global representation based on the instructed attribute manipulation where desired images can be retrieved with a distance metric. The proposed method also provides explainability for its retrieval process to help provide additional information on the attention of the network. Experiments performed on several datasets that are rich in terms of the number of attributes show that FashionSearchNet-v2 outperforms the other state-of-the-art attribute manipulation techniques. Different than our earlier work (FashionSearchNet), we propose several improvements in the learning procedure and show that the proposed FashionSearchNet-v2 can be generalized to different domains other than fashion.
Agility in Software 2.0 -- Notebook Interfaces and MLOps with Buttresses and Rebars
Authors: Markus Borg
Date: 2021-11-28 00:00:00
Tasks: BIG-bench Machine Learning

Artificial intelligence through machine learning is increasingly used in the digital society. Solutions based on machine learning bring both great opportunities, thus coined "Software 2.0," but also great challenges for the engineering community to tackle. Due to the experimental approach used by data scientists when developing machine learning models, agility is an essential characteristic. In this keynote address, we discuss two contemporary development phenomena that are fundamental in machine learning development, i.e., notebook interfaces and MLOps. First, we present a solution that can remedy some of the intrinsic weaknesses of working in notebooks by supporting easy transitions to integrated development environments. Second, we propose reinforced engineering of AI systems by introducing metaphorical buttresses and rebars in the MLOps context. Machine learning-based solutions are dynamic in nature, and we argue that reinforced continuous engineering is required to quality assure the trustworthy AI systems of tomorrow.
AirSPEC: An IoT-empowered Air Quality Monitoring System integrated with a Machine Learning Framework to Detect and Predict defined Air Quality parameters
Authors: Nuwan Bandara, Sahan Hettiarachchi, Phabhani Athukorala
Date: 2021-11-28 00:00:00

The air that surrounds us is the cardinal source of respiration of all life-forms. Therefore, it is undoubtedly vital to highlight that balanced air quality is utmost important to the respiratory health of all living beings, environmental homeostasis, and even economical equilibrium. Nevertheless, a gradual deterioration of air quality has been observed in the last few decades, due to the continuous increment of polluted emissions from automobiles and industries into the atmosphere. Even though many people have scarcely acknowledged the depth of the problem, the persistent efforts of determined parties, including the World Health Organization, have consistently pushed the boundaries for a qualitatively better global air homeostasis, by facilitating technology-driven initiatives to timely detect and predict air quality in regional and global scales. However, the existing frameworks for air quality monitoring lack the capability of real-time responsiveness and flexible semantic distribution. In this paper, a novel Internet of Things framework is proposed which is easily implementable, semantically distributive, and empowered by a machine learning model. The proposed system is equipped with a NodeRED dashboard which processes, visualizes, and stores the primary sensor data that are acquired through a public air quality sensor network, and further, the dashboard is integrated with a machine-learning model to obtain temporal and geo-spatial air quality predictions. ESP8266 NodeMCU is incorporated as a subscriber to the NodeRED dashboard via a message queuing telemetry transport broker to communicate quantitative air quality data or alarming emails to the end-users through the developed web and mobile applications. Therefore, the proposed system could become highly beneficial in empowering public engagement in air quality through an unoppressive, data-driven, and semantic framework.
Topic Driven Adaptive Network for Cross-Domain Sentiment Classification
Authors: Yicheng Zhu, Yiqiao Qiu, Qingyuan Wu, Fu Lee Wang, Yanghui Rao
Date: 2021-11-28 00:00:00
Tasks: Classification, Domain Adaptation, Sentiment Analysis, Sentiment Classification, Topic Models

Cross-domain sentiment classification has been a hot spot these years, which aims to learn a reliable classifier using labeled data from a source domain and evaluate it on a target domain. In this vein, most approaches utilized domain adaptation that maps data from different domains into a common feature space. To further improve the model performance, several methods targeted to mine domain-specific information were proposed. However, most of them only utilized a limited part of domain-specific information. In this study, we first develop a method of extracting domain-specific words based on the topic information derived from topic models. Then, we propose a Topic Driven Adaptive Network (TDAN) for cross-domain sentiment classification. The network consists of two sub-networks: a semantics attention network and a domain-specific word attention network, the structures of which are based on transformers. These sub-networks take different forms of input and their outputs are fused as the feature vector. Experiments validate the effectiveness of our TDAN on sentiment classification across domains. Case studies also indicate that topic models have the potential to add value to cross-domain sentiment classification by discovering interpretable and low-dimensional subspaces.
Multicriteria interpretability driven Deep Learning
Authors: Marco Repetto
Date: 2021-11-28 00:00:00
Tasks: Deep Learning, valid

Deep Learning methods are renowned for their performances, yet their lack of interpretability prevents them from high-stakes contexts. Recent model agnostic methods address this problem by providing post-hoc interpretability methods by reverse-engineering the model's inner workings. However, in many regulated fields, interpretability should be kept in mind from the start, which means that post-hoc methods are valid only as a sanity check after model training. Interpretability from the start, in an abstract setting, means posing a set of soft constraints on the model's behavior by injecting knowledge and annihilating possible biases. We propose a Multicriteria technique that allows to control the feature effects on the model's outcome by injecting knowledge in the objective function. We then extend the technique by including a non-linear knowledge function to account for more complex effects and local lack of knowledge. The result is a Deep Learning model that embodies interpretability from the start and aligns with the recent regulations. A practical empirical example based on credit risk, suggests that our approach creates performant yet robust models capable of overcoming biases derived from data scarcity.
Towards Robust and Automatic Hyper-Parameter Tunning
Authors: Mathieu Tuli, Mahdi S. Hosseini, Konstantinos N. Plataniotis
Date: 2021-11-28 00:00:00
Tasks: Bayesian Optimization

The task of hyper-parameter optimization (HPO) is burdened with heavy computational costs due to the intractability of optimizing both a model's weights and its hyper-parameters simultaneously. In this work, we introduce a new class of HPO method and explore how the low-rank factorization of the convolutional weights of intermediate layers of a convolutional neural network can be used to define an analytical response surface for optimizing hyper-parameters, using only training data. We quantify how this surface behaves as a surrogate to model performance and can be solved using a trust-region search algorithm, which we call autoHyper. The algorithm outperforms state-of-the-art such as Bayesian Optimization and generalizes across model, optimizer, and dataset selection. Our code can be found at \url{https://github.com/MathieuTuli/autoHyper}.
Towards Conditional Generation of Minimal Action Potential Pathways for Molecular Dynamics
Authors: John Kevin Cava, John Vant, Nicholas Ho, Ankita Shukla, Pavan Turaga, Ross Maciejewski, Abhishek Singharoy
Date: 2021-11-28 00:00:00

In this paper, we utilized generative models, and reformulate it for problems in molecular dynamics (MD) simulation, by introducing an MD potential energy component to our generative model. By incorporating potential energy as calculated from TorchMD into a conditional generative framework, we attempt to construct a low-potential energy route of transformation between the helix~$\rightarrow$~coil structures of a protein. We show how to add an additional loss function to conditional generative models, motivated by potential energy of molecular configurations, and also present an optimization technique for such an augmented loss function. Our results show the benefit of this additional loss term on synthesizing realistic molecular trajectories.
ORCHARD: A Benchmark For Measuring Systematic Generalization of Multi-Hierarchical Reasoning
Authors: Bill Tuck Weng Pung, Alvin Chan
Date: 2021-11-28 00:00:00
Tasks: Diagnostic, ListOps, Relational Reasoning, Systematic Generalization

The ability to reason with multiple hierarchical structures is an attractive and desirable property of sequential inductive biases for natural language processing. Do the state-of-the-art Transformers and LSTM architectures implicitly encode for these biases? To answer this, we propose ORCHARD, a diagnostic dataset for systematically evaluating hierarchical reasoning in state-of-the-art neural sequence models. While there have been prior evaluation frameworks such as ListOps or Logical Inference, our work presents a novel and more natural setting where our models learn to reason with multiple explicit hierarchical structures instead of only one, i.e., requiring the ability to do both long-term sequence memorizing, relational reasoning while reasoning with hierarchical structure. Consequently, backed by a set of rigorous experiments, we show that (1) Transformer and LSTM models surprisingly fail in systematic generalization, and (2) with increased references between hierarchies, Transformer performs no better than random.
AI-supported Framework of Semi-Automatic Monoplotting for Monocular Oblique Visual Data Analysis
Authors: Behzad Golparvar, Ruo-Qian Wang
Date: 2021-11-28 00:00:00
Tasks: Camera Pose Estimation, Pose Estimation

In the last decades, the development of smartphones, drones, aerial patrols, and digital cameras enabled high-quality photographs available to large populations and, thus, provides an opportunity to collect massive data of the nature and society with global coverage. However, the data collected with new photography tools is usually oblique - they are difficult to be georeferenced, and huge amounts of data is often obsolete. Georeferencing oblique imagery data may be solved by a technique called monoplotting, which only requires a single image and Digital Elevation Model (DEM). In traditional monoplotting, a human user has to manually choose a series of ground control point (GCP) pairs in the image and DEM and then determine the extrinsic and intrinsic parameters of the camera to establish a pixel-level correspondence between photos and the DEM to enable the mapping and georeferencing of objects in photos. This traditional method is difficult to scale due to several challenges including the labor-intensive inputs, the need of rich experience to identify well-defined GCPs, and limitations in camera pose estimation. Therefore, existing monoplotting methods are rarely used in analyzing large-scale databases or near-real-time warning systems. In this paper, we propose and demonstrate a novel semi-automatic monoplotting framework that provides pixel-level correspondence between photos and DEMs requiring minimal human interventions. A pipeline of analyses was developed including key point detection in images and DEM rasters, retrieving georeferenced 3D DEM GCPs, regularized gradient-based optimization, pose estimation, ray tracing, and the correspondence identification between image pixels and real world coordinates. Two numerical experiments show that the framework is superior in georeferencing visual data in 3-D coordinates, paving a way toward fully automatic monoplotting methodology.
Language models in word sense disambiguation for Polish
Authors: Agnieszka Mykowiecka, Agnieszka A. Mykowiecka, Piotr Rychlik
Date: 2021-11-27 00:00:00
Tasks: Word Sense Disambiguation

In the paper, we test two different approaches to the {unsupervised} word sense disambiguation task for Polish. In both methods, we use neural language models to predict words similar to those being disambiguated and, on the basis of these words, we predict the partition of word senses in different ways. In the first method, we cluster selected similar words, while in the second, we cluster vectors representing their subsets. The evaluation was carried out on texts annotated with plWordNet senses and provided a relatively good result (F1=0.68 for all ambiguous words). The results are significantly better than those obtained for the neural model-based unsupervised method proposed in \cite{waw:myk:17:Sense} and are at the level of the supervised method presented there. The proposed method may be a way of solving word sense disambiguation problem for languages that lack sense annotated data.
Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection
Authors: Hooman Alavizadeh, Julian Jang-Jaccard, Hootan Alavizadeh
Date: 2021-11-27 00:00:00
Tasks: Intrusion Detection, Network Intrusion Detection, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Self-Learning

The rise of the new generation of cyber threats demands more sophisticated and intelligent cyber defense solutions equipped with autonomous agents capable of learning to make decisions without the knowledge of human experts. Several reinforcement learning methods (e.g., Markov) for automated network intrusion tasks have been proposed in recent years. In this paper, we introduce a new generation of network intrusion detection methods that combines a Q-learning-based reinforcement learning with a deep-feed forward neural network method for network intrusion detection. Our proposed Deep Q-Learning (DQL) model provides an ongoing auto-learning capability for a network environment that can detect different types of network intrusions using an automated trial-error approach and continuously enhance its detection capabilities. We provide the details of fine-tuning different hyperparameters involved in the DQL model for more effective self-learning. According to our extensive experimental results based on the NSL-KDD dataset, we confirm that the lower discount factor which is set as 0.001 under 250 episodes of training yields the best performance results. Our experimental results also show that our proposed DQL is highly effective in detecting different intrusion classes and outperforms other similar machine learning approaches.
Label Assistant: A Workflow for Assisted Data Annotation in Image Segmentation Tasks
Authors: Marcel P. Schilling, Luca Rettenberger, Friedrich Münke, Haijun Cui, Anna A. Popova, Pavel A. Levkin, Ralf Mikut, Markus Reischl
Date: 2021-11-27 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

Recent research in the field of computer vision strongly focuses on deep learning architectures to tackle image processing problems. Deep neural networks are often considered in complex image processing scenarios since traditional computer vision approaches are expensive to develop or reach their limits due to complex relations. However, a common criticism is the need for large annotated datasets to determine robust parameters. Annotating images by human experts is time-consuming, burdensome, and expensive. Thus, support is needed to simplify annotation, increase user efficiency, and annotation quality. In this paper, we propose a generic workflow to assist the annotation process and discuss methods on an abstract level. Thereby, we review the possibilities of focusing on promising samples, image pre-processing, pre-labeling, label inspection, or post-processing of annotations. In addition, we present an implementation of the proposal by means of a developed flexible and extendable software prototype nested in hybrid touchscreen/laptop device.
Modeling VI and VDRL feedback functions: searching normative rules through computational simulation
Authors: Paulo Sergio Panse Silveira, Jose de Oliveira Siqueira, Joao Lucas Bernardy, Jessica Santiago, Thiago Cersosimo Meneses, Bianca Sanches Portela, Marcelo Frota Benvenuti
Date: 2021-11-27 00:00:00

In this paper, we present a R script named Beak, built to simulate rates of behavior interacting with schedules of reinforcement. Using Beak, we've simulated data that allows an assessment of different reinforcement feedback functions (RFF). This was made with unparalleled precision, since simulations provide huge samples of data and, more importantly, simulated behavior isn't changed by the reinforcement it produces. Therefore, we can vary it systematically. We've compared different RFF for RI schedules, using as criteria: meaning, precision, parsimony and generality. Our results indicate that the best feedback function for the RI schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the RDRL schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of schedules. Also, they could guide future experiments that use schedules as theoretical and methodological tools.
Normative Disagreement as a Challenge for Cooperative AI
Authors: Julian Stastny, Maxime Riché, Alexander Lyzhov, Johannes Treutlein, Allan Dafoe, Jesse Clifton
Date: 2021-11-27 00:00:00

Cooperation in settings where agents have both common and conflicting interests (mixed-motive environments) has recently received considerable attention in multi-agent learning. However, the mixed-motive environments typically studied have a single cooperative outcome on which all agents can agree. Many real-world multi-agent environments are instead bargaining problems (BPs): they have several Pareto-optimal payoff profiles over which agents have conflicting preferences. We argue that typical cooperation-inducing learning algorithms fail to cooperate in BPs when there is room for normative disagreement resulting in the existence of multiple competing cooperative equilibria, and illustrate this problem empirically. To remedy the issue, we introduce the notion of norm-adaptive policies. Norm-adaptive policies are capable of behaving according to different norms in different circumstances, creating opportunities for resolving normative disagreement. We develop a class of norm-adaptive policies and show in experiments that these significantly increase cooperation. However, norm-adaptiveness cannot address residual bargaining failure arising from a fundamental tradeoff between exploitability and cooperative robustness.