pwc-archive/papers-with-abstracts huggingface dataset:
Solving and Learning Nonlinear PDEs with Gaussian Processes
Authors: Yifan Chen, Bamdad Hosseini, Houman Owhadi, Andrew M Stuart
Date: 2021-03-24 00:00:00
Tasks: Gaussian Processes

We introduce a simple, rigorous, and unified framework for solving nonlinear partial differential equations (PDEs), and for solving inverse problems (IPs) involving the identification of parameters in PDEs, using the framework of Gaussian processes. The proposed approach: (1) provides a natural generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has guaranteed convergence for a very general class of PDEs, and comes equipped with a path to compute error bounds for specific PDE approximations; (3) inherits the state-of-the-art computational complexity of linear solvers for dense kernel matrices. The main idea of our method is to approximate the solution of a given PDE as the maximum a posteriori (MAP) estimator of a Gaussian process conditioned on solving the PDE at a finite number of collocation points. Although this optimization problem is infinite-dimensional, it can be reduced to a finite-dimensional one by introducing additional variables corresponding to the values of the derivatives of the solution at collocation points; this generalizes the representer theorem arising in Gaussian process regression. The reduced optimization problem has the form of a quadratic objective function subject to nonlinear constraints; it is solved with a variant of the Gauss--Newton method. The resulting algorithm (a) can be interpreted as solving successive linearizations of the nonlinear PDE, and (b) in practice is found to converge in a small number of iterations (2 to 10), for a wide range of PDEs. Most traditional approaches to IPs interleave parameter updates with numerical solution of the PDE; our algorithm solves for both parameter and PDE solution simultaneously. Experiments on nonlinear elliptic PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for permeability identification in Darcy flow illustrate the efficacy and scope of our framework.
On Preference Learning Based on Sequential Bayesian Optimization with Pairwise Comparison
Authors: Tanya Ignatenko, Kirill Kondrashov, Marco Cox, Bert de Vries
Date: 2021-03-24 00:00:00
Tasks: Bayesian Optimization

User preference learning is generally a hard problem. Individual preferences are typically unknown even to users themselves, while the space of choices is infinite. Here we study user preference learning from information-theoretic perspective. We model preference learning as a system with two interacting sub-systems, one representing a user with his/her preferences and another one representing an agent that has to learn these preferences. The user with his/her behaviour is modeled by a parametric preference function. To efficiently learn the preferences and reduce search space quickly, we propose the agent that interacts with the user to collect the most informative data for learning. The agent presents two proposals to the user for evaluation, and the user rates them based on his/her preference function. We show that the optimum agent strategy for data collection and preference learning is a result of maximin optimization of the normalized weighted Kullback-Leibler (KL) divergence between true and agent-assigned predictive user response distributions. The resulting value of KL-divergence, which we also call remaining system uncertainty (RSU), provides an efficient performance metric in the absence of the ground truth. This metric characterises how well the agent can predict user and, thus, the quality of the underlying learned user (preference) model. Our proposed agent comprises sequential mechanisms for user model inference and proposal generation. To infer the user model (preference function), Bayesian approximate inference is used in the agent. The data collection strategy is to generate proposals, responses to which help resolving uncertainty associated with prediction of the user responses the most. The efficiency of our approach is validated by numerical simulations. Also a real-life example of preference learning application is provided.
Closed-form geodesics and trust-region method to calculate Riemannian logarithms on Stiefel and its quotient manifolds
Authors: Du Nguyen
Date: 2021-03-12 00:00:00
Tasks: Form

We provide two closed-form geodesic formulas for a family of metrics on Stiefel manifold, parameterized by two positive numbers, having both the embedded and canonical metrics as special cases. The closed-form formulas allow us to compute geodesics by matrix exponential in reduced dimension for low-rank manifolds. Combining with the use of Fr{\'e}chet derivatives to compute the gradient of the square Frobenius distance between a geodesic ending point to a given point on the manifold, we show the logarithm map and geodesic distance between two endpoints on the manifold could be computed by {\it minimizing} this square distance by a {\it trust-region} solver. This leads to a new framework to compute the geodesic distance for manifolds with known geodesic formula but no closed-form logarithm map. We show the approach works well for Stiefel as well as flag manifolds. The logarithm map could be used to compute the Riemannian center of mass for these manifolds equipped with the above metrics. We also deduce simple trigonometric formulas for the Riemannian exponential and logarithm maps on the Grassmann manifold.
A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors
Authors: Zhiyuan Li
Date: 2021-03-24 00:00:00
Tasks: Clustering, Variable Selection, Vocal Bursts Intensity Prediction, Vocal Bursts Valence Prediction

When fitting statistical models, some predictors are often found to be correlated with each other, and functioning together. Many group variable selection methods are developed to select the groups of predictors that are closely related to the continuous or categorical response. These existing methods usually assume the group structures are well known. For example, variables with similar practical meaning, or dummy variables created by categorical data. However, in practice, it is impractical to know the exact group structure, especially when the variable dimensional is large. As a result, the group variable selection results may be selected. To solve the challenge, we propose a two-stage approach that combines a variable clustering stage and a group variable stage for the group variable selection problem. The variable clustering stage uses information from the data to find a group structure, which improves the performance of the existing group variable selection methods. For ultrahigh dimensional data, where the predictors are much larger than observations, we incorporated a variable screening method in the first stage and shows the advantages of such an approach. In this article, we compared and discussed the performance of four existing group variable selection methods under different simulation models, with and without the variable clustering stage. The two-stage method shows a better performance, in terms of the prediction accuracy, as well as in the accuracy to select active predictors. An athlete's data is also used to show the advantages of the proposed method.
Anticipating synchronization with machine learning
Authors: Huawei Fan, Ling-Wei Kong, Ying-Cheng Lai, Xingang Wang
Date: 2021-03-13 00:00:00
Tasks: BIG-bench Machine Learning, Time Series Analysis

In applications of dynamical systems, situations can arise where it is desired to predict the onset of synchronization as it can lead to characteristic and significant changes in the system performance and behaviors, for better or worse. In experimental and real settings, the system equations are often unknown, raising the need to develop a prediction framework that is model free and fully data driven. We contemplate that this challenging problem can be addressed with machine learning. In particular, exploiting reservoir computing or echo state networks, we devise a "parameter-aware" scheme to train the neural machine using asynchronous time series, i.e., in the parameter regime prior to the onset of synchronization. A properly trained machine will possess the power to predict the synchronization transition in that, with a given amount of parameter drift, whether the system would remain asynchronous or exhibit synchronous dynamics can be accurately anticipated. We demonstrate the machine-learning based framework using representative chaotic models and small network systems that exhibit continuous (second-order) or abrupt (first-order) transitions. A remarkable feature is that, for a network system exhibiting an explosive (first-order) transition and a hysteresis loop in synchronization, the machine learning scheme is capable of accurately predicting these features, including the precise locations of the transition points associated with the forward and backward transition paths.
From Semantic Retrieval to Pairwise Ranking: Applying Deep Learning in E-commerce Search
Authors: Rui Li, Yunjiang Jiang, WenYun Yang, Guoyu Tang, Songlin Wang, Chaoyi Ma, wei he, Xi Xiong, Yun Xiao, Eric Yihong Zhao
Date: 2021-03-24 00:00:00
Tasks: Deep Learning, Re-Ranking, Retrieval, Semantic Retrieval

We introduce deep learning models to the two most important stages in product search at JD.com, one of the largest e-commerce platforms in the world. Specifically, we outline the design of a deep learning system that retrieves semantically relevant items to a query within milliseconds, and a pairwise deep re-ranking system, which learns subtle user preferences. Compared to traditional search systems, the proposed approaches are better at semantic retrieval and personalized ranking, achieving significant improvements.
Web Mining for Estimating Regulatory Blockchain Readiness
Authors: Elias Iosif, Klitos Christodoulou, Andreas Vlachos
Date: 2021-03-24 00:00:00
Tasks: Clustering

The regulatory framework of cryptocurrencies (and, in general, blockchain tokens) is of paramount importance. This framework drives nearly all key decisions in the respective business areas. In this work, a computational model is proposed for quantitatively estimating the regulatory stance of countries with respect to cryptocurrencies. This is conducted via web mining utilizing web search engines. The proposed model is experimentally validated. In addition, unsupervised learning (clustering) is applied for better analyzing the automatically derived estimations. Overall, very good performance is achieved by the proposed algorithmic approach.
Using Meta-learning to Recommend Process Discovery Methods
Authors: Sylvio Barbon Jr, Paolo Ceravolo, Ernesto Damiani, Gabriel Marques Tavares
Date: 2021-03-23 00:00:00
Tasks: Management, Meta-Learning

Process discovery methods have obtained remarkable achievements in Process Mining, delivering comprehensible process models to enhance management capabilities. However, selecting the suitable method for a specific event log highly relies on human expertise, hindering its broad application. Solutions based on Meta-learning (MtL) have been promising for creating systems with reduced human assistance. This paper presents a MtL solution for recommending process discovery methods that maximize model quality according to complementary dimensions. Thanks to our MtL pipeline, it was possible to recommend a discovery method with 92% of accuracy using light-weight features that describe the event log. Our experimental analysis also provided significant insights on the importance of log features in generating recommendations, paving the way to a deeper understanding of the discovery algorithms.
Actionable Cognitive Twins for Decision Making in Manufacturing
Authors: Jože M. Rožanec, Jinzhi Lu, Jan Rupnik, Maja Škrjanc, Dunja Mladenić, Blaž Fortuna, Xiaochen Zheng, Dimitris Kiritsis
Date: 2021-03-23 00:00:00
Tasks: Decision Making, Demand Forecasting

Actionable Cognitive Twins are the next generation Digital Twins enhanced with cognitive capabilities through a knowledge graph and artificial intelligence models that provide insights and decision-making options to the users. The knowledge graph describes the domain-specific knowledge regarding entities and interrelationships related to a manufacturing setting. It also contains information on possible decision-making options that can assist decision-makers, such as planners or logisticians. In this paper, we propose a knowledge graph modeling approach to construct actionable cognitive twins for capturing specific knowledge related to demand forecasting and production planning in a manufacturing plant. The knowledge graph provides semantic descriptions and contextualization of the production lines and processes, including data identification and simulation or artificial intelligence algorithms and forecasts used to support them. Such semantics provide ground for inferencing, relating different knowledge types: creative, deductive, definitional, and inductive. To develop the knowledge graph models for describing the use case completely, systems thinking approach is proposed to design and verify the ontology, develop a knowledge graph and build an actionable cognitive twin. Finally, we evaluate our approach in two use cases developed for a European original equipment manufacturer related to the automotive industry as part of the European Horizon 2020 project FACTLOG.
Goal Seeking Quadratic Unconstrained Binary Optimization
Authors: Amit Verma, Mark Lewis
Date: 2021-03-24 00:00:00
Tasks: Decision Making

The Quadratic Unconstrained Binary Optimization (QUBO) modeling and solution framework is a requirement for quantum and digital annealers. However optimality for QUBO problems of any practical size is extremely difficult to achieve. In order to incorporate the problem-specific insights, a diverse set of solutions meeting an acceptable target metric or goal is the preference in high level decision making. In this paper, we present two alternatives for goal-seeking QUBO for minimizing the deviation from a given target as well as a range of values around a target. Experimental results illustrate the efficacy of the proposed approach over Constraint Programming for quickly finding a satisficing set of solutions.
Counterfactual Explanation with Multi-Agent Reinforcement Learning for Drug Target Prediction
Authors: Tri Minh Nguyen, Thomas P Quinn, Thin Nguyen, Truyen Tran
Date: 2021-03-24 00:00:00
Tasks: counterfactual, Counterfactual Explanation, Explainable Artificial Intelligence (XAI), Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning (RL)

Motivation: Many high-performance DTA models have been proposed, but they are mostly black-box and thus lack human interpretability. Explainable AI (XAI) can make DTA models more trustworthy, and can also enable scientists to distill biological knowledge from the models. Counterfactual explanation is one popular approach to explaining the behaviour of a deep neural network, which works by systematically answering the question "How would the model output change if the inputs were changed in this way?". Most counterfactual explanation methods only operate on single input data. It remains an open problem how to extend counterfactual-based XAI methods to DTA models, which have two inputs, one for drug and one for target, that also happen to be discrete in nature. Methods: We propose a multi-agent reinforcement learning framework, Multi-Agent Counterfactual Drug target binding Affinity (MACDA), to generate counterfactual explanations for the drug-protein complex. Our proposed framework provides human-interpretable counterfactual instances while optimizing both the input drug and target for counterfactual generation at the same time. Results: We benchmark the proposed MACDA framework using the Davis dataset and find that our framework produces more parsimonious explanations with no loss in explanation validity, as measured by encoding similarity and QED. We then present a case study involving ABL1 and Nilotinib to demonstrate how MACDA can explain the behaviour of a DTA model in the underlying substructure interaction between inputs in its prediction, revealing mechanisms that align with prior domain knowledge.
Truth and Subjunctive Theories of Knowledge: No Luck?
Authors: Johannes Stern
Date: 2021-03-24 00:00:00

The paper explores applications of Kripke's theory of truth to semantics for anti-luck epistemology, that is, to subjunctive theories of knowledge. Subjunctive theories put forward modal or subjunctive conditions to rule out knowledge by mere luck as to be found in Gettier-style counterexamples to the analysis of knowledge as justified true belief. Because of the subjunctive nature of these conditions the resulting semantics turns out to be non-monotone, even if it is based on non-classical evaluation schemes such as strong Kleene or FDE. This blocks the usual road to fixed-point results for Kripke's theory of truth within these semantics and consequently the paper is predominantly an exploration of fixed point results for Kripke's theory of truth within non-monotone semantics. Using the theory of quasi-inductive definitions we show that in case of the subjunctive theories of knowledge the so-called Kripke jump will have fixed points despite the non-monotonicity of the semantics: Kripke's theory of truth can be successfully applied in the framework of subjunctive theories of knowledge.
Facility Reallocation on the Line
Authors: Bart de Keijzer, Dominik Wojtczak
Date: 2021-03-23 00:00:00

We consider a multi-stage facility reallocation problems on the real line, where a facility is being moved between time stages based on the locations reported by $n$ agents. The aim of the reallocation algorithm is to minimise the social cost, i.e., the sum over the total distance between the facility and all agents at all stages, plus the cost incurred for moving the facility. We study this problem both in the offline setting and online setting. In the offline case the algorithm has full knowledge of the agent locations in all future stages, and in the online setting the algorithm does not know these future locations and must decide the location of the facility on a stage-per-stage basis. We derive the optimal algorithm in both cases. For the online setting we show that its competitive ratio is $(n+2)/(n+1)$. As neither of these algorithms turns out to yield a strategy-proof mechanism, we propose another strategy-proof mechanism which has a competitive ratio of $(n+3)/(n+1)$ for odd $n$ and $(n+4)/n$ for even $n$, which we conjecture to be the best possible. We also consider a generalisation with multiple facilities and weighted agents, for which we show that the optimum can be computed in polynomial time for a fixed number of facilities.
Meta-Learned Invariant Risk Minimization
Authors: Jun-Hyun Bae, Inchul Choi, Minho Lee
Date: 2021-03-24 00:00:00
Tasks: Meta-Learning

Empirical Risk Minimization (ERM) based machine learning algorithms have suffered from weak generalization performance on data obtained from out-of-distribution (OOD). To address this problem, Invariant Risk Minimization (IRM) objective was suggested to find invariant optimal predictor which is less affected by the changes in data distribution. However, even with such progress, IRMv1, the practical formulation of IRM, still shows performance degradation when there are not enough training data, and even fails to generalize to OOD, if the number of spurious correlations is larger than the number of environments. In this paper, to address such problems, we propose a novel meta-learning based approach for IRM. In this method, we do not assume the linearity of classifier for the ease of optimization, and solve ideal bi-level IRM objective with Model-Agnostic Meta-Learning (MAML) framework. Our method is more robust to the data with spurious correlations and can provide an invariant optimal classifier even when data from each distribution are scarce. In experiments, we demonstrate that our algorithm not only has better OOD generalization performance than IRMv1 and all IRM variants, but also addresses the weakness of IRMv1 with improved stability.
Including Sparse Production Knowledge into Variational Autoencoders to Increase Anomaly Detection Reliability
Authors: Tom Hammerbacher, Markus Lange-Hegermann, Gorden Platz
Date: 2021-03-24 00:00:00
Tasks: Anomaly Detection, Time Series, Time Series Analysis

Digitalization leads to data transparency for production systems that we can benefit from with data-driven analysis methods like neural networks. For example, automated anomaly detection enables saving resources and optimizing the production. We study using rarely occurring information about labeled anomalies into Variational Autoencoder neural network structures to overcome information deficits of supervised and unsupervised approaches. This method outperforms all other models in terms of accuracy, precision, and recall. We evaluate the following methods: Principal Component Analysis, Isolation Forest, Classifying Neural Networks, and Variational Autoencoders on seven time series datasets to find the best performing detection methods. We extend this idea to include more infrequently occurring meta information about production processes. This use of sparse labels, both of anomalies or production data, allows to harness any additional information available for increasing anomaly detection performance.
Discriminator Augmented Model-Based Reinforcement Learning
Authors: Behzad Haghgoo, Allan Zhou, Archit Sharma, Chelsea Finn
Date: 2021-03-24 00:00:00
Tasks: model, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

By planning through a learned dynamics model, model-based reinforcement learning (MBRL) offers the prospect of good performance with little environment interaction. However, it is common in practice for the learned model to be inaccurate, impairing planning and leading to poor performance. This paper aims to improve planning with an importance sampling framework that accounts and corrects for discrepancy between the true and learned dynamics. This framework also motivates an alternative objective for fitting the dynamics model: to minimize the variance of value estimation during planning. We derive and implement this objective, which encourages better prediction on trajectories with larger returns. We observe empirically that our approach improves the performance of current MBRL algorithms on two stochastic control problems, and provide a theoretical basis for our method.