pwc-archive/papers-with-abstracts huggingface dataset:
Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints
Authors: Zichao Wang, Andrew S. Lan, Richard G. Baraniuk
Date: 2021-09-09 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Diversity, Math, Question Generation

We study the problem of generating arithmetic math word problems (MWPs) given a math equation that specifies the mathematical computation and a context that specifies the problem scenario. Existing approaches are prone to generating MWPs that are either mathematically invalid or have unsatisfactory language quality. They also either ignore the context or require manual specification of a problem template, which compromises the diversity of the generated MWPs. In this paper, we develop a novel MWP generation approach that leverages i) pre-trained language models and a context keyword selection model to improve the language quality of the generated MWPs and ii) an equation consistency constraint for math equations to improve the mathematical validity of the generated MWPs. Extensive quantitative and qualitative experiments on three real-world MWP datasets demonstrate the superior performance of our approach compared to various baselines.
Generic resources are what you need: Style transfer tasks without task-specific parallel training data
Authors: Huiyuan Lai, Antonio Toral, Malvina Nissim
Date: 2021-09-09 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Style Transfer, Translation

Style transfer aims to rewrite a source text in a different target style while preserving its content. We propose a novel approach to this task that leverages generic resources, and without using any task-specific parallel (source-target) data outperforms existing unsupervised approaches on the two most popular style transfer tasks: formality transfer and polarity swap. In practice, we adopt a multi-step procedure which builds on a generic pre-trained sequence-to-sequence model (BART). First, we strengthen the model's ability to rewrite by further pre-training BART on both an existing collection of generic paraphrases, as well as on synthetic pairs created using a general-purpose lexical resource. Second, through an iterative back-translation approach, we train two models, each in a transfer direction, so that they can provide each other with synthetically generated pairs, dynamically in the training process. Lastly, we let our best reresulting model generate static synthetic pairs to be used in a supervised training regime. Besides methodology and state-of-the-art results, a core contribution of this work is a reflection on the nature of the two tasks we address, and how their differences are highlighted by their response to our approach.
Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach
Authors: Koren Lazar, Benny Saret, Asaf Yehudai, Wayne Horowitz, Nathan Wasserman, Gabriel Stanovsky
Date: 2021-09-09 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Language Modelling

We present models which complete missing text given transliterations of ancient Mesopotamian documents, originally written on cuneiform clay tablets (2500 BCE - 100 CE). Due to the tablets' deterioration, scholars often rely on contextual cues to manually fill in missing parts in the text in a subjective and time-consuming process. We identify that this challenge can be formulated as a masked language modelling task, used mostly as a pretraining objective for contextualized language models. Following, we develop several architectures focusing on the Akkadian language, the lingua franca of the time. We find that despite data scarcity (1M tokens) we can achieve state of the art performance on missing tokens prediction (89% hit@5) using a greedy decoding scheme and pretraining on data from other languages and different time periods. Finally, we conduct human evaluations showing the applicability of our models in assisting experts to transcribe texts in extinct languages.
Semantic Parsing in Task-Oriented Dialog with Recursive Insertion-based Encoder
Authors: Elman Mansimov, Yi Zhang
Date: 2021-09-09 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Nested Named Entity Recognition, Semantic Parsing

We introduce a Recursive INsertion-based Encoder (RINE), a novel approach for semantic parsing in task-oriented dialog. Our model consists of an encoder network that incrementally builds the semantic parse tree by predicting the non-terminal label and its positions in the linearized tree. At the generation time, the model constructs the semantic parse tree by recursively inserting the predicted non-terminal labels at the predicted positions until termination. RINE achieves state-of-the-art exact match accuracy on low- and high-resource versions of the conversational semantic parsing benchmark TOP (Gupta et al., 2018; Chen et al., 2020), outperforming strong sequence-to-sequence models and transition-based parsers. We also show that our model design is applicable to nested named entity recognition task, where it performs on par with state-of-the-art approach designed for that task. Finally, we demonstrate that our approach is 2-3.5 times faster than the sequence-to-sequence model at inference time.
Characterization of Constrained Continuous Multiobjective Optimization Problems: A Feature Space Perspective
Authors: Aljoša Vodopija, Tea Tušar, Bogdan Filipič
Date: 2021-09-09 00:00:00
Tasks: Benchmarking, Multiobjective Optimization

Despite the increasing interest in constrained multiobjective optimization in recent years, constrained multiobjective optimization problems (CMOPs) are still unsatisfactory understood and characterized. For this reason, the selection of appropriate CMOPs for benchmarking is difficult and lacks a formal background. We address this issue by extending landscape analysis to constrained multiobjective optimization. By employing four exploratory landscape analysis techniques, we propose 29 landscape features (of which 19 are novel) to characterize CMOPs. These landscape features are then used to compare eight frequently used artificial test suites against a recently proposed suite consisting of real-world problems based on physical models. The experimental results reveal that the artificial test problems fail to adequately represent some realistic characteristics, such as strong negative correlation between the objectives and constraints. Moreover, our findings show that all the studied artificial test suites have advantages and limitations, and that no "perfect" suite exists. Benchmark designers can use the obtained results to select or generate appropriate CMOP instances based on the characteristics they want to explore.
Low-rank statistical finite elements for scalable model-data synthesis
Authors: Connor Duffin, Edward Cripps, Thomas Stemler, Mark Girolami
Date: 2021-09-10 00:00:00

Statistical learning additions to physically derived mathematical models are gaining traction in the literature. A recent approach has been to augment the underlying physics of the governing equations with data driven Bayesian statistical methodology. Coined statFEM, the method acknowledges a priori model misspecification, by embedding stochastic forcing within the governing equations. Upon receipt of additional data, the posterior distribution of the discretised finite element solution is updated using classical Bayesian filtering techniques. The resultant posterior jointly quantifies uncertainty associated with the ubiquitous problem of model misspecification and the data intended to represent the true process of interest. Despite this appeal, computational scalability is a challenge to statFEM's application to high-dimensional problems typically experienced in physical and industrial contexts. This article overcomes this hurdle by embedding a low-rank approximation of the underlying dense covariance matrix, obtained from the leading order modes of the full-rank alternative. Demonstrated on a series of reaction-diffusion problems of increasing dimension, using experimental and simulated data, the method reconstructs the sparsely observed data-generating processes with minimal loss of information, in both the posterior mean and variance, paving the way for further integration of physical and probabilistic approaches to complex systems.
Interaction Models and Generalized Score Matching for Compositional Data
Authors: Shiqing Yu, Mathias Drton, Ali Shojaie
Date: 2021-09-10 00:00:00

Applications such as the analysis of microbiome data have led to renewed interest in statistical methods for compositional data, i.e., multivariate data in the form of probability vectors that contain relative proportions. In particular, there is considerable interest in modeling interactions among such relative proportions. To this end we propose a class of exponential family models that accommodate general patterns of pairwise interaction while being supported on the probability simplex. Special cases include the family of Dirichlet distributions as well as Aitchison's additive logistic normal distributions. Generally, the distributions we consider have a density that features a difficult to compute normalizing constant. To circumvent this issue, we design effective estimation methods based on generalized versions of score matching. A high-dimensional analysis of our estimation methods shows that the simplex domain is handled as efficiently as previously studied full-dimensional domains.
You Get What You Chat: Using Conversations to Personalize Search-based Recommendations
Authors: Ghazaleh Haratinezhad Torbati, Andrew Yates, Gerhard Weikum
Date: 2021-09-10 00:00:00
Tasks: Re-Ranking

Prior work on personalized recommendations has focused on exploiting explicit signals from user-specific queries, clicks, likes, and ratings. This paper investigates tapping into a different source of implicit signals of interests and tastes: online chats between users. The paper develops an expressive model and effective methods for personalizing search-based entity recommendations. User models derived from chats augment different methods for re-ranking entity answers for medium-grained queries. The paper presents specific techniques to enhance the user models by capturing domain-specific vocabularies and by entity-based expansion. Experiments are based on a collection of online chats from a controlled user study covering three domains: books, travel, food. We evaluate different configurations and compare chat-based user models against concise user profiles from questionnaires. Overall, these two variants perform on par in terms of NCDG@20, but each has advantages in certain domains.
Personalized Entity Search by Sparse and Scrutable User Profiles
Authors: Ghazaleh Haratinezhad Torbati, Andrew Yates, Gerhard Weikum
Date: 2021-09-10 00:00:00
Tasks: Re-Ranking

Prior work on personalizing web search results has focused on considering query-and-click logs to capture users individual interests. For product search, extensive user histories about purchases and ratings have been exploited. However, for general entity search, such as for books on specific topics or travel destinations with certain features, personalization is largely underexplored. In this paper, we address personalization of book search, as an exemplary case of entity search, by exploiting sparse user profiles obtained through online questionnaires. We devise and compare a variety of re-ranking methods based on language models or neural learning. Our experiments show that even very sparse information about individuals can enhance the effectiveness of the search results.
1st-Order Dynamics on Nonlinear Agents for Resource Allocation over Uniformly-Connected Networks
Authors: Mohammadreza Doostmohammadian, Alireza Aghasi, Maria Vrakopoulou, Themistoklis Charalambous
Date: 2021-09-10 00:00:00
Tasks: Quantization

A general nonlinear $1$st-order consensus-based solution for distributed constrained convex optimization is proposed with network resource allocation applications. The solution is used to optimize continuously-differentiable strictly convex cost functions over weakly-connected undirected networks, while it is anytime feasible and models various nonlinearities to account for imperfections and constraints on the (physical model of) agents in terms of limited actuation capabilities, e.g., quantization and saturation. Due to such inherent nonlinearities, the existing linear solutions considering ideal agent models may not necessarily converge with guaranteed optimality and anytime feasibility. Some applications also impose specific nonlinearities, e.g., convergence in fixed/finite-time or sign-based robust disturbance-tolerant dynamics. Our proposed distributed protocol generalizes such nonlinear models. Putting convex set analysis together with nonsmooth Lyapunov analysis, we prove convergence, (i) regardless of the particular type of nonlinearity, and (ii) with weak network-connectivity requirements (uniform-connectivity).
GPS-Denied Navigation Using Low-Cost Inertial Sensors and Recurrent Neural Networks
Authors: Ahmed AbdulMajuid, Osama Mohamady, Mohannad Draz, Gamal El-bayoumi
Date: 2021-09-10 00:00:00
Tasks: Position

Autonomous missions of drones require continuous and reliable estimates for the drone's attitude, velocity, and position. Traditionally, these states are estimated by applying Extended Kalman Filter (EKF) to Accelerometer, Gyroscope, Barometer, Magnetometer, and GPS measurements. When the GPS signal is lost, position and velocity estimates deteriorate quickly, especially when using low-cost inertial sensors. This paper proposes an estimation method that uses a Recurrent Neural Network (RNN) to allow reliable estimation of a drone's position and velocity in the absence of GPS signal. The RNN is trained on a public dataset collected using Pixhawk. This low-cost commercial autopilot logs the raw sensor measurements (network inputs) and corresponding EKF estimates (ground truth outputs). The dataset is comprised of 548 different flight logs with flight durations ranging from 4 to 32 minutes. For training, 465 flights are used, totaling 45 hours. The remaining 83 flights totaling 8 hours are held out for validation. Error in a single flight is taken to be the maximum absolute difference in 3D position (MPE) between the RNN predictions (without GPS) and the ground truth (EKF with GPS). On the validation set, the median MPE is 35 meters. MPE values as low as 2.7 meters in a 5-minutes flight could be achieved using the proposed method. The MPE in 90% of the validation flights is bounded below 166 meters. The network was experimentally tested and worked in real-time.
Risk-perception-aware control design under dynamic spatial risks
Authors: Aamodh Suresh, Sonia Martinez
Date: 2021-09-09 00:00:00

This work proposes a novel risk-perception-aware (RPA) control design using non-rational perception of risks associated with uncertain dynamic spatial costs. We use Cumulative Prospect Theory (CPT) to model the risk perception of a decision maker (DM) and use it to construct perceived risk functions that transform the uncertain dynamic spatial cost to deterministic perceived risks of a DM. These risks are then used to build safety sets which can represent risk-averse to risk-insensitive perception. We define a notions of "inclusiveness" and "versatility" based on safety sets and use it to compare with other models such as Conditional value at Risk (CVaR) and Expected risk (ER). We theoretically prove that CPT is the most "inclusive" and "versatile" model of the lot in the context of risk-perception-aware controls. We further use the perceived risk function along with ideas from control barrier functions (CBF) to construct a class of perceived risk CBFs. For a class of truncated-Gaussian costs, we find sufficient geometric conditions for the validity of this class of CBFs, thus guaranteeing safety. Then, we generate perceived-safety-critical controls using a Quadratic program (QP) to guide an agent safely according to a given perceived risk model. We present simulations in a 2D environment to illustrate the performance of the proposed controller.
Adjoint Differentiation for generic matrix functions
Authors: Andrei Goloubentsev, Dmitri Goloubentsev, Evgeny Lakshtanov
Date: 2021-09-10 00:00:00
Tasks: Form, regression

We derive a formula for the adjoint $\overline{A}$ of a square-matrix operation of the form $C=f(A)$, where $f$ is holomorphic in the neighborhood of each eigenvalue. We then apply the formula to derive closed-form expressions in particular cases of interest such as the case when we have a spectral decomposition $A=UDU^{-1}$, the spectrum cut-off $C=A_+$ and the Nearest Correlation Matrix routine. Finally, we explain how to simplify the computation of adjoints for regularized linear regression coefficients.
Large-vocabulary Audio-visual Speech Recognition in Noisy Environments
Authors: Wentao Yu, Steffen Zeiler, Dorothea Kolossa
Date: 2021-09-10 00:00:00
Tasks: Audio-Visual Speech Recognition, Lipreading, speech-recognition, Speech Recognition, Video Recognition, Visual Speech Recognition

Audio-visual speech recognition (AVSR) can effectively and significantly improve the recognition rates of small-vocabulary systems, compared to their audio-only counterparts. For large-vocabulary systems, however, there are still many difficulties, such as unsatisfactory video recognition accuracies, that make it hard to improve over audio-only baselines. In this paper, we specifically consider such scenarios, focusing on the large-vocabulary task of the LRS2 database, where audio-only performance is far superior to video-only accuracies, making this an interesting and challenging setup for multi-modal integration. To address the inherent difficulties, we propose a new fusion strategy: a recurrent integration network is trained to fuse the state posteriors of multiple single-modality models, guided by a set of model-based and signal-based stream reliability measures. During decoding, this network is used for stream integration within a hybrid recognizer, where it can thus cope with the time-variant reliability and information content of its multiple feature inputs. We compare the results with end-to-end AVSR systems as well as with competitive hybrid baseline models, finding that the new fusion strategy shows superior results, on average even outperforming oracle dynamic stream weighting, which has so far marked the -- realistically unachievable -- upper bound for standard stream weighting. Even though the pure lipreading performance is low, audio-visual integration is helpful under all -- clean, noisy, and reverberant -- conditions. On average, the new system achieves a relative word error rate reduction of 42.18\% compared to the audio-only model, pointing at a high effectiveness of the proposed integration approach.
Directional MCLP Analysis and Reconstruction for Spatial Speech Communication
Authors: Srikanth Raj Chetupalli, Thippur V. Sreenivas
Date: 2021-09-09 00:00:00
Tasks: Position

Spatial speech communication, i.e., the reconstruction of spoken signal along with the relative speaker position in the enclosure (reverberation information) is considered in this paper. Directional, diffuse components and the source position information are estimated at the transmitter, and perceptually effective reproduction is considered at the receiver. We consider spatially distributed microphone arrays for signal acquisition, and node specific signal estimation, along with its direction of arrival (DoA) estimation. Short-time Fourier transform (STFT) domain multi-channel linear prediction (MCLP) approach is used to model the diffuse component and relative acoustic transfer function is used to model the direct signal component. Distortion-less array response constraint and the time-varying complex Gaussian source model are used in the joint estimation of source DoA and the constituent signal components, separately at each node. The intersection between DoA directions at each node is used to compute the source position. Signal components computed at the node nearest to the estimated source position are taken as the signals for transmission. At the receiver, a four channel loud speaker (LS) setup is used for spatial reproduction, in which the source spatial image is reproduced relative to a chosen virtual listener position in the transmitter enclosure. Vector base amplitude panning (VBAP) method is used for direct component reproduction using the LS setup and the diffuse component is reproduced equally from all the loud speakers after decorrelation. This scheme of spatial speech communication is shown to be effective and more natural for hands-free telecommunication, through either loudspeaker listening or binaural headphone listening with head related transfer function (HRTF) based presentation.
Uncertainty-Aware Capacity Allocation in Flow-Based Market Coupling
Authors: Richard Weinhold, Robert Mieth
Date: 2021-09-10 00:00:00
Tasks: Management

The effective allocation of cross-border trading capacities is one of the central challenges for the implementation of a pan-European internal energy market. In contrast to traditional power flow-ignorant methods like net transfer capacities (NTC), flow-based market coupling (FBMC) has been shown to increase price convergence between market areas, while improving congestion management effectiveness. However, explicitly analysing FBMC for a future power system with a very high share of intermittent renewable generation is often overlooked in the current literature. This paper provides a comprehensive summary on the technical specification of the FBMC process and FBMC modeling methods. It discusses implications of policy considerations and explicitly discusses the impact of high-shares of intermittent generation on FBMC performance. Further, we propose probabilistic security margins compatible with the current FBMC implementation to better account for renewable uncertainty in FBMC modeling. We conduct numerical experiments on the IEEE 118 bus test system to showcase the proposed model formulations and our data and implementation is published open source.
Development and Validation of a Scalable Fast Load Shedding Technique for Industrial Power Systems
Authors: Andrea Petriccioli, Samuele Grillo, David Comunello, Andrea Cacace
Date: 2021-09-10 00:00:00

The work aims to improve the existing fast load shedding algorithm for industrial power system to increase performance, reliability, and scalability for future expansions. The paper illustrates the development of a scalable algorithm to compute the shedding matrix, and the test performed on a model of the electric grid of an offshore platform. From this model it is possible to study the impact on the transients of various parameters, such as spinning reserve and delay time. Subsequently, the code is converted into Structured Text and implemented on an ABB PLC. The scalability of the load shedding algorithm is thus verified, confirming its performance with respect to the computation of the shedding matrix and the usefulness of the dynamic simulations during the design phase of the plant.
Model Predictive Control with Infeasible Reference Trajectories
Authors: Ivo Batkovic, Mohammad Ali, Paolo Falcone, Mario Zanon
Date: 2021-09-10 00:00:00
Tasks: model, Model Predictive Control

Model Predictive Control (MPC) formulations are typically built on the requirement that a feasible reference trajectory is available. In practical settings, however, references that are infeasible with respect to the system dynamics are used for convenience. In this paper, we prove under which conditions an MPC formulation is Input-to-State Stable~(ISS) in closed-loop when an infeasible reference is used, and that with proper terminal conditions, asymptotic stability towards an optimal reference may be achieved. We illustrate the theoretical results with a four-dimensional robotic joint example.