pwc-archive/papers-with-abstracts huggingface dataset:
ThingTalk: An Extensible, Executable Representation Language for Task-Oriented Dialogues
Authors: Monica S. Lam, Giovanni Campagna, Mehrad Moradshahi, Sina J. Semnani, Silei Xu
Date: 2022-03-23 00:00:00
Tasks: Semantic Parsing

Task-oriented conversational agents rely on semantic parsers to translate natural language to formal representations. In this paper, we propose the design and rationale of the ThingTalk formal representation, and how the design improves the development of transactional task-oriented agents. ThingTalk is built on four core principles: (1) representing user requests directly as executable statements, covering all the functionality of the agent, (2) representing dialogues formally and succinctly to support accurate contextual semantic parsing, (3) standardizing types and interfaces to maximize reuse between agents, and (4) allowing multiple, independently-developed agents to be composed in a single virtual assistant. ThingTalk is developed as part of the Genie Framework that allows developers to quickly build transactional agents given a database and APIs. We compare ThingTalk to existing representations: SMCalFlow, SGD, TreeDST. Compared to the others, the ThingTalk design is both more general and more cost-effective. Evaluated on the MultiWOZ benchmark, using ThingTalk and associated tools yields a new state of the art accuracy of 79% turn-by-turn.
Adversarial Training for Improving Model Robustness? Look at Both Prediction and Interpretation
Authors: Hanjie Chen, Yangfeng Ji
Date: 2022-03-23 00:00:00
Tasks: text-classification, Text Classification

Neural language models show vulnerability to adversarial examples which are semantically similar to their original counterparts with a few words replaced by their synonyms. A common way to improve model robustness is adversarial training which follows two steps-collecting adversarial examples by attacking a target model, and fine-tuning the model on the augmented dataset with these adversarial examples. The objective of traditional adversarial training is to make a model produce the same correct predictions on an original/adversarial example pair. However, the consistency between model decision-makings on two similar texts is ignored. We argue that a robust model should behave consistently on original/adversarial example pairs, that is making the same predictions (what) based on the same reasons (how) which can be reflected by consistent interpretations. In this work, we propose a novel feature-level adversarial training method named FLAT. FLAT aims at improving model robustness in terms of both predictions and interpretations. FLAT incorporates variational word masks in neural networks to learn global word importance and play as a bottleneck teaching the model to make predictions based on important words. FLAT explicitly shoots at the vulnerability problem caused by the mismatch between model understandings on the replaced words and their synonyms in original/adversarial example pairs by regularizing the corresponding global word importance scores. Experiments show the effectiveness of FLAT in improving the robustness with respect to both predictions and interpretations of four neural network models (LSTM, CNN, BERT, and DeBERTa) to two adversarial attacks on four text classification tasks. The models trained via FLAT also show better robustness than baseline models on unforeseen adversarial examples across different attacks.
Spiking Neural Streaming Binary Arithmetic
Authors: James B. Aimone, Aaron J. Hill, William M. Severa, Craig M. Vineyard
Date: 2022-03-23 00:00:00

Boolean functions and binary arithmetic operations are central to standard computing paradigms. Accordingly, many advances in computing have focused upon how to make these operations more efficient as well as exploring what they can compute. To best leverage the advantages of novel computing paradigms it is important to consider what unique computing approaches they offer. However, for any special-purpose co-processor, Boolean functions and binary arithmetic operations are useful for, among other things, avoiding unnecessary I/O on-and-off the co-processor by pre- and post-processing data on-device. This is especially true for spiking neuromorphic architectures where these basic operations are not fundamental low-level operations. Instead, these functions require specific implementation. Here we discuss the implications of an advantageous streaming binary encoding method as well as a handful of circuits designed to exactly compute elementary Boolean and binary operations.
Multi Expression Programming for solving classification problems
Authors: Mihai Oltean
Date: 2022-03-16 00:00:00
Tasks: Classification, Multi-class Classification

Multi Expression Programming (MEP) is a Genetic Programming variant which encodes multiple solutions in a single chromosome. This paper introduces and deeply describes several strategies for solving binary and multi-class classification problems within the \textit{multi solutions per chromosome} paradigm of MEP. Extensive experiments on various classification problems are performed. MEP shows similar or better performances than other methods used for comparison (namely Artificial Neural Networks and Linear Genetic Programming).
MMES: Mixture Model based Evolution Strategy for Large-Scale Optimization
Authors: Xiaoyu He, Zibin Zheng, Yuren Zhou
Date: 2022-03-15 00:00:00
Tasks: global-optimization

This work provides an efficient sampling method for the covariance matrix adaptation evolution strategy (CMA-ES) in large-scale settings. In contract to the Gaussian sampling in CMA-ES, the proposed method generates mutation vectors from a mixture model, which facilitates exploiting the rich variable correlations of the problem landscape within a limited time budget. We analyze the probability distribution of this mixture model and show that it approximates the Gaussian distribution of CMA-ES with a controllable accuracy. We use this sampling method, coupled with a novel method for mutation strength adaptation, to formulate the mixture model based evolution strategy (MMES) -- a CMA-ES variant for large-scale optimization. The numerical simulations show that, while significantly reducing the time complexity of CMA-ES, MMES preserves the rotational invariance, is scalable to high dimensional problems, and is competitive against the state-of-the-arts in performing global optimization.
Novel Metric based on Walsh Coefficients for measuring problem difficulty in Estimation of Distribution Algorithms
Authors: Saeed Ghadiri, Amin Nikanjam
Date: 2022-02-24 00:00:00
Tasks: Evolutionary Algorithms

Estimation of distribution algorithms are evolutionary algorithms that use extracted information from the population instead of traditional genetic operators to generate new solutions. This information is represented as a probabilistic model and the effectiveness of these algorithms is dependent on the quality of these models. However, some studies have shown that even multivariate EDAs fail to build a proper model in some problems. Usually, in these problems, there is intrinsic pairwise independence between variables. In the literature, there are few studies that investigate the difficulty and the nature of problems that can not be solved by EDAs easily. This paper proposes a new metric for measuring problem difficulty by examining the properties of model-building mechanisms in EDAs. For this purpose, we use the estimated Walsh coefficients of dependent and independent variables. The proposed metric is used to evaluate the difficulty of some well-known benchmark problems in EDAs. Different metrics like Fitness Distance Correlation (FDC) are used to compare how well the proposed metric measures problem difficulty for EDAs. Results indicate that the proposed metric can accurately predict the EDA's performance in different problems.
Supervised Training of Siamese Spiking Neural Networks with Earth Mover's Distance
Authors: Mateusz Pabian, Dominik Rzepka, Mirosław Pawlak
Date: 2022-02-20 00:00:00

This study adapts the highly-versatile siamese neural network model to the event data domain. We introduce a supervised training framework for optimizing Earth Mover's Distance (EMD) between spike trains with spiking neural networks (SNN). We train this model on images of the MNIST dataset converted into spiking domain with novel conversion schemes. The quality of the siamese embeddings of input images was evaluated by measuring the classifier performance for different dataset coding types. The models achieved performance similar to existing SNN-based approaches (F1-score of up to 0.9386) while using only about 15% of hidden layer neurons to classify each example. Furthermore, models which did not employ a sparse neural code were about 45% slower than their sparse counterparts. These properties make the model suitable for low energy consumption and low prediction latency applications.
GEMA: An open-source Python library for self-organizing-maps
Authors: Alvaro J. Garcia-Tejedor, Alberto Nogales
Date: 2022-02-17 00:00:00
Tasks: BIG-bench Machine Learning

Organizations have realized the importance of data analysis and its benefits. This in combination with Machine Learning algorithms has allowed to solve problems more easily, making these processes less time-consuming. Neural networks are the Machine Learning technique that is recently obtaining very good best results. This paper describes an open-source Python library called GEMA developed to work with a type of neural network model called Self-Organizing-Maps. GEMA is freely available under GNU General Public License at GitHub (https://github.com/ufvceiec/GEMA). The library has been evaluated in different a particular use case obtaining accurate results.
Temporal Heterogeneity Improves Speed and Convergence in Genetic Algorithms
Authors: Yoshio Martinez, Katya Rodriguez, Carlos Gershenson
Date: 2022-02-02 00:00:00

Genetic algorithms have been used in recent decades to solve a broad variety of search problems. These algorithms simulate natural selection to explore a parameter space in search of solutions for a broad variety of problems. In this paper, we explore the effects of introducing temporal heterogeneity in genetic algorithms. In particular, we set the crossover probability to be inversely proportional to the individual's fitness, i.e., better solutions change slower than those with a lower fitness. As case studies, we apply heterogeneity to solve the $N$-Queens and Traveling Salesperson problems. We find that temporal heterogeneity consistently improves search without having prior knowledge of the parameter space.
Battle royale optimizer with a new movement strategy
Authors: Sara Akan, Taymaz Akan
Date: 2022-01-19 00:00:00

Gamed-based is a new stochastic metaheuristics optimization category that is inspired by traditional or digital game genres. Unlike SI-based algorithms, in-dividuals do not work together with the goal of defeating other individuals and winning the game. Battle royale optimizer (BRO) is a Gamed-based me-taheuristic optimization algorithm that has been recently proposed for the task of continuous problems. This paper proposes a modified BRO (M-BRO) in order to improve balance between exploration and exploitation. For this matter, an additional movement operator has been used in the movement strategy. Moreover, no extra parameters are required for the proposed ap-proach. Furthermore, the complexity of this modified algorithm is the same as the original one. Experiments are performed on a set of 19 (unimodal and multimodal) benchmark functions (CEC 2010). The proposed method has been compared with the original BRO alongside six well-known/recently proposed optimization algorithms. The results show that BRO with additional movement operator performs well to solve complex numerical optimization problems compared to the original BRO and other competitors.
On the Kullback-Leibler divergence between pairwise isotropic Gaussian-Markov random fields
Authors: Alexandre L. M. Levada
Date: 2022-03-24 00:00:00
Tasks: Denoising, Image Denoising, Metric Learning

The Kullback-Leibler divergence or relative entropy is an information-theoretic measure between statistical models that play an important role in measuring a distance between random variables. In the study of complex systems, random fields are mathematical structures that models the interaction between these variables by means of an inverse temperature parameter, responsible for controlling the spatial dependence structure along the field. In this paper, we derive closed-form expressions for the Kullback-Leibler divergence between two pairwise isotropic Gaussian-Markov random fields in both univariate and multivariate cases. The proposed equation allows the development of novel similarity measures in image processing and machine learning applications, such as image denoising and unsupervised metric learning.
Bayesian Deep Learning with Multilevel Trace-class Neural Networks
Authors: Neil K. Chada, Ajay Jasra, Kody J. H. Law, Sumeetpal S. Singh
Date: 2022-03-24 00:00:00
Tasks: Bayesian Inference, Deep Learning, Uncertainty Quantification

In this article we consider Bayesian inference associated to deep neural networks (DNNs) and in particular, trace-class neural network (TNN) priors which can be preferable to traditional DNNs as (a) they are identifiable and (b) they possess desirable convergence properties. TNN priors are defined on functions with infinitely many hidden units, and have strongly convergent Karhunen-Loeve-type approximations with finitely many hidden units. A practical hurdle is that the Bayesian solution is computationally demanding, requiring simulation methods, so approaches to drive down the complexity are needed. In this paper, we leverage the strong convergence of TNN in order to apply Multilevel Monte Carlo (MLMC) to these models. In particular, an MLMC method that was introduced is used to approximate posterior expectations of Bayesian TNN models with optimal computational complexity, and this is mathematically proved. The results are verified with several numerical experiments on model problems arising in machine learning, including regression, classification, and reinforcement learning.
Robustness Against Weak or Invalid Instruments: Exploring Nonlinear Treatment Models with Machine Learning
Authors: Zijian Guo, Mengchu Zheng, Peter Bühlmann
Date: 2022-03-24 00:00:00
Tasks: BIG-bench Machine Learning, Causal Inference

We discuss causal inference for observational studies with possibly invalid instrumental variables. We propose a novel methodology called two-stage curvature identification (TSCI) by exploring the nonlinear treatment model with machine learning. {The first-stage machine learning enables improving the instrumental variable's strength and adjusting for different forms of violating the instrumental variable assumptions.} The success of TSCI requires the instrumental variable's effect on treatment to differ from its violation form. A novel bias correction step is implemented to remove bias resulting from the potentially high complexity of machine learning. Our proposed \texttt{TSCI} estimator is shown to be asymptotically unbiased and Gaussian even if the machine learning algorithm does not consistently estimate the treatment model. Furthermore, we design a data-dependent method to choose the best among several candidate violation forms. We apply TSCI to study the effect of education on earnings.
Learning the Dynamics of Autonomous Linear Systems From Multiple Trajectories
Authors: Lei Xin, George Chiu, Shreyas Sundaram
Date: 2022-03-24 00:00:00

We consider the problem of learning the dynamics of autonomous linear systems (i.e., systems that are not affected by external control inputs) from observations of multiple trajectories of those systems, with finite sample guarantees. Existing results on learning rate and consistency of autonomous linear system identification rely on observations of steady state behaviors from a single long trajectory, and are not applicable to unstable systems. In contrast, we consider the scenario of learning system dynamics based on multiple short trajectories, where there are no easily observed steady state behaviors. We provide a finite sample analysis, which shows that the dynamics can be learned at a rate $\mathcal{O}(\frac{1}{\sqrt{N}})$ for both stable and unstable systems, where $N$ is the number of trajectories, when the initial state of the system has zero mean (which is a common assumption in the existing literature). We further generalize our result to the case where the initial state has non-zero mean. We show that one can adjust the length of the trajectories to achieve a learning rate of $\mathcal{O}(\sqrt{\frac{\log{N}}{N})}$ for strictly stable systems and a learning rate of $\mathcal{O}(\frac{(\log{N})^d}{\sqrt{N}})$ for marginally stable systems, where $d$ is some constant.
Adaptive Regularization of B-Spline Models for Scientific Data
Authors: David Lenz, Raine Yeh, Vijay Mahadevan, Iulian Grindeanu, Tom Peterka
Date: 2022-03-23 00:00:00

B-spline models are a powerful way to represent scientific data sets with a functional approximation. However, these models can suffer from spurious oscillations when the data to be approximated are not uniformly distributed. Model regularization (i.e., smoothing) has traditionally been used to minimize these oscillations; unfortunately, it is sometimes impossible to sufficiently remove unwanted artifacts without smoothing away key features of the data set. In this article, we present a method of model regularization that preserves significant features of a data set while minimizing artificial oscillations. Our method varies the strength of a smoothing parameter throughout the domain automatically, removing artifacts in poorly-constrained regions while leaving other regions unchanged. The behavior of our method is validated on a collection of two- and three-dimensional data sets produced by scientific simulations.
Semantic system for searching of employees
Authors: Mariya Evtimova-Gardair, Tasho Tashev
Date: 2022-03-24 00:00:00
Tasks: Cultural Vocal Bursts Intensity Prediction

Many people have stress to leave their job and start a new one because of the new environment and not enough knowledge about the culture and structure about the new organization they are going to work in. New employees in company normally need to integrate in their working place environment quicker to start doing their job. That makes them ask a lot of questions to their colleagues and sometimes their colleagues are too busy to answer those questions. In the literature is defined that this problem could be solved when new employees use digital system for information as the proposed system for searching of information. Furthermore, the quality of the returned results from the searching system is defined as a standard for the efficiency of the searching systems. Because of this, it is proposed a semantic system for searching information of employees in a company that will help to better orient new employees in a company, to know the position and the function of each employee in the company
Predicting the longevity of resources shared in scientific publications
Authors: Daniel E. Acuna, Jian Jian, Tong Zeng, Lizhen Liang, Han Zhuang
Date: 2022-03-24 00:00:00
Tasks: Articles

Research has shown that most resources shared in articles (e.g., URLs to code or data) are not kept up to date and mostly disappear from the web after some years (Zeng et al., 2019). Little is known about the factors that differentiate and predict the longevity of these resources. This article explores a range of explanatory features related to the publication venue, authors, references, and where the resource is shared. We analyze an extensive repository of publications and, through web archival services, reconstruct how they looked at different time points. We discover that the most important factors are related to where and how the resource is shared, and surprisingly little is explained by the author's reputation or prestige of the journal. By examining the places where long-lasting resources are shared, we suggest that it is critical to disseminate and create standards with modern technologies. Finally, we discuss implications for reproducibility and recognizing scientific datasets as first-class citizens.
CorpusVis: Visual Analysis of Digital Sheet Music Collections
Authors: Matthias Miller, Julius Rauscher, Daniel A. Keim, Mennatallah El-Assady
Date: 2022-03-23 00:00:00
Tasks: Rhythm

Manually investigating sheet music collections is challenging for music analysts due to the magnitude and complexity of underlying features, structures, and contextual information. However, applying sophisticated algorithmic methods would require advanced technical expertise that analysts do not necessarily have. Bridging this gap, we contribute CorpusVis, an interactive visual workspace, enabling scalable and multi-faceted analysis. Our proposed visual analytics dashboard provides access to computational methods, generating varying perspectives on the same data. The proposed application uses metadata including composers, type, epoch, and low-level features, such as pitch, melody, and rhythm. To evaluate our approach, we conducted a pair analytics study with nine participants. The qualitative results show that CorpusVis supports users in performing exploratory and confirmatory analysis, leading them to new insights and findings. In addition, based on three exemplary workflows, we demonstrate how to apply our approach to different tasks, such as exploring musical features or comparing composers.
Food Recommendations for Reducing Water Footprint
Authors: Ignazio Gallo, Nicola Landro, Riccardo La Grassa, Andrea Turconi
Date: 2022-04-24 00:00:00
Proceeding: Sustainability 2022 4
Tasks: Food recommendation, Retrieval

Most existing food-related research efforts focus on recipe retrieval, user preference-based food recommendation, kitchen assistance, or nutritional and caloric estimation of dishes, ignoring personalized and conscious food recommendations resources of the planet. Therefore, in this work, we present a personalized food recommendation scheme, mapping the ingredients to the most resource-friendly dishes on the planet and in particular, selecting recipes that contain ingredients that consume as little water as possible for their production. The system proposed here is able to understand the user’s behavior and to suggest tailor-made recipes with lower water quantity used in production. By continuously using the system, the user can gradually reduce their water footprint and benefit from a healthier diet. The proposed recommendation system was compared with the results of two papers available in the literature that represent the state of the art, obtaining similar results. Therefore, the results of the presented recommendation system can be considered reliable.