pwc-archive/papers-with-abstracts huggingface dataset:
Building Knowledge-Grounded Dialogue Systems with Graph-Based Semantic Modeling
Authors: Yizhe Yang, Heyan Huang, Yang Gao, Jiawei Li and
Date: 2022-04-27 00:00:00
Tasks: Dialogue Generation, Response Generation, Text Generation

The knowledge-grounded dialogue task aims to generate responses that convey information from given knowledge documents. However, it is a challenge for the current sequence-based model to acquire knowledge from complex documents and integrate it to perform correct responses without the aid of an explicit semantic structure. To address these issues, we propose a novel graph structure, Grounded Graph ($G^2$), that models the semantic structure of both dialogue and knowledge to facilitate knowledge selection and integration for knowledge-grounded dialogue generation. We also propose a Grounded Graph Aware Transformer ($G^2AT$) model that fuses multi-forms knowledge (both sequential and graphic) to enhance knowledge-grounded response generation. Our experiments results show that our proposed model outperforms the previous state-of-the-art methods with more than 10\% gains in response generation and nearly 20\% improvement in factual consistency. Further, our model reveals good generalization ability and robustness. By incorporating semantic structures as prior knowledge in deep neural networks, our model provides an effective way to aid language generation.
Document-Level Relation Extraction with Sentences Importance Estimation and Focusing
Authors: Wang Xu, Kehai Chen, Lili Mou, Tiejun Zhao
Date: 2022-04-27 00:00:00
Proceeding: NAACL 2022 7
Tasks: Dialog Relation Extraction, Document-level Relation Extraction, Relation, Relation Extraction, Sentence

Document-level relation extraction (DocRE) aims to determine the relation between two entities from a document of multiple sentences. Recent studies typically represent the entire document by sequence- or graph-based models to predict the relations of all entity pairs. However, we find that such a model is not robust and exhibits bizarre behaviors: it predicts correctly when an entire test document is fed as input, but errs when non-evidence sentences are removed. To this end, we propose a Sentence Importance Estimation and Focusing (SIEF) framework for DocRE, where we design a sentence importance score and a sentence focusing loss, encouraging DocRE models to focus on evidence sentences. Experimental results on two domains show that our SIEF not only improves overall performance, but also makes DocRE models more robust. Moreover, SIEF is a general framework, shown to be effective when combined with a variety of base DocRE models.
Data-Driven Adaptive Simultaneous Machine Translation
Authors: Guangxu Xun, Mingbo Ma, Yuchen Bian, Xingyu Cai, Jiaji Huang, Renjie Zheng, Junkun Chen, Jiahong Yuan, Kenneth Church, Liang Huang
Date: 2022-04-27 00:00:00
Tasks: Machine Translation, Sentence, Translation

In simultaneous translation (SimulMT), the most widely used strategy is the wait-k policy thanks to its simplicity and effectiveness in balancing translation quality and latency. However, wait-k suffers from two major limitations: (a) it is a fixed policy that can not adaptively adjust latency given context, and (b) its training is much slower than full-sentence translation. To alleviate these issues, we propose a novel and efficient training scheme for adaptive SimulMT by augmenting the training corpus with adaptive prefix-to-prefix pairs, while the training complexity remains the same as that of training full-sentence translation models. Experiments on two language pairs show that our method outperforms all strong baselines in terms of translation quality and latency.
Better Query Graph Selection for Knowledge Base Question Answering
Authors: Yonghui Jia, Wenliang Chen
Date: 2022-04-27 00:00:00
Tasks: Knowledge Base Question Answering, Question Answering, Semantic Parsing

This paper presents a novel approach based on semantic parsing to improve the performance of Knowledge Base Question Answering (KBQA). Specifically, we focus on how to select an optimal query graph from a candidate set so as to retrieve the answer from knowledge base (KB). In our approach, we first propose to linearize the query graph into a sequence, which is used to form a sequence pair with the question. It allows us to use mature sequence modeling, such as BERT, to encode the sequence pair. Then we use a ranking method to sort candidate query graphs. In contrast to the previous studies, our approach can efficiently model semantic interactions between the graph and the question as well as rank the candidate graphs from a global view. The experimental results show that our system achieves the top performance on ComplexQuestions and the second best performance on WebQuestions.
Executive Function: A Contrastive Value Policy for Resampling and Relabeling Perceptions via Hindsight Summarization?
Authors: Chris Lengerich, Ben Lengerich
Date: 2022-04-27 00:00:00
Tasks: Continual Learning, Few-Shot Learning, Prompt Engineering

We develop the few-shot continual learning task from first principles and hypothesize an evolutionary motivation and mechanism of action for executive function as a contrastive value policy which resamples and relabels perception data via hindsight summarization to minimize attended prediction error, similar to an online prompt engineering problem. This is made feasible by the use of a memory policy and a pretrained network with inductive biases for a grammar of learning and is trained to maximize evolutionary survival. We show how this model of executive function can be used to implement hypothesis testing as a stream of consciousness and may explain observations of human few-shot learning and neuroanatomy.
Developing Universal Dependency Treebanks for Magahi and Braj
Authors: Mohit Raj, Shyam Ratan, Deepak Alok, Ritesh Kumar, Atul Kr. Ojha
Date: 2022-04-26 00:00:00

In this paper, we discuss the development of treebanks for two low-resourced Indian languages - Magahi and Braj based on the Universal Dependencies framework. The Magahi treebank contains 945 sentences and Braj treebank around 500 sentences marked with their lemmas, part-of-speech, morphological features and universal dependencies. This paper gives a description of the different dependency relationship found in the two languages and give some statistics of the two treebanks. The dataset will be made publicly available on Universal Dependency (UD) repository (https://github.com/UniversalDependencies/UD_Magahi-MGTB/tree/master) in the next(v2.10) release.
Efficient Machine Translation Domain Adaptation
Authors: Pedro Henrique Martins, Zita Marinho, André F. T. Martins
Date: 2022-04-26 00:00:00
Proceeding: SpaNLP (ACL) 2022 5
Tasks: Domain Adaptation, Language Modeling, Language Modelling, Machine Translation, Retrieval, Translation

Machine translation models struggle when translating out-of-domain text, which makes domain adaptation a topic of critical importance. However, most domain adaptation methods focus on fine-tuning or training the entire or part of the model on every new domain, which can be costly. On the other hand, semi-parametric models have been shown to successfully perform domain adaptation by retrieving examples from an in-domain datastore (Khandelwal et al., 2021). A drawback of these retrieval-augmented models, however, is that they tend to be substantially slower. In this paper, we explore several approaches to speed up nearest neighbor machine translation. We adapt the methods recently proposed by He et al. (2021) for language modeling, and introduce a simple but effective caching strategy that avoids performing retrieval when similar contexts have been seen before. Translation quality and runtimes for several domains show the effectiveness of the proposed solutions.
Run Time Analysis for Random Local Search on Generalized Majority Functions
Authors: Carola Doerr, Martin S. Krejca
Date: 2022-04-27 00:00:00
Tasks: Evolutionary Algorithms

Run time analysis of evolutionary algorithms recently makes significant progress in linking algorithm performance to algorithm parameters. However, settings that study the impact of problem parameters are rare. The recently proposed W-model provides a good framework for such analyses, generating pseudo-Boolean optimization problems with tunable properties. We initiate theoretical research of the W-model by studying how one of its properties -- neutrality -- influences the run time of random local search. Neutrality creates plateaus in the search space by first performing a majority vote for subsets of the solution candidate and then evaluating the smaller-dimensional string via a low-level fitness function. We prove upper bounds for the expected run time of random local search on this MAJORITY problem for its entire parameter spectrum. To this end, we provide a theorem, applicable to many optimization algorithms, that links the run time of MAJORITY with its symmetric version HASMAJORITY, where a sufficient majority is needed to optimize the subset. We also introduce a generalized version of classic drift theorems as well as a generalized version of Wald's equation, both of which we believe to be of independent interest.
On the Dynamics of Inference and Learning
Authors: David S. Berman, Jonathan J. Heckman, Marc Klinger
Date: 2022-04-19 00:00:00

Statistical Inference is the process of determining a probability distribution over the space of parameters of a model given a data set. As more data becomes available this probability distribution becomes updated via the application of Bayes' theorem. We present a treatment of this Bayesian updating process as a continuous dynamical system. Statistical inference is then governed by a first order differential equation describing a trajectory or flow in the information geometry determined by a parametric family of models. We solve this equation for some simple models and show that when the Cram\'{e}r-Rao bound is saturated the learning rate is governed by a simple $1/T$ power-law, with $T$ a time-like variable denoting the quantity of data. The presence of hidden variables can be incorporated in this setting, leading to an additional driving term in the resulting flow equation. We illustrate this with both analytic and numerical examples based on Gaussians and Gaussian Random Processes and inference of the coupling constant in the 1D Ising model. Finally we compare the qualitative behaviour exhibited by Bayesian flows to the training of various neural networks on benchmarked data sets such as MNIST and CIFAR10 and show how that for networks exhibiting small final losses the simple power-law is also satisfied.
RankMat : Matrix Factorization with Calibrated Distributed Embedding and Fairness Enhancement
Authors: Hao Wang
Date: 2022-04-27 00:00:00
Tasks: Fairness, Recommendation Systems, Word Embeddings

Matrix Factorization is a widely adopted technique in the field of recommender system. Matrix Factorization techniques range from SVD, LDA, pLSA, SVD++, MatRec, Zipf Matrix Factorization and Item2Vec. In recent years, distributed word embeddings have inspired innovation in the area of recommender systems. Word2vec and GloVe have been especially emphasized in many industrial application scenario such as Xiaomi's recommender system. In this paper, we propose a new matrix factorization inspired by the theory of power law and GloVe. Instead of the exponential nature of GloVe model, we take advantage of Pareto Distribution to model our loss function. Our method is explainable in theory and easy-to-implement in practice. In the experiment section, we prove our approach is superior to vanilla matrix factorization technique and comparable with GloVe-based model in both accuracy and fairness metrics.
MovieMat: Context-aware Movie Recommendation with Matrix Factorization by Matrix Fitting
Authors: Hao Wang
Date: 2022-04-27 00:00:00
Tasks: Collaborative Filtering, Fairness, Learning-To-Rank, Marketing, Movie Recommendation, Recommendation Systems

Movie Recommender System is widely applied in commercial environments such as NetFlix and Tubi. Classic recommender models utilize technologies such as collaborative filtering, learning to rank, matrix factorization and deep learning models to achieve lower marketing expenses and higher revenues. However, audience of movies have different ratings of the same movie in different contexts. Important movie watching contexts include audience mood, location, weather, etc. Tobe able to take advantage of contextual information is of great benefit to recommender builders. However, popular techniques such as tensor factorization consumes an impractical amount of storage, which greatly reduces its feasibility in real world environment. In this paper, we take advantage of the MatMat framework, which factorizes matrices by matrix fitting to build a context-aware movie recommender system that is superior to classic matrix factorization and comparable in the fairness metric.
On Survivorship Bias in MS MARCO
Authors: Prashansa Gupta, Sean MacAvaney
Date: 2022-04-27 00:00:00
Tasks: valid

Survivorship bias is the tendency to concentrate on the positive outcomes of a selection process and overlook the results that generate negative outcomes. We observe that this bias could be present in the popular MS MARCO dataset, given that annotators could not find answers to 38--45% of the queries, leading to these queries being discarded in training and evaluation processes. Although we find that some discarded queries in MS MARCO are ill-defined or otherwise unanswerable, many are valid questions that could be answered had the collection been annotated more completely (around two thirds using modern ranking techniques). This survivability problem distorts the MS MARCO collection in several ways. We find that it affects the natural distribution of queries in terms of the type of information needed. When used for evaluation, we find that the bias likely yields a significant distortion of the absolute performance scores observed. Finally, given that MS MARCO is frequently used for model training, we train models based on subsets of MS MARCO that simulates more survivorship bias. We find that models trained in this setting are up to 9.9% worse when evaluated on versions of the dataset with more complete annotations, and up to 3.5% worse at zero-shot transfer. Our findings are complementary to other recent suggestions for further annotation of MS MARCO, but with a focus on discarded queries. Code and data for reproducing the results of this paper are available in an online appendix.
A Practical Two-stage Ranking Framework for Cross-market Recommendation
Authors: Zeyuan Chen, He Wang, Xiangyu Zhu, Haiyan Wu, Congcong Gu, Shumeng Liu, Jinchao Huang, Wei zhang
Date: 2022-04-27 00:00:00
Tasks: Graph Neural Network, Vocal Bursts Valence Prediction

Cross-market recommendation aims to recommend products to users in a resource-scarce target market by leveraging user behaviors from similar rich-resource markets, which is crucial for E-commerce companies but receives less research attention. In this paper, we present our detailed solution adopted in the cross-market recommendation contest, i.e., WSDM CUP 2022. To better utilize collaborative signals and similarities between target and source markets, we carefully consider multiple features as well as stacking learning models consisting of deep graph recommendation models (Graph Neural Network, DeepWalk, etc.) and traditional recommendation models (ItemCF, UserCF, Swing, etc.). Furthermore, We adopt tree-based ensembling methods, e.g., LightGBM, which show superior performance in prediction task to generate final results. We conduct comprehensive experiments on the XMRec dataset, verifying the effectiveness of our model. The proposed solution of our team WSDM_Coggle_ is selected as the second place submission.
SBMLDiagrams: A python package to process and visualize SBML layout and render
Authors: Jin Xu, Jessie Jiang, Herbert M. Sauro
Date: 2022-04-26 00:00:00

Summary: The Systems Biology Markup Language (SBML) is an extensible standard format for exchanging biochemical models. One of the extensions for SBML is the SBML Layout and Render package. This allows modelers to describe a biochemical model as a pathway diagram. However, up to now there has been little support to help users easily add and retrieve such information from SBML. In this application note, we describe a new Python package called SBMLDiagrams. This package allows a user to add layout and render information or retrieve it using a straightforward Python API. The package uses skia-python to support the rendering of the diagrams, allowing export to commons formats such as PNG or PDF. Availability: SBMLDiagrams is publicly available and licensed under the liberal MIT open-source license. The package is available for all major platforms. The source code has been deposited at GitHub (github.com/sys-bio/SBMLDiagrams). Users can install the package using the standard pip installation mechanism: pip install SBMLDiagrams. Contact: hsauro@uw.edu.
An open-source simulation package for power electronics education
Authors: Mahesh B. Patil, V. V. S. Pavan Kumar Hari, Ruchita D. Korgaonkar, Kumar Appaiah
Date: 2022-04-25 00:00:00

Extension of the open-source simulation package GSEIM for power electronics applications is presented. Recent developments in GSEIM, including those oriented specifically towards power electronic circuits, are described. Some examples of electrical element templates, which form a part of the GSEIM library, are discussed. Representative simulation examples in power electronics are presented to bring out important features of the simulator. Advantages of GSEIM for educational purposes are discussed. Finally, plans regarding future developments in GSEIM are presented.
Surrogate Assisted Evolutionary Multi-objective Optimisation applied to a Pressure Swing Adsorption system
Authors: Liezl Stander, Matthew Woolway, Terence L. Van Zyl
Date: 2022-03-28 00:00:00
Tasks: BIG-bench Machine Learning, Computational Efficiency, Evolutionary Algorithms

Chemical plant design and optimisation have proven challenging due to the complexity of these real-world systems. The resulting complexity translates into high computational costs for these systems' mathematical formulations and simulation models. Research has illustrated the benefits of using machine learning surrogate models as substitutes for computationally expensive models during optimisation. This paper extends recent research into optimising chemical plant design and operation. The study further explores Surrogate Assisted Genetic Algorithms (SA-GA) in more complex variants of the original plant design and optimisation problems, such as the inclusion of parallel and feedback components. The novel extension to the original algorithm proposed in this study, Surrogate Assisted NSGA-\Romannum{2} (SA-NSGA), was tested on a popular literature case, the Pressure Swing Adsorption (PSA) system. We further provide extensive experimentation, comparing various meta-heuristic optimisation techniques and numerous machine learning models as surrogates. The results for both sets of systems illustrate the benefits of using Genetic Algorithms as an optimisation framework for complex chemical plant system design and optimisation for both single and multi-objective scenarios. We confirm that Random Forest surrogate assisted Evolutionary Algorithms can be scaled to increasingly complex chemical systems with parallel and feedback components. We further find that combining a Genetic Algorithm framework with Machine Learning Surrogate models as a substitute for long-running simulation models yields significant computational efficiency improvements, 1.7 - 1.84 times speedup for the increased complexity examples and a 2.7 times speedup for the Pressure Swing Adsorption system.
Sequence-Based Target Coin Prediction for Cryptocurrency Pump-and-Dump
Authors: Sihao Hu, Zhen Zhang, Shengliang Lu, Bingsheng He, Zhao Li
Date: 2022-04-21 00:00:00

With the proliferation of pump-and-dump schemes (P&Ds) in the cryptocurrency market, it becomes imperative to detect such fraudulent activities in advance to alert potentially susceptible investors. In this paper, we focus on predicting the pump probability of all coins listed in the target exchange before a scheduled pump time, which we refer to as the target coin prediction task. Firstly, we conduct a comprehensive study of the latest 709 P&D events organized in Telegram from Jan. 2019 to Jan. 2022. Our empirical analysis reveals some interesting patterns of P&Ds, such as that pumped coins exhibit intra-channel homogeneity and inter-channel heterogeneity. Here channel refers a form of group in Telegram that is frequently used to coordinate P&D events. This observation inspires us to develop a novel sequence-based neural network, dubbed SNN, which encodes a channel's P&D event history into a sequence representation via the positional attention mechanism to enhance the prediction accuracy. Positional attention helps to extract useful information and alleviates noise, especially when the sequence length is long. Extensive experiments verify the effectiveness and generalizability of proposed methods. Additionally, we release the code and P&D dataset on GitHub: https://github.com/Bayi-Hu/Pump-and-Dump-Detection-on-Cryptocurrency, and regularly update the dataset.
Multi-task Learning-based CSI Feedback Design in Multiple Scenarios
Authors: Xiangyi Li, Jiajia Guo, Chao-Kai Wen, Shi Jin, Shuangfeng Han, XiaoYun Wang
Date: 2022-04-27 00:00:00
Tasks: Decoder, Multi-Task Learning

For frequency division duplex systems, the essential downlink channel state information (CSI) feedback includes the links of compression, feedback, decompression and reconstruction to reduce the feedback overhead. One efficient CSI feedback method is the Auto-Encoder (AE) structure based on deep learning, yet facing problems in actual deployments, such as selecting the deployment mode when deploying in a cell with multiple complex scenarios. Rather than designing an AE network with huge complexity to deal with CSI of all scenarios, a more realistic mode is to divide the CSI dataset by region/scenario and use multiple relatively simple AE networks to handle subregions' CSI. However, both require high memory capacity for user equipment (UE) and are not suitable for low-level devices. In this paper, we propose a new user-friendly-designed framework based on the latter multi-tasking mode. Via Multi-Task Learning, our framework, Single-encoder-to-Multiple-decoders (S-to-M), designs the multiple independent AEs into a joint architecture: a shared encoder corresponds to multiple task-specific decoders. We also complete our framework with GateNet as a classifier to enable the base station autonomously select the right task-specific decoder corresponding to the subregion. Experiments on the simulating multi-scenario CSI dataset demonstrate our proposed S-to-M's advantages over the other benchmark modes, i.e., significantly reducing the model complexity and the UE's memory consumption