Reviewerly/exHarmony huggingface dataset:
Adversarial Music: Real world Audio Adversary against Wake-word Detection System

Voice Assistants (VAs) such as Amazon Alexa or Google Assistant rely on wake-word detection to respond to people's commands, which could potentially be vulnerable to audio adversarial examples. In this work, we target our attack on the wake-word detection system. Our goal is to jam the model with some inconspicuous background music to deactivate the VAs while our audio adversary is present. We implemented an emulated wake-word detection system of Amazon Alexa based on recent publications. We validated our models against the real Alexa in terms of wake-word detection accuracy. Then we computed our audio adversaries with consideration of expectation over transform and we implemented our audio adversary with a differentiable synthesizer. Next we verified our audio adversaries digitally on hundreds of samples of utterances collected from the real world. Our experiments show that we can effectively reduce the recognition F1 score of our emulated model from 93.4% to 11.0%. Finally, we tested our audio adversary over the air, and verified it works effectively against Alexa, reducing its F1 score from 92.5% to 11.0%. To the best of our knowledge, this is the first real-world adversarial attack against a commercial grade VA wake-word detection system. Our demo video is included in the supplementary material.
Variational Structured Semantic Inference for Diverse Image Captioning

Despite the exciting progress in image captioning, generating diverse captions for a given image remains as an open problem. Existing methods typically apply generative models such as Variational Auto-Encoder to diversify the captions, which however neglect two key factors of diverse expression, i.e., the lexical diversity and the syntactic diversity. To model these two inherent diversities in image captioning, we propose a Variational Structured Semantic Inferring model (termed VSSI-cap) executed in a novel structured encoder-inferer-decoder schema. VSSI-cap mainly innovates in a novel structure, i.e., Variational Multi-modal Inferring tree (termed VarMI-tree). In particular, conditioned on the visual-textual features from the encoder, the VarMI-tree models the lexical and syntactic diversities by inferring their latent variables (with variations) in an approximate posterior inference guided by a visual semantic prior. Then, a reconstruction loss and the posterior-prior KL-divergence are jointly estimated to optimize the VSSI-cap model. Finally, diverse captions are generated upon the visual features and the latent variables from this structured encoder-inferer-decoder model. Experiments on the benchmark dataset show that the proposed VSSI-cap achieves significant improvements over the state-of-the-arts.
Delta-training: Simple Semi-Supervised Text Classification using Pretrained Word Embeddings

We propose a novel and simple method for semi-supervised text classification.The method stems from the hypothesis that a classifier with pretrained word embeddings always outperforms the same classifier with randomly initialized word embeddings, as empirically observed in NLP tasks.Our method first builds two sets of classifiers as a form of model ensemble, and then initializes their word embeddings differently: one using random, the other using pretrained word embeddings.We focus on different predictions between the two classifiers on unlabeled data while following the self-training framework.We also use earlystopping in meta-epoch to improve the performance of our method.Our method, Deltatraining, outperforms the self-training and the co-training framework in 4 different text classification datasets, showing robustness against error accumulation.
Language Modeling for Code-Switching: Evaluation, Integration of Monolingual Data, and Discriminative Training

We focus on the problem of language modeling for code-switched language, in the context of automatic speech recognition (ASR).Language modeling for code-switched language is challenging for (at least) three reasons: (1) lack of available large-scale codeswitched data for training; (2) lack of a replicable evaluation setup that is ASR directed yet isolates language modeling performance from the other intricacies of the ASR system; and(3) the reliance on generative modeling.We tackle these three issues: we propose an ASRmotivated evaluation setup which is decoupled from an ASR system and the choice of vocabulary, and provide an evaluation dataset for English-Spanish code-switching.This setup lends itself to a discriminative training approach, which we demonstrate to work better than generative language modeling.Finally, we explore a variety of training protocols and verify the effectiveness of training with large amounts of monolingual data followed by finetuning with small amounts of code-switched data, for both the generative and discriminative cases.
Answering questions by learning to rank - Learning to rank by answering questions

Answering multiple-choice questions in a setting in which no supporting documents are explicitly provided continues to stand as a core problem in natural language processing.The contribution of this article is two-fold.First, it describes a method which can be used to semantically rank documents extracted from Wikipedia or similar natural language corpora.Second, we propose a model employing the semantic ranking that holds the first place in two of the most popular leaderboards for answering multiple-choice questions: ARC Easy and Challenge.To achieve this, we introduce a self-attention based neural network that latently learns to rank documents by their importance related to a given question, whilst optimizing the objective of predicting the correct answer.These documents are considered relevant contexts for the underlying question.We have published the ranked documents so that they can be used off-the-shelf to improve downstream decision models.
An Experimental Method to Estimate Running Time of Evolutionary Algorithms for Continuous Optimization

Running time analysis is a fundamental problem of critical importance in evolutionary computation. However, the analysis results have rarely been applied to advanced evolutionary algorithms (EAs) in practice, let alone their variants for continuous optimization. In this paper, an experimental method is proposed for analyzing the running time of EAs that are widely used for solving continuous optimization problems. Based on Glivenko-Cantelli theorem, the proposed method simulates the distribution of gain, which is introduced by average gain model to characterize progress during the optimization process. Data fitting techniques are subsequently adopted to obtain a desired function for further analyses. To verify the validity of the proposed method, experiments were conducted to estimate the upper bounds on expected first hitting time of various evolutionary strategies, such as (1, $\lambda $ ) evolution strategy, standard evolution strategy, covariance matrix adaptation evolution strategy, and its improved variants. The results suggest that all estimated upper bounds are correct. Backed up by the proposed method, state-of-the-art EAs for continuous optimization will have identical results about the running time as simplified schemes, which will bridge the gap between theoretical foundation and applications of evolutionary computation.
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning

Popular metrics used for evaluating image captioning systems, such as BLEU and CIDEr, provide a single score to gauge the system's overall effectiveness.This score is often not informative enough to indicate what specific errors are made by a given system.In this study, we present a fine-grained evaluation method REO for automatically measuring the performance of image captioning systems.REO assesses the quality of captions from three perspectives: 1) Relevance to the ground truth, 2) Extraness of the content that is irrelevant to the ground truth, and 3) Omission of the elements in the images and human references.Experiments on three benchmark datasets demonstrate that our method achieves a higher consistency with human judgments and provides more intuitive evaluation results than alternative metrics. 1
Effective Use of Transformer Networks for Entity Tracking

Tracking entities in procedural language requires understanding the transformations arising from actions on entities as well as those entities' interactions.While self-attention-based pre-trained language encoders like GPT and BERT have been successfully applied across a range of natural language understanding tasks, their ability to handle the nuances of procedural texts is still untested.In this paper, we explore the use of pre-trained transformer networks for entity tracking tasks in procedural text.First, we test standard lightweight approaches for prediction with pre-trained transformers, and find that these approaches underperform even simple baselines.We show that much stronger results can be attained by restructuring the input to guide the transformer model to focus on a particular entity.Second, we assess the degree to which transformer networks capture the process dynamics, investigating such factors as merged entities and oblique entity references.On two different tasks, ingredient detection in recipes and QA over scientific processes, we achieve state-ofthe-art results, but our models still largely attend to shallow context clues and do not form complex representations of intermediate entity or process state. 1 Seq. of Steps water mixture sugarRoots absorb water from soil.
Towards an Ontology for Public Procurement Based on the Open Contracting Data Standard

The release of a growing amount of open procurement data led to various initiatives for harmonising the data being provided. Among others, the Open Contracting Data Standard (OCDS) is highly relevant due to its high practical value and increasing traction. OCDS defines a common data model for publishing structured data throughout most of the stages of a contracting process. OCDS is document-oriented and focuses on packaging and delivering relevant data in an iterative and event-driven manner through a series of releases. Ontologies, beyond providing uniform access to heterogeneous procurement data, could enable integration with related data sets such as with supplier data for advanced analytics and insight extraction. Therefore, we developed an ontology, the “OCDS ontology”, by using OCDS’ main domain perspective and vocabulary, since it is an essential source of domain knowledge. In this paper, we provide an overview of the developed ontology.
Inpainting-Based Virtual Try-on Network for Selective Garment Transfer

Image-based garment transfer systems aim to swap the desired clothes from a model to arbitrary users. However, existing works cannot provide the capacity for users to try on various fashion articles according to their wishes, i.e., users can decide which article (e.g., tops, pants or both) to be swapped. In this paper, we propose an Inpainting-based Virtual Try-On Network (I-VTON) which allows the user to try on arbitrary clothes from the model image in a selective manner. To realize the selectivity, we reshape the virtual try-on as a task of image inpainting. Firstly, the texture from the garment and the user are extracted respectively to form a coarse result. In this phase, users can decide which clothes they hope to try on via an interactive texture control mechanism. Secondly, the missing regions in the coarse result are recovered via a Texture Inpainting Network (TIN). We introduce a triplet training strategy to ensure the naturalness of the final result. Qualitative and quantitative experimental results demonstrate that I-VTON outperforms the state-of-the-art methods on both the garment details and the user identity. It is also confirmed our approach can flexibly transfer the clothes in a selective manner.
Planning for Hybrid Systems via Satisfiability Modulo Theories

Planning for hybrid systems is important for dealing with real-world applications, and PDDL+ supports this representation of domains with mixed discrete and continuous dynamics. In this paper we present a new approach for planning for hybrid systems, based on encoding the planning problem as a Satisfiability Modulo Theories (SMT) formula. This is the first SMT encoding that can handle the whole set of PDDL+ features (including processes and events), and is implemented in the planner SMTPlan. SMTPlan not only covers the full semantics of PDDL+, but can also deal with non-linear polynomial continuous change without discretization. This allows it to generate plans with non-linear dynamics that are correct-by-construction. The encoding is based on the notion of happenings, and can be applied on domains with nonlinear continuous change. We describe the encoding in detail and provide in-depth examples. We apply this encoding in an iterative deepening planning algorithm. Experimental results show that the approach dramatically outperforms existing work in finding plans for PDDL+ problems. We also present experiments which explore the performance of the proposed approach on temporal planning problems, showing that the scalability of the approach is limited by the size of the discrete search space. We further extend the encoding to include planning with control parameters. The extended encoding allows the definition of actions to include infinite domain parameters, called control parameters. We present experiments on a set of problems with control parameters to demonstrate the positive effect they provide to the approach of planning via SMT.
Missing Data Imputation With OLS-Based Autoencoder for Intelligent Manufacturing

Motivated by the global economy that is greatly shaped by the landscape changes in energy and manufacturing where more and more devices and systems are interconnected, intelligent manufacturing in which data mining is of great importance is studied. In this article, an energy monitoring platform for small- and medium-sized enterprises developed by the point energy team (www.pointenergy.org) is first introduced, which monitors and records the energy consumption of manufacturing processes at various levels of granularity. In processing the collected data, the incompleteness in the data due to various factors needs to be addressed first otherwise it may lead to the inaccurate portrayal of the system and poor generalization of the resultant model trained by the data. Hence, a novel orthogonal-least-square-based autoencoder is proposed to generate new samples for the imputation of missing values. This approach is to learn the representative code from the original samples by constructing an improved encoder network in which the hidden neurons are orthogonal with each other. The new samples are then generated through the decoder network. The proposed approach selects the hidden neurons one by one based on the OLS estimation until an adequate network is built. The classical techniques and other generative models are compared to verify the effectiveness of the proposed algorithm. For these methods, the optimal parameters are estimated based on the performance metric of the cross-validation mean square error. In the experiment, two real industrial datasets from a baking process and a polymer extrusion process are adopted and the percentage of missing values varies from 0.02 to 0.25. The experimental results confirm that the proposed method offers stable performance in the presence of different missing ratios, and it outperforms significantly alternative approaches while the missing ratio is greater than 0.05.
Polarization-state tracking based on Kalman filter in continuous-variable quantum key distribution

Continuous-variable quantum key distribution with a real local oscillator (LO) has been extensively studied recently due to its security and simplicity. Making it operate automatically and resistant to channel interference is the key to further long-term system operation. To overcome the dynamic changes in the state of polarization (SOP) of quantum signal caused by the random birefringence effect in fiber, Kalman filter is employed to estimate the polarization misalignment, thereby achieving polarization demultiplexing at the data level, and ultimately recovering the quantum signal with the help of a two-step phase compensation. The signal transmission and processing is simulated, which verifies the SOP tracking ability and the immunity to the fast phase drift. A proof-of-principle experiment is also conducted, and results show that it can resist the interference of SOP rotation at 1 krad/s, and has the ability to distribute 8.4 kbps secret key rate within 20 km when only considering SOP tracking imperfection, confirming its feasibility under harsh channel conditions.
Governance of Decentralized Autonomous Organizations

This paper draws parallels between the OECD principle on disclosure and transparency and the decentralized autonomous organization (DAO) governance on a blockchain.It provides an overview of the DAO concept based on self-executing smart contracts on a blockchain.The blockchain keeps a record of data and tranasctions in a decentralized and immutable form.The self-executing smart contracts on a blockchain ensure transparency and automation of the decision-making process in a DAO.The paper concludes that a DAO governance structure provides for transparency and enables shareholders to exercise their rights in an informed way.Considering the early stage of development of DAOs, however, caution is needed especially with regards to potential protocol vulnerabilities and legal uncertainties.
Learning a No-Reference Quality Predictor of Stereoscopic Images by Visual Binocular Properties

In this work, we develop a novel no-reference (NR) quality assessment metric for stereoscopic images based on monocular and binocular features, motivated by visual perception properties of the human visual system (HVS) named binocular rivalry and binocular integration. To be more specific, we first calculate the normalized intensity feature maps of right- and left-view images through local contrast normalization, where statistical intensity features are extracted by the histogram of the normalized intensity feature map to represent monocular features. Then, we compute the disparity map of stereoscopic image, with which we extract structure feature map of stereoscopic image based on local binary pattern (LBP). We further extract statistical structure features and statistical depth features from structure feature map and disparity map by histogram to represent binocular features. Finally, we adopt support vector regression (SVR) to train the mapping function from the extracted monocular and binocular features to subjective quality scores. Comparison experiments are conducted on four large-scale stereoscopic image databases and the results demonstrate the promising performance of the proposed method in stereoscopic image quality assessment.
Towards Universal Semantic Representation

Natural language understanding at the semantic level and independent of language variations is of great practical value.Existing approaches such as semantic role labeling (SRL) and abstract meaning representation (AMR) still have features related to the peculiarities of the particular language.In this work we describe various challenges and possible solutions in designing a semantic representation that is universal across a variety of languages.
Arabic–Chinese Neural Machine Translation: Romanized Arabic as Subword Unit for Arabic-sourced Translation

Morphologically rich and complex languages such as Arabic, pose a major challenge to neural machine translation (NMT) due to the large number of rare words and the inability of NMT to translate them.Unknown word (UNK) symbols are used to represent out-of-vocabulary words because NMT typically operates with a fixed vocabulary size.These rare words can be effectively encoded as sequences of subword units by using algorithms, such as byte pair encoding (BPE), to tackle the UNK problem.However, for languages with highly inflected and morphological variations, such as Arabic, the aforementioned method has its own limitations that make it not effective enough for translation quality.To alleviate the UNK problem and address the inconvenient behavior of BPE when translating the Arabic language, we propose to utilize a romanization system that converts Arabic scripts to subword units.We investigate the effect of our approach on NMT performance under various segmentation scenarios and compare the results with systems trained on original Arabic form.In addition, we integrate Romanized Arabic as an input factor for Arabic-sourced NMT compared with well-known factors, namely, lemma, part-of-speech tags, and morph features.Extensive experiments on Arabic-Chinese translation demonstrate that the proposed approaches can effectively tackle the UNK problem and significantly improve the translation quality for Arabic-sourced translation.Additional experiments in this study focus on developing the NMT system on Chinese-Arabic translation.Before implementing our experiments, we first propose standard criteria for the data filtering of a parallel corpus, which helps in filtering out its noise.
Measuring Online Debaters’ Persuasive Skill from Text over Time

Online debates allow people to express their persuasive abilities and provide exciting opportunities for understanding persuasion. Prior studies have focused on studying persuasion in debate content, but without accounting for each debater’s history or exploring the progression of a debater’s persuasive ability. We study debater skill by modeling how participants progress over time in a collection of debates from Debate.org . We build on a widely used model of skill in two-player games and augment it with linguistic features of a debater’s content. We show that online debaters’ skill levels do tend to improve over time. Incorporating linguistic profiles leads to more robust skill estimation than winning records alone. Notably, we find that an interaction feature combining uncertainty cues (hedging) with terms strongly associated with either side of a particular debate (fightin’ words) is more predictive than either feature on its own, indicating the importance of fine- grained linguistic features.
On the Role of Style in Parsing Speech with Neural Models

The differences in written text and conversational speech are substantial; previous parsers trained on treebanked text have given very poor results on spontaneous speech. For spoken language, the mismatch in style also extends to prosodic cues, though it is less well understood. This paper re-examines the use of written text in parsing speech in the context of recent advances in neural language processing. We show that neural approaches facilitate using written text to improve parsing of spontaneous speech, and that prosody further improves over this state-of-the-art result. Further, we find an asymmetric degradation from read vs. spontaneous mismatch, with spontaneous speech more generally useful for training parsers.