Reviewerly/exHarmony huggingface dataset:
Hierarchical Multiple Kernel Clustering

Current multiple kernel clustering algorithms compute a partition with the consensus kernel or graph learned from the pre-specified ones, while the emerging late fusion methods firstly construct multiple partitions from each kernel separately, and then obtain a consensus one with them. However, both of them directly distill the clustering information from kernels or graphs to partition matrices, where the sudden dimension drop would result in loss of advantageous details for clustering. In this paper, we provide a brief insight of the aforementioned issue and propose a hierarchical approach to perform clustering while preserving advantageous details maximumly. Specifically, we gradually group samples into fewer clusters, together with generating a sequence of intermediary matrices of descending sizes. The consensus partition with is simultaneously learned and conversely guides the construction of intermediary matrices. Nevertheless, this cyclic process is modeled into an unified objective and an alternative algorithm is designed to solve it. In addition, the proposed method is validated and compared with other representative multiple kernel clustering algorithms on benchmark datasets, demonstrating state-of-the-art performance by a large margin.
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders

Encoder pre-training is promising in end-toend Speech Translation (ST), given the fact that speech-to-translation data is scarce.But ST encoders are not simple instances of Automatic Speech Recognition (ASR) or Machine Translation (MT) encoders.For example, we find that ASR encoders lack the global context representation, which is necessary for translation, whereas MT encoders are not designed to deal with long but locally attentive acoustic sequences.In this work, we propose a Stacked Acoustic-and-Textual Encoding (SATE) method for speech translation.Our encoder begins with processing the acoustic sequence as usual, but later behaves more like an MT encoder for a global representation of the input sequence.In this way, it is straightforward to incorporate the pre-trained models into the system.Also, we develop an adaptor module to alleviate the representation inconsistency between the pre-trained ASR encoder and MT encoder, and develop a multi-teacher knowledge distillation method to preserve the pre-training knowledge.Experimental results on the LibriSpeech En-Fr and MuST-C En-De ST tasks show that our method achieves state-of-the-art BLEU scores of 18.3 and 25.2.To our knowledge, we are the first to develop an end-to-end ST system that achieves comparable or even better BLEU performance than the cascaded ST counterpart when large-scale ASR and MT data is available 1 .
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task

Yun Tang, Juan Pino, Xian Li, Changhan Wang, Dmitriy Genzel. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Knowledge-driven Data Construction for Zero-shot Evaluation in Commonsense Question Answering

Recent developments in pre-trained neural language modeling have led to leaps in accuracy on common-sense question-answering benchmarks. However, there is increasing concern that models overfit to specific tasks, without learning to utilize external knowledge or perform general semantic reasoning. In contrast, zero-shot evaluations have shown promise as a more robust measure of a model’s general reasoning abilities. In this paper, we propose a novel neuro-symbolic framework for zero-shot question answering across commonsense tasks. Guided by a set of hypotheses, the framework studies how to transform various pre-existing knowledge resources into a form that is most effective for pre-training models. We vary the set of language models, training regimes, knowledge sources, and data generation strategies, and measure their impact across tasks. Extending on prior work, we devise and compare four constrained distractor-sampling strategies. We provide empirical results across five commonsense question-answering tasks with data generated from five external knowledge resources. We show that, while an individual knowledge graph is better suited for specific tasks, a global knowledge graph brings consistent gains across different tasks. In addition, both preserving the structure of the task as well as generating fair and informative questions help language models learn more effectively.
Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment

The cross-lingual language models are typically pretrained with masked language modeling on multilingual text or parallel sentences.In this paper, we introduce denoising word alignment as a new cross-lingual pre-training task.Specifically, the model first self-labels word alignments for parallel sentences.Then we randomly mask tokens in a bitext pair.Given a masked token, the model uses a pointer network to predict the aligned token in the other language.We alternately perform the above two steps in an expectationmaximization manner.Experimental results show that our method improves cross-lingual transferability on various datasets, especially on the token-level tasks, such as question answering, and structured prediction.Moreover, the model can serve as a pretrained word aligner, which achieves reasonably low error rates on the alignment benchmarks.The code and pretrained parameters are available at github.com/CZWin32768/XLM-Align.
MILE: A Multi-Level Framework for Scalable Graph Embedding

Recently there has been a surge of interest in designing graph embedding methods. Few, if any, can scale to a large-sized graph with millions of nodes due to both computational complexity and memory requirements. In this paper, we relax this limitation by introducing the MultI-Level Embedding (MILE) framework – a generic methodology allowing contemporary graph embedding methods to scale to large graphs. MILE repeatedly coarsens the graph into smaller ones using a hybrid matching technique to maintain the backbone structure of the graph. It then applies existing embedding methods on the coarsest graph and refines the embeddings to the original graph through a graph convolution neural network that it learns. The proposed MILE framework is agnostic to the underlying graph embedding techniques and can be applied to many existing graph embedding methods without modifying them. We employ our framework on several popular graph embedding techniques and conduct embedding for real-world graphs. Experimental results on five large-scale datasets demonstrate that MILE significantly boosts the speed (order of magnitude) of graph embedding while generating embeddings of better quality, for the task of node classification. MILE can comfortably scale to a graph with 9 million nodes and 40 million edges, on which existing methods run out of memory or take too long to compute on a modern workstation. Our code and data are publicly available with detailed instructions for adding new base embedding methods: https://github.com/jiongqian/MILE.
Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Recently an impressive development in immersive technologies, such as Augmented Reality (AR), Virtual Reality (VR) and 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> video, has been witnessed. However, methods for quality assessment have not been keeping up. This paper studies quality assessment of 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> video from the cross-lab tests (involving ten laboratories and more than 300 participants) carried out by the Immersive Media Group (IMG) of the Video Quality Experts Group (VQEG). These tests were addressed to assess and validate subjective evaluation methodologies for 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> video. Audiovisual quality, simulator sickness symptoms, and exploration behavior were evaluated with short (from 10 seconds to 30 seconds) 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> sequences. The following factors&#x2019; influences were also analyzed: assessment methodology, sequence duration, Head-Mounted Display (HMD) device, uniform and non-uniform coding degradations, and simulator sickness assessment methods. The obtained results have demonstrated the validity of Absolute Category Rating (ACR) and Degradation Category Rating (DCR) for subjective tests with 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> videos, the possibility of using 10-second videos (with or without audio) when addressing quality evaluation of coding artifacts, as well as any commercial HMD (satisfying minimum requirements). Also, more efficient methods than the long Simulator Sickness Questionnaire (SSQ) have been proposed to evaluate related symptoms with 360<inline-formula><tex-math notation="LaTeX">${^\circ }$</tex-math></inline-formula> videos. These results have been instrumental for the development of the ITU-T Recommendation P.919. Finally, the annotated dataset from the tests is made publicly available for the research community.
Beyond Accuracy: Behavioral Testing of NLP Models with Checklist (Extended Abstract)

Although measuring held-out accuracy has been the primary approach to evaluate generalization, it often overestimates the performance of NLP models, while alternative approaches for evaluating models either focus on individual tasks or on specific behaviors. Inspired by principles of behavioral testing in software engineering, we introduce CheckList, a task-agnostic methodology for testing NLP models. CheckList includes a matrix of general linguistic capabilities and test types that facilitate comprehensive test ideation, as well as a software tool to generate a large and diverse number of test cases quickly. We illustrate the utility of CheckList with tests for three tasks, identifying critical failures in both commercial and state-of-art models. In a user study, a team responsible for a commercial sentiment analysis model found new and actionable bugs in an extensively tested model. In another user study, NLP practitioners with CheckList created twice as many tests, and found almost three times as many bugs as users without it.
Fast binomial-code holonomic quantum computation with ultrastrong light-matter coupling

We propose a protocol for bosonic binomial-code nonadiabatic holonomic quantum computation in a system composed of an artificial atom ultrastrongly coupled to a cavity resonator. In our protocol, the binomial codes, formed by superpositions of Fock states, can greatly save physical resources to correct errors in quantum computation. We apply to the system strong driving fields designed by shortcuts-to-adiabatic methods. This reduces the gate time to tens of nanoseconds. Noise induced by control imperfections can be suppressed by a systematic-error-sensitivity nullification method. As a result, this protocol can rapidly $(\ensuremath{\sim}35\phantom{\rule{0.16em}{0ex}}\mathrm{ns})$ generate fault-tolerant and high-fidelity ($\ensuremath{\gtrsim}98%$ with experimentally realistic parameters) quantum gates.
A TinyML Platform for On-Device Continual Learning With Quantized Latent Replays

In the last few years, research and development on Deep Learning models & techniques for ultra-low-power devices– in a word, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TinyML</i> – has mainly focused on a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">train-then-deploy</i> assumption, with static models that cannot be adapted to newly collected data without cloud-based data collection and fine-tuning. Latent Replay-based Continual Learning (CL) techniques (Pellegrini <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> , 2020) enable online, serverless adaptation in principle, but so far they have still been too computation- and memory-hungry for ultra-low-power TinyML devices, which are typically based on microcontrollers. In this work, we introduce a HW/SW platform for end-to-end CL based on a 10-core <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FP32</i> -enabled parallel ultra-low-power (PULP) processor. We rethink the baseline Latent Replay CL algorithm, leveraging quantization of the frozen stage of the model and Latent Replays (LRs) to reduce their memory cost with minimal impact on accuracy. In particular, 8-bit compression of the LR memory proves to be almost lossless (−0.26% with 3000LR) compared to the full-precision baseline implementation, but requires <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4\times $ </tex-math></inline-formula> less memory, while 7-bit can also be used with an additional minimal accuracy degradation (up to 5%). We also introduce optimized primitives for forward and backward propagation on the PULP processor, together with data tiling strategies to fully exploit its memory hierarchy, while maximizing efficiency. Our results show that by combining these techniques, continual learning can be achieved in practice using less than 64MB of memory – an amount compatible with embedding in TinyML devices. On an advanced 22nm prototype of our platform, called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">VEGA</i> , the proposed solution performs on average <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$65 \times $ </tex-math></inline-formula> faster than a low-power STM32 L4 microcontroller, being <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$37\times $ </tex-math></inline-formula> more energy efficient – enough for a lifetime of 535h when learning a new mini-batch of data once every minute.
Low-altitude small-sized object detection using lightweight feature-enhanced convolutional neural network

Unauthorized operations referred to as "black flights" of unmanned aerial vehicles (UAVs) pose a significant danger to public safety, and existing low-attitude object detection algorithms encounter difficulties in balancing detection precision and speed.Additionally, their accuracy is insufficient, particularly for small objects in complex environments.To solve these problems, we propose a lightweight feature-enhanced convolutional neural network able to perform detection with high precision detection for low-attitude flying objects in real time to provide guidance information to suppress black-flying UAVs.The proposed network consists of three modules.A lightweight and stable feature extraction module is used to reduce the computational load and stably extract more low-level feature, an enhanced feature processing module significantly improves the feature extraction ability of the model, and an accurate detection module integrates low-level and advanced features to improve the multiscale detection accuracy in complex environments, particularly for small objects.The proposed method achieves a detection speed of 147 frames per second (FPS) and a mean average precision (mAP) of 90.97% for a dataset composed of flying objects, indicating its potential for low-altitude object detection.Furthermore, evaluation results based on microsoft common objects in context (MS COCO) indicate that the proposed method is also applicable to object detection in general.
Detecting phishing websites using machine learning technique

In recent years, advancements in Internet and cloud technologies have led to a significant increase in electronic trading in which consumers make online purchases and transactions. This growth leads to unauthorized access to users' sensitive information and damages the resources of an enterprise. Phishing is one of the familiar attacks that trick users to access malicious content and gain their information. In terms of website interface and uniform resource locator (URL), most phishing webpages look identical to the actual webpages. Various strategies for detecting phishing websites, such as blacklist, heuristic, Etc., have been suggested. However, due to inefficient security technologies, there is an exponential increase in the number of victims. The anonymous and uncontrollable framework of the Internet is more vulnerable to phishing attacks. Existing research works show that the performance of the phishing detection system is limited. There is a demand for an intelligent technique to protect users from the cyber-attacks. In this study, the author proposed a URL detection technique based on machine learning approaches. A recurrent neural network method is employed to detect phishing URL. Researcher evaluated the proposed method with 7900 malicious and 5800 legitimate sites, respectively. The experiments' outcome shows that the proposed method's performance is better than the recent approaches in malicious URL detection.
Extending quantum probabilistic error cancellation by noise scaling

We propose a general framework for quantum error mitigation that combines and generalizes two techniques: probabilistic error cancellation (PEC) and zero-noise extrapolation (ZNE). Similar to PEC, the proposed method represents ideal operations as linear combinations of noisy operations that are implementable on hardware. However, instead of assuming a fixed level of hardware noise, we extend the set of implementable operations by noise scaling. By construction, this method encompasses both PEC and ZNE as particular cases and allows us to investigate a larger set of hybrid techniques. For example, gate extrapolation can be used to implement PEC without requiring knowledge of the device's noise model, e.g., avoiding gate-set tomography. Alternatively, probabilistic error reduction can be used to estimate expectation values at intermediate virtual noise strengths (below the hardware level), leading to partially mitigated results at a lower sampling cost. Moreover, multiple results obtained with different noise-reduction factors can be further postprocessed with ZNE to better approximate the zero-noise limit.
TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations

Joint extraction of entities and relations from unstructured texts to form factual triples is a fundamental task of constructing a Knowledge Base (KB).A common method is to decode triples by predicting entity pairs to obtain the corresponding relation.However, it is still challenging to handle this task efficiently, especially for the overlapping triple problem.To address such a problem, this paper proposes a novel efficient entities and relations extraction model called TDEER, which stands for Translating Decoding Schema for Joint Extraction of Entities and Relations.Unlike the common approaches, the proposed translating decoding schema regards the relation as a translating operation from subject to objects, i.e., TDEER decodes triples as subject + relation → objects.TDEER can naturally handle the overlapping triple problem, because the translating decoding schema can recognize all possible triples, including overlapping and non-overlapping triples.To enhance model robustness, we introduce negative samples to alleviate error accumulation at different stages.Extensive experiments on public datasets demonstrate that TDEER produces competitive results compared with the state-of-the-art (SOTA) baselines.Furthermore, the computation complexity analysis indicates that TDEER is more efficient than powerful baselines.Especially, the proposed TDEER is 2 times faster than the recent SOTA models.
Multi-robot task allocation problem with multiple nonlinear criteria using branch and bound and genetic algorithms

Abstract The paper proposes the formulation of a single-task robot (ST), single-robot task (SR), time-extended assignment (TA), multi-robot task allocation (MRTA) problem with multiple, nonlinear criteria using discrete variables that drastically reduce the computation burden. Obtaining an allocation is addressed by a Branch and Bound (B&amp;B) algorithm in low scale problems and by a genetic algorithm (GA) specifically developed for the proposed formulation in larger scale problems. The GA crossover and mutation strategies design ensure that the descendant allocations of each generation will maintain a certain level of feasibility, reducing greatly the range of possible descendants, and accelerating their convergence to a sub-optimal allocation. The proposed MRTA algorithms are simulated and analyzed in the context of a thermosolar power plant, for which the spatially distributed Direct Normal Irradiance (DNI) is estimated using a heterogeneous fleet composed of both aerial and ground unmanned vehicles. Three optimization criteria are simultaneously considered: distance traveled, time required to complete the task and energetic feasibility. Even though this paper uses a thermosolar power plant as a case study, the proposed algorithms can be applied to any MRTA problem that uses a multi-criteria and nonlinear cost function in an equivalent way. The performance and response of the proposed algorithms are compared for four different scenarios. The results show that the B&amp;B algorithm can find the global optimal solution in a reasonable time for a case with four robots and six tasks. For larger problems, the genetic algorithm approaches the global optimal solution in much less computation time. Moreover, the trade-off between computation time and accuracy can be easily carried out by tuning the parameters of the genetic algorithm according to the available computational power.
Applications of Discrete Wavelet Transform for Feature Extraction to Increase the Accuracy of Monitoring Systems of Liquid Petroleum Products

This paper presents a methodology to monitor the liquid petroleum products which pass through transmission pipes. A simulation setup consisting of an X-ray tube, a detector, and a pipe was established using a Monte Carlo n-particle X-version transport code to investigate a two-by-two mixture of four different petroleum products, namely, ethylene glycol, crude oil, gasoline, and gasoil, in deferent volumetric ratios. After collecting the signals of each simulation, discrete wavelet transform (DWT) was applied as the feature extraction system. Then, the statistical feature, named the standard deviation, was calculated from the approximation of the fifth level, and the details of the second to fifth level provide appropriate inputs for neural network training. Three multilayer perceptron neural networks were utilized to predict the volume ratio of three types of petroleum products, and the volume ratio of the fourth product could easily be obtained from the results of the three presented networks. Finally, a root mean square error of less than 1.77 was obtained in predicting the volume ratio, which was much more accurate than in previous research. This high accuracy was due to the use of DWT for feature extraction.