Reviewerly/exHarmony huggingface dataset:
QPP: Real-Time Quantization Parameter Prediction for Deep Neural Networks

Modern deep neural networks (DNNs) cannot be effectively used in mobile and embedded devices due to strict requirements for computational complexity, memory, and power consumption. The quantization of weights and feature maps (activations) is a popular approach to solve this problem. Training-aware quantization often shows excellent results but requires a full dataset, which is not always available. Post-training quantization methods, in turn, are applied without fine-tuning but still work well for many classes of tasks like classification, segmentation, and so on. However, they either imply a big overhead for quantization parameters (QPs) calculation at runtime (dynamic methods) or lead to an accuracy drop if pre-computed static QPs are used (static methods). Moreover, most inference frame-works don’t support dynamic quantization. Thus we propose a novel quantization approach called QPP: quantization parameter prediction. With a small subset of a training dataset or unlabeled data from the same domain, we find the predictor that can accurately estimate QPs of activations given only the NN’s input data. Such a predictor allows us to avoid complex calculation of precise values of QPs while maintaining the quality of the model. To illustrate our method’s efficiency, we added QPP into two dynamic approaches: 1) Dense+Sparse quantization, where the predetermined percentage of activations are not quantized, 2) standard quantization with equal quantization steps. We provide experiments on a wide set of tasks including super-resolution, facial landmark, segmentation, and classification.
Spirals of Sustainable Academic Motivation, Creativity, and Trust of Higher Education Staff

Sustainability has to penetrate more and more into higher education. It should not focus only on traditional elements. It should also enter new, but for future improvement, extremely important areas. Based on this premise, creativity and motivation, when additionally interconnected and supported by trust that is provided and achieved, decide on the progress and sustainability of universities. This connection is gaining importance especially from the point of view of building solid foundations and mechanisms that functionally preserve the potential effects of these elements in the future. For this reason and following the nature, importance, and content of sustainable academic motivation (SAM), the paper introduces two new concepts: sustainable academic creativity (SAC) and sustainable academic trust (SAT). For further original contributions, the paper hypothesizes the existence of mutual—spiral—relations of sustainable academic motivation (SAM), sustainable academic creativity (SAC), and sustainable academic trust (SAT). The empirical section tests the validity of this claim in the universities of two countries: the Slovak Republic and Poland. A survey performed on a sample of n=181 pedagogical, scientific, management, and administrative staff in higher education confirms the existence of these spirals. The results indicate the spiral effect of motivation when connected with creativity and trust and show that it is accented by the crucial principles of sustainability (responsibility, novelty, usefulness, progress, etc.). Therefore, the paper’s conclusion contains the explanations for the potential occurrence of three types of sustainably mutual systems and complexes. These are: (a) individual sustainable systems of SAM, SAC, and SAT; (b) group/sectional sustainable systems of SAM, SAC, and SAT; and (c) the global sustainable complex of SAM, SAC, and SAT in the university.
Examining the Inductive Bias of Neural Language Models with Artificial Languages

Since language models are used to model a wide variety of languages, it is natural to ask whether the neural architectures used for the task have inductive biases towards modeling particular types of languages.Investigation of these biases has proved complicated due to the many variables that appear in the experimental setup.Languages vary in many typological dimensions, and it is difficult to single out one or two to investigate without the others acting as confounders.We propose a novel method for investigating the inductive biases of language models using artificial languages.These languages are constructed to allow us to create parallel corpora across languages that differ only in the typological feature being investigated, such as word order.We then use them to train and test language models.This constitutes a fully controlled causal framework, and demonstrates how grammar engineering can serve as a useful tool for analyzing neural models.Using this method, we find that commonly used neural architectures exhibit different inductive biases: LSTMs display little preference with respect to word ordering, while transformers display a clear preference for some orderings over others.Further, we find that neither the inductive bias of the LSTM nor that of the transformer appears to reflect any tendencies that we see in attested natural languages.
Generalising without Forgetting for Lifelong Person Re-Identification

Existing person re-identification (Re-ID) methods mostly prepare all training data in advance, while real-world Re-ID data are inherently captured over time or from different locations, which requires a model to be incrementally generalised from sequential learning of piecemeal new data without forgetting what is already learned. In this work, we call this lifelong person Re-ID, characterised by solving a problem of unseen class identification subject to continuous new domain generalisation and adaptation with class imbalanced learning. We formulate a new Generalising without Forgetting method (GwFReID) for lifelong Re-ID and design a comprehensive learning objective that accounts for classification coherence, distribution coherence and representation coherence in a unified framework. This design helps to simultaneously learn new information, distil old knowledge and solve class imbalance, which enables GwFReID to incrementally improve model generalisation without catastrophic forgetting of what is already learned. Extensive experiments on eight Re-ID benchmarks, CIFAR-100 and ImageNet show the superiority of GwFReID over the state-of-the-art methods.
Decoupled Dialogue Modeling and Semantic Parsing for Multi-Turn Text-to-SQL

Recently, Text-to-SQL for multi-turn dialogue has attracted great interest.Here, the user input of the current turn is parsed into the corresponding SQL query of the appropriate database, given all previous dialogue history.Current approaches mostly employ end-to-end models and consequently face two challenges.First, dialogue history modeling and Text-to-SQL parsing are implicitly combined, hence it is hard to carry out interpretable analysis and obtain targeted improvement.Second, SQL annotation of multi-turn dialogue is very expensive, leading to training data sparsity.In this paper, we propose a novel decoupled multi-turn Text-to-SQL framework, where an utterance rewrite model first explicitly solves completion of dialogue context, and then a single-turn Text-to-SQL parser follows.A dual learning approach is also proposed for the utterance rewrite model to address the data sparsity problem.Compared with end-to-end approaches, the proposed decoupled method can achieve excellent performance without any annotated in-domain data.With just a few annotated rewrite cases, the decoupled method outperforms the released state-of-the-art endto-end models on both SParC and CoSQL datasets.
MLMLM: Link Prediction with Mean Likelihood Masked Language Model

Knowledge Bases (KBs) are easy to query, verifiable, and interpretable.They however scale with man-hours and high-quality data.Masked Language Models (MLMs), such as BERT, scale with computing power as well as unstructured raw text data.The knowledge contained within these models is however not directly interpretable.We propose to perform link prediction with MLMs to address both the KBs scalability issues and the MLMs interpretability issues.By committing the knowledge embedded in MLMs to a KB, it becomes interpretable.To do that we introduce MLMLM, Mean Likelihood Masked Language Model, an approach comparing the mean likelihood of generating the different entities to perform link prediction in a tractable manner.We obtain State of the Art (SotA) results on the WN18RR dataset and SotA results on the Precision@1 metric on the WikidataM5 inductive and transductive setting.We also obtain convincing results on link prediction on previously unseen entities, making MLMLM a suitable approach to introducing new entities to a KB.
Language Tags Matter for Zero-Shot Neural Machine Translation

Multilingual Neural Machine Translation (MNMT) has aroused widespread interest due to its efficiency.An exciting advantage of MNMT models is that they could also translate between unsupervised (zero-shot) language directions.Language tag (LT) strategies are often adopted to indicate the translation directions in MNMT.In this paper, we demonstrate that the LTs are not only indicators for translation directions but also crucial to zero-shot translation qualities.Unfortunately, previous work tends to ignore the importance of LT strategies.We demonstrate that a proper LT strategy could enhance the consistency of semantic representations and alleviate the off-target issue in zero-shot directions.Experimental results show that by ignoring the source language tag (SLT) and adding the target language tag (TLT) to the encoder, the zero-shot translations could achieve a +8 BLEU score difference over other LT strategies in IWSLT17, Europarl, TED talks translation tasks.
Rejection Sampling for Weighted Jaccard Similarity Revisited

Efficiently computing the weighted Jaccard similarity has become an active research topic in machine learning and theory. For sparse data, the standard technique is based on the consistent weighed sampling (CWS). For dense data, however, methods based on rejection sampling (RS) can be much more efficient. Nevertheless, existing RS methods are still slow for practical purposes. In this paper, we propose to improve RS by a strategy, which we call efficient rejection sampling (ERS), based on ``early stopping + densification''. We analyze the statistical property of ERS and provide experimental results to compare ERS with RS and other algorithms for hashing weighted Jaccard. The results demonstrate that ERS significantly improves the existing methods for estimating the weighted Jaccard similarity in relatively dense data.
Element Intervention for Open Relation Extraction

Open relation extraction aims to cluster relation instances referring to the same underlying relation, which is a critical step for general relation extraction.Current OpenRE models are commonly trained on the datasets generated from distant supervision, which often results in instability and makes the model easily collapsed.In this paper, we revisit the procedure of OpenRE from a causal view.By formulating OpenRE using a structural causal model, we identify that the above-mentioned problems stem from the spurious correlations from entities and context to the relation type.To address this issue, we conduct Element Intervention, which intervenes on the context and entities respectively to obtain the underlying causal effects of them.We also provide two specific implementations of the interventions based on entity ranking and context contrasting.Experimental results on unsupervised relation extraction datasets show that our methods outperform previous state-of-the-art methods and are robust across different datasets 1 .
Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification

Fact verification is a challenging task that requires simultaneously reasoning and aggregating over multiple retrieved pieces of evidence to evaluate the truthfulness of a claim.Existing approaches typically (i) explore the semantic interaction between the claim and evidence at different granularity levels but fail to capture their topical consistency during the reasoning process, which we believe is crucial for verification; (ii) aggregate multiple pieces of evidence equally without considering their implicit stances to the claim, thereby introducing spurious information.To alleviate the above issues, we propose a novel topicaware evidence reasoning and stance-aware aggregation model for more accurate fact verification, with the following four key properties: 1) checking topical consistency between the claim and evidence; 2) maintaining topical coherence among multiple pieces of evidence; 3) ensuring semantic similarity between the global topic information and the semantic representation of evidence; 4) aggregating evidence based on their implicit stances to the claim.Extensive experiments conducted on the two benchmark datasets demonstrate the superiority of the proposed model over several state-of-the-art approaches for fact verification.
Hypothesis Disparity Regularized Mutual Information Maximization

We propose a hypothesis disparity regularized mutual information maximization (HDMI) approach to tackle unsupervised hypothesis transfer---as an effort towards unifying hypothesis transfer learning (HTL) and unsupervised domain adaptation (UDA)---where the knowledge from a source domain is transferred solely through hypotheses and adapted to the target domain in an unsupervised manner. In contrast to the prevalent HTL and UDA approaches that typically use a single hypothesis, HDMI employs multiple hypotheses to leverage the underlying distributions of the source and target hypotheses. To better utilize the crucial relationship among different hypotheses---as opposed to unconstrained optimization of each hypothesis independently---while adapting to the unlabeled target domain through mutual information maximization, HDMI incorporates a hypothesis disparity regularization that coordinates the target hypotheses jointly learn better target representations while preserving more transferable source knowledge with better-calibrated prediction uncertainty. HDMI achieves state-of-the-art adaptation performance on benchmark datasets for UDA in the context of HTL, without the need to access the source data during the adaptation.
Storage Fit Learning with Feature Evolvable Streams

Feature evolvable learning has been widely studied in recent years where old features will vanish and new features will emerge when learning with streams. Conventional methods usually assume that a label will be revealed after prediction at each time step. However, in practice, this assumption may not hold whereas no label will be given at most time steps. A good solution is to leverage the technique of manifold regularization to utilize the previous similar data to assist the refinement of the online model. Nevertheless, this approach needs to store all previous data which is impossible in learning with streams that arrive sequentially in large volume. Thus we need a buffer to store part of them. Considering that different devices may have different storage budgets, the learning approaches should be flexible subject to the storage budget limit. In this paper, we propose a new setting: Storage-Fit Feature-Evolvable streaming Learning (SF2EL) which incorporates the issue of rarely-provided labels into feature evolution. Our framework is able to fit its behavior for different storage budgets when learning with feature evolvable streams with unlabeled data. Besides, both theoretical and empirical results validate that our approach can preserve the merit of the original feature evolvable learning i.e., can always track the best baseline and thus perform well at any time step.
DABT: A Dependency-aware Bug Triaging Method

In software engineering practice, fixing a bug promptly reduces the associated costs. On the other hand, the manual bug fixing process can be time-consuming, cumbersome, and error-prone. In this work, we introduce a bug triaging method, called Dependency-aware Bug Triaging (DABT), which leverages natural language processing and integer programming to assign bugs to appropriate developers. Unlike previous works that mainly focus on one aspect of the bug reports, DABT considers the textual information, cost associated with each bug, and dependency among them. Therefore, this comprehensive formulation covers the most important aspect of the previous works while considering the blocking effect of the bugs. We report the performance of the algorithm on three open-source software systems, i.e., EclipseJDT, LibreOffice, and Mozilla. Our result shows that DABT is able to reduce the number of overdue bugs up to 12\%. It also decreases the average fixing time of the bugs by half. Moreover, it reduces the complexity of the bug dependency graph by prioritizing blocking bugs.
Controllable Open-ended Question Generation with A New Question Type Ontology

We investigate the less-explored task of generating open-ended questions that are typically answered by multiple sentences.We first define a new question type ontology which differentiates the nuanced nature of questions better than widely used question words.A new dataset with 4, 959 questions is labeled based on the new ontology.We then propose a novel question type-aware question generation framework, augmented by a semantic graph representation, to jointly predict question focuses and produce the question.Based on this framework, we further use both exemplars and automatically generated templates to improve controllability and diversity.Experiments on two newly collected large-scale datasets show that our model improves question quality over competitive comparisons based on automatic metrics.Human judges also rate our model outputs highly in answerability, coverage of scope, and overall quality.Finally, our model variants with templates can produce questions with enhanced controllability and diversity.
Evo-RL

In this work, we propose a novel approach for reinforcement learning driven by evolutionary computation. Our algorithm, dubbed as Evolutionary-Driven Reinforcement Learning (Evo-RL), embeds the reinforcement learning algorithm in an evolutionary cycle, where we distinctly differentiate between purely evolvable (instinctive) behaviour versus purely learnable behaviour. Furthermore, we propose that this distinction is decided by the evolutionary process, thus allowing Evo-RL to be adaptive to different environments. In addition, Evo-RL facilitates learning on environments with reward-less states, which makes it more suited for real-world problems with incomplete information. To show that Evo-RL leads to state-of-the-art performance, we present the performance of different state-of-the-art reinforcement learning algorithms when operating within Evo-RL and compare it with the case when these same algorithms are executed independently. Results show that reinforcement learning algorithms embedded within our Evo-RL approach significantly outperform the stand-alone versions of the same RL algorithms on OpenAI Gym control problems with rewardless states constrained by the same computational budget.
Fock state-enhanced expressivity of quantum machine learning models

Abstract The data-embedding process is one of the bottlenecks of quantum machine learning, potentially negating any quantum speedups. In light of this, more effective data-encoding strategies are necessary. We propose a photonic-based bosonic data-encoding scheme that embeds classical data points using fewer encoding layers and circumventing the need for nonlinear optical components by mapping the data points into the high-dimensional Fock space. The expressive power of the circuit can be controlled via the number of input photons. Our work sheds some light on the unique advantages offered by quantum photonics on the expressive power of quantum machine learning models. By leveraging the photon-number dependent expressive power, we propose three different noisy intermediate-scale quantum-compatible binary classification methods with different scaling of required resources suitable for different supervised classification tasks.
Balancer Genetic Algorithm—A Novel Task Scheduling Optimization Approach in Cloud Computing

Task scheduling is one of the core issues in cloud computing. Tasks are heterogeneous, and they have intensive computational requirements. Tasks need to be scheduled on Virtual Machines (VMs), which are resources in a cloud environment. Due to the immensity of search space for possible mappings of tasks to VMs, meta-heuristics are introduced for task scheduling. In scheduling makespan and load balancing, Quality of Service (QoS) parameters are crucial. This research contributes a novel load balancing scheduler, namely Balancer Genetic Algorithm (BGA), which is presented to improve makespan and load balancing. Insufficient load balancing can cause an overhead of utilization of resources, as some of the resources remain idle. BGA inculcates a load balancing mechanism, where the actual load in terms of million instructions assigned to VMs is considered. A need to opt for multi-objective optimization for improvement in load balancing and makespan is also emphasized. Skewed, normal and uniform distributions of workload and different batch sizes are used in experimentation. BGA has exhibited significant improvement compared with various state-of-the-art approaches for makespan, throughput and load balancing.
Quantum machine learning with adaptive linear optics

We study supervised learning algorithms in which a quantum device is used to perform a computational subroutine - either for prediction via probability estimation, or to compute a kernel via estimation of quantum states overlap. We design implementations of these quantum subroutines using Boson Sampling architectures in linear optics, supplemented by adaptive measurements. We then challenge these quantum algorithms by deriving classical simulation algorithms for the tasks of output probability estimation and overlap estimation. We obtain different classical simulability regimes for these two computational tasks in terms of the number of adaptive measurements and input photons. In both cases, our results set explicit limits to the range of parameters for which a quantum advantage can be envisaged with adaptive linear optics compared to classical machine learning algorithms: we show that the number of input photons and the number of adaptive measurements cannot be simultaneously small compared to the number of modes. Interestingly, our analysis leaves open the possibility of a near-term quantum advantage with a single adaptive measurement.