Reviewerly/exHarmony huggingface dataset:
How effective is BERT without word ordering? Implications for language understanding and data privacy

Ordered word sequences contain the rich structures that define language.However, it's often not clear if or how modern pretrained language models utilize these structures.We show that the token representations and selfattention activations within BERT are surprisingly resilient to shuffling the order of input tokens, and that for several GLUE language understanding tasks, shuffling only minimally degrades performance, e.g., by 4% for QNLI.While bleak from the perspective of language understanding, our results have positive implications for cases where copyright or ethics necessitates the consideration of bag-of-words data (vs.full documents).We simulate such a scenario for three sensitive classification tasks, demonstrating minimal performance degradation vs. releasing full language sequences.
On Training Instance Selection for Few-Shot Neural Text Generation

Large-scale pretrained language models have led to dramatic improvements in text generation.Impressive performance can be achieved by finetuning only on a small number of instances (few-shot setting).Nonetheless, almost all previous work simply applies random sampling to select the few-shot training instances.Little to no attention has been paid to the selection strategies and how they would affect model performance.In this work, we present a study on training instance selection in few-shot neural text generation.The selection decision is made based only on the unlabeled data so as to identify the most worthwhile data points that should be annotated under some budget of labeling cost.Based on the intuition that the few-shot training instances should be diverse and representative of the entire data distribution, we propose a simple selection strategy with K-means clustering.We show that even with the naive clustering-based approach, the generation models consistently outperform random sampling on three text generation tasks: data-to-text generation, document summarization and question generation.The code and training data are made available at https://gitlab.com/erniecyc/few-selector.We hope that this work will call for more attention on this largely unexplored area.* Equal contribution.X.shen is now at Amazon Alexa AI.F1 1. k selected 2. few-shot generation 3. annotation budget
Integrated Directional Gradients: Feature Interaction Attribution for Neural NLP Models

In this paper, we introduce Integrated Directional Gradients (IDG), a method for attributing importance scores to groups of features, indicating their relevance to the output of a neural network model for a given input.The success of Deep Neural Networks has been attributed to their ability to capture higher level feature interactions.Hence, in the last few years capturing the importance of these feature interactions has received increased prominence in ML interpretability literature.In this paper, we formally define the feature group attribution problem and outline a set of axioms that any intuitive feature group attribution method should satisfy.Earlier, cooperative game theory inspired axiomatic methods only borrowed axioms from solution concepts (such as Shapley value) for individual feature attributions and introduced their own extensions to model interactions.In contrast, our formulation is inspired by axioms satisfied by characteristic functions as well as solution concepts in cooperative game theory literature.We believe that characteristic functions are much better suited to model importance of groups compared to just solution concepts.We demonstrate that our proposed method, IDG, satisfies all the axioms.Using IDG we analyze two state-of-the-art text classifiers on three benchmark datasets for sentiment analysis.Our experiments show that IDG is able to effectively capture semantic interactions in linguistic models via negations and conjunctions.
Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLP

Retrieval is a core component for open-domain NLP tasks.In open-domain tasks, multiple entities can share a name, making disambiguation an inherent yet under-explored problem.We propose an evaluation benchmark for assessing the entity disambiguation capabilities of these retrievers, which we call Ambiguous Entity Retrieval (AmbER) sets.We define an AmbER set as a collection of entities that share a name along with queries about those entities.By covering the set of entities for polysemous names, AmbER sets act as a challenging test of entity disambiguation.We create AmbER sets for three popular open-domain tasks: fact checking, slot filling, and question answering, and evaluate a diverse set of retrievers.We find that the retrievers exhibit popularity bias, significantly under-performing on rarer entities that share a name, e.g., they are twice as likely to retrieve erroneous documents on queries for the less popular entity under the same name.These experiments on AmbER sets show their utility as an evaluation tool and highlight the weaknesses of popular retrieval systems.
Harmonized Dense Knowledge Distillation Training for Multi-Exit Architectures

Multi-exit architectures, in which a sequence of intermediate classifiers are introduced at different depths of the feature layers, perform adaptive computation by early exiting ``easy" samples to speed up the inference. In this paper, a novel Harmonized Dense Knowledge Distillation (HDKD) training method for multi-exit architecture is designed to encourage each exit to flexibly learn from all its later exits. In particular, a general dense knowledge distillation training objective is proposed to incorporate all possible beneficial supervision information for multi-exit learning, where a harmonized weighting scheme is designed for the multi-objective optimization problem consisting of multi-exit classification loss and dense distillation loss. A bilevel optimization algorithm is introduced for alternatively updating the weights of multiple objectives and the multi-exit network parameters. Specifically, the loss weighting parameters are optimized with respect to its performance on validation set by gradient descent. Experiments on CIFAR100 and ImageNet show that the HDKD strategy harmoniously improves the performance of the state-of-the-art multi-exit neural networks. Moreover, this method does not require within architecture modifications and can be effectively combined with other previously-proposed training techniques and further boosts the performance.
A Multi-Task Learning Framework for Multi-Target Stance Detection

Multi-target stance detection aims to identify the stance taken toward a pair of different targets from the same text, and typically, there are multiple target pairs per dataset.Existing works generally train one model for each target pair.However, they fail to learn target-specific representations and are prone to overfitting.In this paper, we propose a new training strategy under the multi-task learning setting by training one model on all target pairs, which helps the model learn more universal representations and alleviate overfitting.Moreover, in order to extract more accurate target-specific representations, we propose a multi-task learning network which can jointly train our model with a stance (dis)agreement detection task that is designed to identify agreement and disagreement between stances in paired texts.Experimental results demonstrate that our proposed model outperforms the best-performing baseline by 12.39% in macro-averaged F1-score.Our resources are publicly available on GitHub. 1
Deep Just-In-Time Inconsistency Detection Between Comments and Source Code

Natural language comments convey key aspects of source code such as implementation, usage, and pre- and post-conditions. Failure to update comments accordingly when the corresponding code is modified introduces inconsistencies, which is known to lead to confusion and software bugs. In this paper, we aim to detect whether a comment becomes inconsistent as a result of changes to the corresponding body of code, in order to catch potential inconsistencies just-in-time, i.e., before they are committed to a code base. To achieve this, we develop a deep-learning approach that learns to correlate a comment with code changes. By evaluating on a large corpus of comment/code pairs spanning various comment types, we show that our model outperforms multiple baselines by significant margins. For extrinsic evaluation, we show the usefulness of our approach by combining it with a comment update model to build a more comprehensive automatic comment maintenance system which can both detect and resolve inconsistent comments based on code changes.
Total Generate: Cycle in Cycle Generative Adversarial Networks for Generating Human Faces, Hands, Bodies, and Natural Scenes

We propose a novel and unified Cycle in Cycle Generative Adversarial Network (C2GAN) for generating human faces, hands, bodies, and natural scenes.Our proposed C2GAN is a cross-modal model exploring the joint exploitation of the input image data and guidance data in an interactive manner.C2GAN contains two different generators, i.e., an imagegeneration generator and a guidance-generation generator.Both generators are mutually connected and trained in an end-toend fashion and explicitly form three cycled subnets, i.e., one image generation cycle and two guidance generation cycles.Each cycle aims at reconstructing the input domain and simultaneously produces a useful output involved in the generation of another cycle.In this way, the cycles constrain each other implicitly providing complementary information from both image and guidance modalities and bringing an extra supervision gradient across the cycles, facilitating a more robust optimization of the whole model.Extensive results on four guided image-to-image translation subtasks demonstrate that the proposed C2GAN is effective in generating more realistic images compared with stateof-the-art models 1 .
3D Reconstruction with Single-Shot Structured Light RGB Line Pattern

Single-shot 3D reconstruction technique is very important for measuring moving and deforming objects. After many decades of study, a great number of interesting single-shot techniques have been proposed, yet the problem remains open. In this paper, a new approach is proposed to reconstruct deforming and moving objects with the structured light RGB line pattern. The structured light RGB line pattern is coded using parallel red, green, and blue lines with equal intervals to facilitate line segmentation and line indexing. A slope difference distribution (SDD)-based image segmentation method is proposed to segment the lines robustly in the HSV color space. A method of exclusion is proposed to index the red lines, the green lines, and the blue lines respectively and robustly. The indexed lines in different colors are fused to obtain a phase map for 3D depth calculation. The quantitative accuracies of measuring a calibration grid and a ball achieved by the proposed approach are 0.46 and 0.24 mm, respectively, which are significantly lower than those achieved by the compared state-of-the-art single-shot techniques.
Deep Unfolding With Normalizing Flow Priors for Inverse Problems

Many application domains, spanning from computational photography to medical imaging, require recovery of high-fidelity images from noisy, incomplete or partial/compressed measurements. State of the art methods for solving these inverse problems combine deep learning with iterative model-based solvers, a concept known as deep algorithm unfolding. By combining a-priori knowledge of the forward measurement model with learned (proximal) mappings based on deep networks, these methods yield solutions that are both physically feasible (data-consistent) and perceptually plausible. However, current proximal mappings only implicitly learn such image priors. In this paper, we propose to make these image priors fully explicit by embedding deep generative models in the form of normalizing flows within the unfolded proximal gradient algorithm. We demonstrate that the proposed method outperforms competitive baselines on various image recovery tasks, spanning from image denoising to inpainting and deblurring.
Semi-supervised Object Detection with Adaptive Class-Rebalancing Self-Training

While self-training achieves state-of-the-art results in semi-supervised object detection (SSOD), it severely suffers from foreground-background and foreground-foreground imbalances in SSOD. In this paper, we propose an Adaptive Class-Rebalancing Self-Training (ACRST) with a novel memory module called CropBank to alleviate these imbalances and generate unbiased pseudo-labels. Besides, we observe that both self-training and data-rebalancing procedures suffer from noisy pseudo-labels in SSOD. Therefore, we contribute a simple yet effective two-stage pseudo-label filtering scheme to obtain accurate supervision. Our method achieves competitive performance on MS-COCO and VOC benchmarks. When using only 1% labeled data of MS-COCO, our method achieves 17.02 mAP improvement over the supervised method and 5.32 mAP gains compared with state-of-the-arts.
Comparing Local and Central Differential Privacy Using Membership Inference Attacks

Attacks that aim to identify the training data of neural networks represent a severe threat to the privacy of individuals in the training dataset. A possible protection is offered by anonymization of the training data or training function with differential privacy. However, data scientists can choose between local and central differential privacy, and need to select meaningful privacy parameters $$\epsilon $$ . A comparison of local and central differential privacy based on the privacy parameters furthermore potentially leads data scientists to incorrect conclusions, since the privacy parameters are reflecting different types of mechanisms. Instead, we empirically compare the relative privacy-accuracy trade-off of one central and two local differential privacy mechanisms under a white-box membership inference attack. While membership inference only reflects a lower bound on inference risk and differential privacy formulates an upper bound, our experiments with several datasets show that the privacy-accuracy trade-off is similar for both types of mechanisms despite the large difference in their upper bound. This suggests that the upper bound is far from the practical susceptibility to membership inference. Thus, small $$\epsilon $$ in central differential privacy and large $$\epsilon $$ in local differential privacy result in similar membership inference risks, and local differential privacy can be a meaningful alternative to central differential privacy for differentially private deep learning besides the comparatively higher privacy parameters.
Exploring the Blockchain Technology: Issues, Applications and Research Potential

The blockchain is revolutionizing the current IT industry by increasing its integration with other prominent technologies like Artificial Intelligence, Internet of Things, Big Data and Cloud Computing to name a few. It works as a distributed network with no central authority, where the data is continuously added in the form of blocks. The blocks are validated by the network itself, which makes it transparent and secure. It was first used as a ledger for the transactions of a crypto currency named bitcoin, but with time, innumerable industries have started to implement it. And because of its promising future, it would not take much time to spread in all the IT related areas globally. The blockchain technology is contemplated to be a technology that is still in its infancy and needs a thorough research so that it can be utilized to its full capacity. Consequently, in order to provide a thorough analysis, this paper gives a systematic and comprehensive review of architecture and working of the blockchain along with its types and important characteristics. This paper also discusses about the prominent blockchain platforms available today along with the description of diverse blockchain supported application areas. The issues related to the blockchain technology and the areas in which the work can be done in future are also pointed in this paper, which would be helpful for practitioners and researchers who are willing to work on this fast growing platform.
Barriers and dynamical paths in alternating Gibbs sampling of restricted Boltzmann machines

Restricted Boltzmann machines (RBM) are bilayer neural networks used for the unsupervised learning of model distributions from data. The bipartite architecture of RBM naturally defines an elegant sampling procedure, called alternating Gibbs sampling (AGS), where the configurations of the latent-variable layer are sampled conditional to the data-variable layer and vice versa. We study here the performance of AGS on several analytically tractable models borrowed from statistical mechanics. We show that standard AGS is not more efficient than classical Metropolis-Hastings (MH) sampling of the effective energy landscape defined on the data layer. However, RBM can identify meaningful representations of training data in their latent space. Furthermore, using these representations and combining Gibbs sampling with the MH algorithm in the latent space can enhance the sampling performance of the RBM when the hidden units encode weakly dependent features of the data. We illustrate our findings on three datasets: Bars and Stripes and MNIST, well known in machine learning, and the so-called lattice proteins dataset, introduced in theoretical biology to study the sequence-to-structure mapping in proteins.
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning

Deep learning (DL) based language models achieve high performance on various benchmarks for Natural Language Inference (NLI).And at this time, symbolic approaches to NLI are receiving less attention.Both approaches (symbolic and DL) have their advantages and weaknesses.However, currently, no method combines them in a system to solve the task of NLI.To merge symbolic and deep learning methods, we propose an inference framework called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment.Our framework models the NLI task as a classic search problem and uses the beam search algorithm to search for optimal inference paths.Experiments show that our joint logic and neural inference system improves accuracy on the NLI task and can achieve state-of-art accuracy on the SICK and MED datasets.
SemEval-2021 Task 8: MeasEval – Extracting Counts and Measurements and their Related Contexts

We describe MeasEval, a SemEval task of extracting counts, measurements, and related context from scientific documents, which is of significant importance to the creation of Knowledge Graphs that distill information from the scientific literature.This is a new task in 2021, for which over 75 submissions from 25 participants were received.We expect the data developed for this task and the findings reported to be valuable to the scientific knowledge extraction, metrology, and automated knowledge base construction communities.
Unifying and benchmarking state-of-the-art quantum error mitigation techniques

Error mitigation is an essential component of achieving a practical quantum advantage in the near term, and a number of different approaches have been proposed. In this work, we recognize that many state-of-the-art error mitigation methods share a common feature: they are data-driven, employing classical data obtained from runs of different quantum circuits. For example, Zero-noise extrapolation (ZNE) uses variable noise data and Clifford-data regression (CDR) uses data from near-Clifford circuits. We show that Virtual Distillation (VD) can be viewed in a similar manner by considering classical data produced from different numbers of state preparations. Observing this fact allows us to unify these three methods under a general data-driven error mitigation framework that we call UNIfied Technique for Error mitigation with Data (UNITED). In certain situations, we find that our UNITED method can outperform the individual methods (i.e., the whole is better than the individual parts). Specifically, we employ a realistic noise model obtained from a trapped ion quantum computer to benchmark UNITED, as well as other state-of-the-art methods, in mitigating observables produced from random quantum circuits and the Quantum Alternating Operator Ansatz (QAOA) applied to Max-Cut problems with various numbers of qubits, circuit depths and total numbers of shots. We find that the performance of different techniques depends strongly on shot budgets, with more powerful methods requiring more shots for optimal performance. For our largest considered shot budget (<mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mn>10</mml:mn><mml:mrow class="MJX-TeXAtom-ORD"><mml:mn>10</mml:mn></mml:mrow></mml:msup></mml:math>), we find that UNITED gives the most accurate mitigation. Hence, our work represents a benchmarking of current error mitigation methods and provides a guide for the regimes when certain methods are most useful.
Mobile Robot Path Planning in a Trajectory with Multiple Obstacles Using Genetic Algorithms

Path planning is an essential algorithm to help robots complete their task in the field quickly. However, some path planning algorithms are computationally expensive and cannot adapt to new environments with a distinctly different set of obstacles. This paper presents optimal path planning based on a genetic algorithm (GA) that is proposed to be carried out in a dynamic environment with various obstacles. First, the points of the feasible path are found by performing a local search procedure. Then, the points are optimized to find the shortest path. When the optimal path is calculated, the position of the points on the path is smoothed to avoid obstacles in the environment. Thus, the average fitness values and the GA generation are better than the traditional method. The simulation results show that the proposed algorithm successfully finds the optimal path in an environment with multiple obstacles. Compared to a traditional GA-based method, our proposed algorithm has a smoother route due to path optimization. Therefore, this makes the proposed method advantageous in a dynamic environment.