Reviewerly/exHarmony huggingface dataset:
Enhanced Back-Translation for Low Resource Neural Machine Translation Using Self-training

Improving neural machine translation (NMT) models using the back-translations of the monolingual target data (synthetic parallel data) is currently the state-of-the-art approach for training improved translation systems. The quality of the backward system - which is trained on the available parallel data and used for the back-translation - has been shown in many studies to affect the performance of the final NMT model. In low resource conditions, the available parallel data is usually not enough to train a backward model that can produce the qualitative synthetic data needed to train a standard translation model. This work proposes a self-training strategy where the output of the backward model is used to improve the model itself through the forward translation technique. The technique was shown to improve baseline low resource IWSLT'14 English-German and IWSLT'15 English-Vietnamese backward translation models by 11.06 and 1.5 BLEUs respectively. The synthetic data generated by the improved English-German backward model was used to train a forward model which out-performed another forward model trained using standard back-translation by 2.7 BLEU.
Making sense of raw input

How should a machine intelligence perform unsupervised structure discovery over streams of sensory input? One approach to this problem is to cast it as an apperception task [1]. Here, the task is to construct an explicit interpretable theory that both explains the sensory sequence and also satisfies a set of unity conditions, designed to ensure that the constituents of the theory are connected in a relational structure. However, the original formulation of the apperception task had one fundamental limitation: it assumed the raw sensory input had already been parsed using a set of discrete categories, so that all the system had to do was receive this already-digested symbolic input, and make sense of it. But what if we don't have access to pre-parsed input? What if our sensory sequence is raw unprocessed information? The central contribution of this paper is a neuro-symbolic framework for distilling interpretable theories out of streams of raw, unprocessed sensory experience. First, we extend the definition of the apperception task to include ambiguous (but still symbolic) input: sequences of sets of disjunctions. Next, we use a neural network to map raw sensory input to disjunctive input. Our binary neural network is encoded as a logic program, so the weights of the network and the rules of the theory can be solved jointly as a single SAT problem. This way, we are able to jointly learn how to perceive (mapping raw sensory information to concepts) and apperceive (combining concepts into declarative rules).
Privacy-Oriented Technique for COVID-19 Contact Tracing (PROTECT) Using Homomorphic Encryption: Design and Development Study

Background Various techniques are used to support contact tracing, which has been shown to be highly effective against the COVID-19 pandemic. To apply the technology, either quarantine authorities should provide the location history of patients with COVID-19, or all users should provide their own location history. This inevitably exposes either the patient’s location history or the personal location history of other users. Thus, a privacy issue arises where the public good (via information release) comes in conflict with privacy exposure risks. Objective The objective of this study is to develop an effective contact tracing system that does not expose the location information of the patient with COVID-19 to other users of the system, or the location information of the users to the quarantine authorities. Methods We propose a new protocol called PRivacy Oriented Technique for Epidemic Contact Tracing (PROTECT) that securely shares location information of patients with users by using the Brakerski/Fan-Vercauteren homomorphic encryption scheme, along with a new, secure proximity computation method. Results We developed a mobile app for the end-user and a web service for the quarantine authorities by applying the proposed method, and we verified their effectiveness. The proposed app and web service compute the existence of intersections between the encrypted location history of patients with COVID-19 released by the quarantine authorities and that of the user saved on the user’s local device. We also found that this contact tracing smartphone app can identify whether the user has been in contact with such patients within a reasonable time. Conclusions This newly developed method for contact tracing shares location information by using homomorphic encryption, without exposing the location information of patients with COVID-19 and other users. Homomorphic encryption is challenging to apply to practical issues despite its high security value. In this study, however, we have designed a system using the Brakerski/Fan-Vercauteren scheme that is applicable to a reasonable size and developed it to an operable format. The developed app and web service can help contact tracing for not only the COVID-19 pandemic but also other epidemics.
Multi-robot Coordination and Planning in Uncertain and Adversarial Environments

Deploying a team of robots that can carefully coordinate their actions can make the entire system robust to individual failures. In this report, we review recent algorithmic development in making multi-robot systems robust to environmental uncertainties, failures, and adversarial attacks. We find the following three trends in the recent research in the area of multi-robot coordination: (1) resilient coordination to either withstand failures and/or attack or recover from failures/attacks; (2) risk-aware coordination to manage the trade-off risk and reward, where the risk stems due to environmental uncertainty; (3) Graph Neural Networks based coordination to learn decentralized multi-robot coordination policies. These algorithms have been applied to tasks such as formation control, task assignment and scheduling, search and planning, and informative data collection. In order for multi-robot systems to become practical, we need coordination algorithms that can scale to large teams of robots dealing with dynamically changing, failure-prone, contested, and uncertain environments. There has been significant recent research on multi-robot coordination that has contributed resilient and risk-aware algorithms to deal with these issues and reduce the gap between theory and practice. Learning-based approaches have been seen to be promising, especially since they can learn who, when, and how to communicate for effective coordination. However, these algorithms have also been shown to be vulnerable to adversarial attacks, and as such developing learning-based coordination strategies that are resilient to such attacks and robust to uncertainties is an important open area of research.
A Distributed Intrusion Detection Approach Based on Machine Leaning Techniques for a Cloud Security

Nowadays, the rapid evolution of computer network and large amount of data which are generated by systems require an accurate monitoring and high security of data and resources. Thereby, the security task of personal information is highly requested. The cloud computing technology allows users and organizations several capabilities to store and process data in a privately owned cloud or on a third party server to make data accessing mechanisms more relevant. Then, it is useful to monitor and protect data housed on the cloud by detecting and stopping intrusive activities and attacks. For this reason, many security tools are available to monitor transactions in cloud and have been taken to prevent networks from intrusions. It is also very useful to choose a cloud provider that considers the security and protection like an essential goal. The intrusion detection and prevention is a recent technology used to secure data in cloud computing environments. In this paper, our main is to propose and design a new distributed intrusion detection system as a service to improve a cloud security. We describe all solutions that we suggest to validate various parts of proposed system for detecting intrusions and defending against attacks in cloud.
DA-Transformer: Distance-aware Transformer

Transformer has achieved great success in the NLP field by composing various advanced models like BERT and GPT.However, Transformer and its existing variants may not be optimal in capturing token distances because the position or distance embeddings used by these methods usually cannot keep the precise information of real distances, which may not be beneficial for modeling the orders and relations of contexts.In this paper, we propose DA-Transformer, which is a distanceaware Transformer that can exploit the real distance.We propose to incorporate the real distances between tokens to re-scale the raw self-attention weights, which are computed by the relevance between attention query and key.Concretely, in different self-attention heads the relative distance between each pair of tokens is weighted by different learnable parameters, which control the different preferences on long-or short-term information of these heads.Since the raw weighted real distances may not be optimal for adjusting selfattention weights, we propose a learnable sigmoid function to map them into re-scaled coefficients that have proper ranges.We first clip the raw self-attention weights via the ReLU function to keep non-negativity and introduce sparsity, and then multiply them with the rescaled coefficients to encode real distance information into self-attention.Extensive experiments on five benchmark datasets show that DA-Transformer can effectively improve the performance of many tasks and outperform the vanilla Transformer and its several variants.
Rule-enhanced iterative complementation for knowledge graph reasoning

Knowledge graph (KG) reasoning aims to infer missing valid triples from observed triples, thereby improving the semantics of the whole KG. The general KG reasoning involves rule-based and embedding-based methods. The former can provide an interpretable reasoning process but has low efficiency, while the latter is the converse. Therefore, some hybrid methods have been proposed, but there are still two challenges: the completeness of rule learning and the determination of hidden triples. To address these challenges, this paper proposes a rule-enhanced iterative complementation (Rule-IC) method, which involves three components: rule learning, an embedding learner and a triple discriminator. Such an iterative process enriches the semantics of KG and further increases the completeness of rule learning to generate hidden triples. In order to precisely determine the validity of hidden triples, a multi-relational graph convolutional network (GCN) with attentive message passing is introduced as a triple discriminator. The embedding learner for KG reasoning and the GCN discriminator complement each other by valid hidden triples. In addition, the performance of these three components improves overall during the iterative process. Experimental results show that most evaluation metrics of Rule-IC are better than those of several baselines on four common KGs. Furthermore, it is scalable and can be extended to all KG embedding models theoretically.
Blockchain in Indian Public Distribution System: a conceptual framework to prevent leakage of the supplies and its enablers and disablers

Purpose The purpose of this paper is to propose a conceptual framework for the application of blockchain in the Public Distribution System (PDS) in India to manage the supply of food grains to the targeted beneficiaries. The framework will help prevent diversions and leakages of grains at the warehouse and Fair Price Shop (FPS) level. The paper also identifies the enablers and disablers in the context of the framework. Design/methodology/approach This paper will firstly review the previous literature in PDS and blockchain-enabled agricultural and food supply chains. The study then proposes a framework that could be implemented in the PDS in India using blockchain technology. Findings The proposed framework provides an effective way to combat corruption, exclusion errors of targeted beneficiaries, leakage of PDS food grains and is cost-effective. The identified enablers and disablers give an insight into the application of blockchain in PDS in India. Research limitations/implications The research work may have implications for the Ministry of Food and PDS (Central Government), Food Corporation of India and State Governments to manage the supply of the grains more efficiently and effectively. Originality/value The current study caters to the implementation of blockchain technology starting from the warehouse level to the FPSs and consumers and simultaneously connecting them to concerned authorities to ensure transparency and accountability.
Learn to Resolve Conversational Dependency: A Consistency Training Framework for Conversational Question Answering

One of the main challenges in conversational question answering (CQA) is to resolve the conversational dependency, such as anaphora and ellipsis.However, existing approaches do not explicitly train QA models on how to resolve the dependency, and thus these models are limited in understanding human dialogues.In this paper, we propose a novel framework, EXCORD (Explicit guidance on how to resolve Conversational Dependency) to enhance the abilities of QA models in comprehending conversational context.EXCORD first generates self-contained questions that can be understood without the conversation history, then trains a QA model with the pairs of original and self-contained questions using a consistency-based regularizer.In our experiments, we demonstrate that EXCORD significantly improves the QA models' performance by up to 1.2 F1 on QuAC (Choi et al., 2018), and 5.2 F1 on CANARD (Elgohary et al., 2019), while addressing the limitations of the existing approaches.
On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study

In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions.Researchers hope that models trained on these more challenging datasets will rely less on superficial patterns, and thus be less brittle.However, despite ADC's intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models.In this paper, we conduct a large-scale controlled study focused on question answering, assigning workers at random to compose questions either (i) adversarially (with a model in the loop); or (ii) in the standard fashion (without a model).Across a variety of models and datasets, we find that models trained on adversarial data usually perform better on other adversarial datasets but worse on a diverse collection of out-of-domain evaluation sets.Finally, we provide a qualitative analysis of adversarial (vs standard) data, identifying key differences and offering guidance for future research. 1
One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers

Pre-trained language models (PLMs) achieve great success in NLP.However, their huge model sizes hinder their applications in many practical systems.Knowledge distillation is a popular technique to compress PLMs, which learns a small student model from a large teacher PLM.However, the knowledge learned from a single teacher may be limited and even biased, resulting in low-quality student model.In this paper, we propose a multi-teacher knowledge distillation framework named MT-BERT for pre-trained language model compression, which can train high-quality student model from multiple teacher PLMs.In MT-BERT we design a multi-teacher co-finetuning method to jointly finetune multiple teacher PLMs in downstream tasks with shared pooling and prediction layers to align their output space for better collaborative teaching.In addition, we propose a multi-teacher hidden loss and a multi-teacher distillation loss to transfer the useful knowledge in both hidden states and soft labels from multiple teacher PLMs to the student model.Experiments on three benchmark datasets validate the effectiveness of MT-BERT in compressing PLMs.
Latent Independent Excitation for Generalizable Sensor-based Cross-Person Activity Recognition

In wearable-sensor-based activity recognition, it is often assumed that the training and test samples follow the same data distribution. This assumption neglects practical scenarios where the activity patterns inevitably vary from person to person. To solve this problem, transfer learning and domain adaptation approaches are often leveraged to reduce the gaps between different participants. Nevertheless, these approaches require additional information (i.e., labeled or unlabeled data, meta-information) from the target domain during the training stage. In this paper, we introduce a novel method named Generalizable Independent Latent Excitation (GILE) for human activity recognition, which greatly enhances the cross-person generalization capability of the model. Our proposed method is superior to existing methods in the sense that it does not require any access to the target domain information. Besides, this novel model can be directly applied to various target domains without re-training or fine-tuning. Specifically, the proposed model learns to automatically disentangle domain-agnostic and domain-specific features, the former of which are expected to be invariant across various persons. To further remove correlations between the two types of features, a novel Independent Excitation mechanism is incorporated in the latent feature space. Comprehensive experimental evaluations are conducted on three benchmark datasets to demonstrate the superiority of the proposed method over the state-of-the-art solutions.
Further compactifying linear optical unitaries

Quantum integrated photonics requires large-scale linear optical circuitry, and for many applications, it is desirable to have a universally programmable circuit, able to implement an arbitrary unitary transformation on a number of modes. This has been achieved using the Reck scheme, consisting of a network of Mach–Zehnder interferometers containing a variable phase shifter in one path as well as an external phase shifter after each Mach–Zehnder. It subsequently became apparent that with symmetric Mach–Zehnders containing a phase shifter in both paths, the external phase shifters are redundant, resulting in a more compact circuit. The rectangular Clements scheme improves on the Reck scheme in terms of circuit depth, but it has been thought that an external phase-shifter was necessary after each Mach–Zehnder. Here, we show that the Clements scheme can be realized using symmetric Mach–Zehnders, requiring only a small number of external phase-shifters that do not contribute to the depth of the circuit. This will result in a significant saving in the length of these devices, allowing more complex circuits to fit onto a photonic chip, and reducing the propagation losses associated with these circuits. We also discuss how similar savings can be made to alternative schemes, which have robustness to imbalanced beam-splitters.
FairOD: Fairness-aware Outlier Detection

Fairness and Outlier Detection (OD) are closely related, as it is exactly the goal of OD to spot rare, minority samples in a given population. However, when being a minority (as defined by protected variables, such as race/ethnicity/sex/age) does not reflect positive-class membership (such as criminal/fraud), OD produces unjust outcomes. Surprisingly, fairness-aware OD has been almost untouched in prior work, as fair machine learning literature mainly focuses on supervised settings. Our work aims to bridge this gap. Specifically, we develop desiderata capturing well-motivated fairness criteria for OD, and systematically formalize the fair OD problem. Further, guided by our desiderata, we propose FairOD, a fairness-aware outlier detector that has the following desirable properties: FairOD (1) exhibits treatment parity at test time, (2) aims to flag equal proportions of samples from all groups (i.e. obtain group fairness, via statistical parity), and (3) strives to flag truly high-risk samples within each group. Extensive experiments on a diverse set of synthetic and real world datasets show that FairOD produces outcomes that are fair with respect to protected variables, while performing comparable to (and in some cases, even better than) fairness-agnostic detectors in terms of detection performance.
Online Active Learning for Drifting Data Streams

Classification methods for streaming data are not new, but very few current frameworks address all three of the most common problems with these tasks: concept drift, noise, and the exorbitant costs associated with labeling the unlabeled instances in data streams. Motivated by this gap in the field, we developed an active learning framework based on a dual-query strategy and Ebbinghaus’s law of human memory cognition. Called CogDQS, the query strategy samples only the most representative instances for manual annotation based on local density and uncertainty, thus significantly reducing the cost of labeling. The policy for discerning drift from noise and replacing outdated instances with new concepts is based on the three criteria of the Ebbinghaus forgetting curve: recall, the fading period, and the memory strength. Simulations comparing CogDQS with baselines on six different data streams containing gradual drift or abrupt drift with and without noise show that our approach produces accurate, stable models with good generalization ability at minimal labeling, storage, and computation costs.
A Rotation-Invariant Optical and SAR Image Registration Algorithm Based on Deep and Gaussian Features

Traditional feature matching methods of optical and synthetic aperture radar (SAR) used gradient are sensitive to non-linear radiation distortions (NRD) and the rotation between two images. To address this problem, this study presents a novel approach to solving the rigid body rotation problem by a two-step process. The first step proposes a deep learning neural network named RotNET to predict the rotation relationship between two images. The second step uses a local feature descriptor based on the Gaussian pyramid named Gaussian pyramid features of oriented gradients (GPOG) to match two images. The RotNET uses a neural network to analyze the gradient histogram of the two images to derive the rotation relationship between optical and SAR images. Subsequently, GPOG is depicted a keypoint by using the histogram of Gaussian pyramid to make one-cell block structure which is simpler and more stable than HOG structure-based descriptors. Finally, this paper designs experiments to prove that the gradient histogram of the optical and SAR images can reflect the rotation relationship and the RotNET can correctly predict them. The similarity map test and the image registration results obtained on experiments show that GPOG descriptor is robust to SAR speckle noise and NRD.