OpenRaiser/Intern-Atlas huggingface dataset:
Clustering based Contrastive Learning for Improving Face Representations
Venue: arXiv cs.CV 2020
Year: 2020

A good clustering algorithm can discover natural groupings in data. These groupings, if used wisely, provide a form of weak supervision for learning representations. In this work, we present Clustering-based Contrastive Learning (CCL), a new clustering-based representation learning approach that uses labels obtained from clustering along with video constraints to learn discriminative face features. We demonstrate our method on the challenging task of learning representations for video face clustering. Through several ablation studies, we analyze the impact of creating pair-wise positive and negative labels from different sources. Experiments on three challenging video face clustering datasets: BBT-0101, BF-0502, and ACCIO show that CCL achieves a new state-of-the-art on all datasets.
AR: Auto-Repair the Synthetic Data for Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Compared with only using limited authentic parallel data as training corpus, many studies have proved that incorporating synthetic parallel data, which generated by back translation (BT) or forward translation (FT, or selftraining), into the NMT training process can significantly improve translation quality. However, as a well-known shortcoming, synthetic parallel data is noisy because they are generated by an imperfect NMT system. As a result, the improvements in translation quality bring by the synthetic parallel data are greatly diminished. In this paper, we propose a novel Auto- Repair (AR) framework to improve the quality of synthetic data. Our proposed AR model can learn the transformation from low quality (noisy) input sentence to high quality sentence based on large scale monolingual data with BT and FT techniques. The noise in synthetic parallel data will be sufficiently eliminated by the proposed AR model and then the repaired synthetic parallel data can help the NMT models to achieve larger improvements. Experimental results show that our approach can effective improve the quality of synthetic parallel data and the NMT model with the repaired synthetic data achieves consistent improvements on both WMT14 EN!DE and IWSLT14 DE!EN translation tasks.
Understanding Learning Dynamics for Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

Despite the great success of NMT, there still remains a severe challenge: it is hard to interpret the internal dynamics during its training process. In this paper we propose to understand learning dynamics of NMT by using a recent proposed technique named Loss Change Allocation (LCA)~\citep{lan-2019-loss-change-allocation}. As LCA requires calculating the gradient on an entire dataset for each update, we instead present an approximate to put it into practice in NMT scenario. %motivated by the lesson from sgd. Our simulated experiment shows that such approximate calculation is efficient and is empirically proved to deliver consistent results to the brute-force implementation. In particular, extensive experiments on two standard translation benchmark datasets reveal some valuable findings.
Clique-Width of Point Configurations
Venue: arXiv cs.LO 2020
Year: 2020

While structural width parameters (of the input) belong to the standard toolbox of graph algorithms, it is not the usual case in computational geometry. As a case study we propose a natural extension of the structural graph parameter of clique-width to geometric point configurations represented by their order type. We study basic properties of this clique-width notion, and relate it to the monadic second-order logic of point configurations. As an application, we provide several linear FPT time algorithms for geometric point problems which are NP-hard in general, in the special case that the input point set is of bounded clique-width and the clique-width expression is also given.
Hyper-spectral NIR and MIR data and optimal wavebands for detection of apple tree diseases
Venue: arXiv cs.CV 2020
Year: 2020

Plant diseases can lead to dramatic losses in yield and quality of food, becoming a problem of high priority for farmers. Apple scab, moniliasis, and powdery mildew are the most significant apple tree diseases worldwide and may cause between 50% and 60% in yield losses annually; they are controlled by fungicide use with huge financial and time expenses. This research proposes a modern approach for analyzing the spectral data in Near-Infrared and Mid-Infrared ranges of the apple tree diseases at different stages. Using the obtained spectra, we found optimal spectral bands for detecting particular disease and discriminating it from other diseases and healthy trees. The proposed instrument will provide farmers with accurate, real-time information on different stages of apple tree diseases, enabling more effective timing, and selecting the fungicide application, resulting in better control and increasing yield. The obtained dataset, as well as scripts in Matlab for processing data and finding optimal spectral bands, are available via the link: https://yadi.sk/d/ZqfGaNlYVR3TUA
Finding the Optimal Vocabulary Size for Neural Machine Translation
Venue: arXiv cs.CL 2020
Year: 2020

We cast neural machine translation (NMT) as a classification task in an autoregressive setting and analyze the limitations of both classification and autoregression components. Classifiers are known to perform better with balanced class distributions during training. Since the Zipfian nature of languages causes imbalanced classes, we explore its effect on NMT. We analyze the effect of various vocabulary sizes on NMT performance on multiple languages with many data sizes, and reveal an explanation for why certain vocabulary sizes are better than others.
The n-dimensional k-vector and its application to orthogonal range searching
Venue: arXiv cs.DS 2020
Year: 2020

This work focuses on the definition and study of the n-dimensional k-vector, an algorithm devised to perform orthogonal range searching in static databases with multiple dimensions. The methodology first finds the order in which to search the dimensions, and then, performs the search using a modified projection method. In order to determine the dimension order, the algorithm uses the k-vector, a range searching technique for one dimension that identifies the number of elements contained in the searching range. Then, using this information, the algorithm predicts and selects the best approach to deal with each dimension. The algorithm has a worst case complexity of $\mathcal{O}(nd(k/n)^{2/d})$, where $k$ is the number of elements retrieved, $n$ is the number of elements in the database, and $d$ is the number of dimensions of the database. This work includes a detailed description of the methodology as well as a study of the algorithm performance.
Nonlinear Function Inversion using k-vector
Venue: arXiv cs.DS 2020
Year: 2020

This work introduces a general numerical technique to invert one dimensional analytic or tabulated nonlinear functions in assigned ranges of interest. The proposed approach is based on an optimal version of the k-vector range searching, an ad-hoc modification devised for function inversion. The optimality consists of retrieving always the same number of data ($1,2,\dots$) for a specified searching range to initiate the root solver. This provides flexibility to adapt the technique to a variety of root solvers (e.g., bisection, Newton, etc.), using a specified number of starting points. The proposed method allows to build an inverse function toolbox for a set of specified nonlinear functions. In particular, the method is suitable when intensive inversions of the same function are required. The inversion is extremely fast (almost instantaneous), but it requires a one-time preprocessing effort.
Deep Reinforcement Learning-Aided Random Access
Venue: arXiv cs.IT 2020
Year: 2020

We consider a system model comprised of an access point (AP) and K Internet of Things (IoT) nodes that sporadically become active in order to send data to the AP. The AP is assumed to have N time-frequency resource blocks that it can allocate to the IoT nodes that wish to send data, where N < K. The main problem is how to allocate the N time-frequency resource blocks to the IoT nodes in each time slot such that the average packet rate is maximized. For this problem, we propose a deep reinforcement learning (DRL)-aided random access (RA) scheme, where an intelligent DRL agent at the AP learns to predict the activity of the IoT nodes in each time slot and grants time-frequency resource blocks to the IoT nodes predicted as active. Next, the IoT nodes that are missclassified as non-active by the DRL agent, as well as unseen or newly arrived nodes in the cell, employ the standard RA scheme in order to obtain time-frequency resource blocks. We leverage expert knowledge for faster training of the DRL agent. Our numerical results show significant improvements in terms of average packet rate when the proposed DRL-aided RA scheme is implemented compared to the existing solution used in practice, the standard RA scheme.
DRAMDig: A Knowledge-assisted Tool to Uncover DRAM Address Mapping
Venue: arXiv cs.CR 2020
Year: 2020

As recently emerged rowhammer exploits require undocumented DRAM address mapping, we propose a generic knowledge-assisted tool, DRAMDig, which takes domain knowledge into consideration to efficiently and deterministically uncover the DRAM address mappings on any Intel-based machines. We test DRAMDig on a number of machines with different combinations of DRAM chips and microarchitectures ranging from Intel Sandy Bridge to Coffee Lake. Comparing to previous works, DRAMDig deterministically reverse-engineered DRAM address mappings on all the test machines with only 7.8 minutes on average. Based on the uncovered mappings, we perform double-sided rowhammer tests and the results show that DRAMDig induced significantly more bit flips than previous works, justifying the correctness of the uncovered DRAM address mappings.
Learning to Recover Reasoning Chains for Multi-Hop Question Answering via Cooperative Games
Venue: arXiv cs.CL 2020
Year: 2020

We propose the new problem of learning to recover reasoning chains from weakly supervised signals, i.e., the question-answer pairs. We propose a cooperative game approach to deal with this problem, in which how the evidence passages are selected and how the selected passages are connected are handled by two models that cooperate to select the most confident chains from a large set of candidates (from distant supervision). For evaluation, we created benchmarks based on two multi-hop QA datasets, HotpotQA and MedHop; and hand-labeled reasoning chains for the latter. The experimental results demonstrate the effectiveness of our proposed approach.
A Learning Framework for n-bit Quantized Neural Networks toward FPGAs
Venue: arXiv cs.LG 2020
Year: 2020

The quantized neural network (QNN) is an efficient approach for network compression and can be widely used in the implementation of FPGAs. This paper proposes a novel learning framework for n-bit QNNs, whose weights are constrained to the power of two. To solve the gradient vanishing problem, we propose a reconstructed gradient function for QNNs in back-propagation algorithm that can directly get the real gradient rather than estimating an approximate gradient of the expected loss. We also propose a novel QNN structure named n-BQ-NN, which uses shift operation to replace the multiply operation and is more suitable for the inference on FPGAs. Furthermore, we also design a shift vector processing element (SVPE) array to replace all 16-bit multiplications with SHIFT operations in convolution operation on FPGAs. We also carry out comparable experiments to evaluate our framework. The experimental results show that the quantized models of ResNet, DenseNet and AlexNet through our learning framework can achieve almost the same accuracies with the original full-precision models. Moreover, when using our learning framework to train our n-BQ-NN from scratch, it can achieve state-of-the-art results compared with typical low-precision QNNs. Experiments on Xilinx ZCU102 platform show that our n-BQ-NN with our SVPE can execute 2.9 times faster than with the vector processing element (VPE) in inference. As the SHIFT operation in our SVPE array will not consume Digital Signal Processings (DSPs) resources on FPGAs, the experiments have shown that the use of SVPE array also reduces average energy consumption to 68.7% of the VPE array with 16-bit.
Vanishing Point Guided Natural Image Stitching
Venue: arXiv cs.CV 2020
Year: 2020

Recently, works on improving the naturalness of stitching images gain more and more extensive attention. Previous methods suffer the failures of severe projective distortion and unnatural rotation, especially when the number of involved images is large or images cover a very wide field of view. In this paper, we propose a novel natural image stitching method, which takes into account the guidance of vanishing points to tackle the mentioned failures. Inspired by a vital observation that mutually orthogonal vanishing points in Manhattan world can provide really useful orientation clues, we design a scheme to effectively estimate prior of image similarity. Given such estimated prior as global similarity constraints, we feed it into a popular mesh deformation framework to achieve impressive natural stitching performances. Compared with other existing methods, including APAP, SPHP, AANAP, and GSP, our method achieves state-of-the-art performance in both quantitative and qualitative experiments on natural image stitching.
Trust-based Multiagent Consensus or Weightings Aggregation
Venue: arXiv cs.AI 2020
Year: 2020

We introduce a framework for reaching a consensus amongst several agents communicating via a trust network on conflicting information about their environment. We formalise our approach and provide an empirical and theoretical analysis of its properties.
Building a Norwegian Lexical Resource for Medical Entity Recognition
Venue: arXiv cs.CL 2020
Year: 2020

We present a large Norwegian lexical resource of categorized medical terms. The resource merges information from large medical databases, and contains over 77,000 unique entries, including automatically mapped terms from a Norwegian medical dictionary. We describe the methodology behind this automatic dictionary entry mapping based on keywords and suffixes and further present the results of a manual evaluation performed on a subset by a domain expert. The evaluation indicated that ca. 80% of the mappings were correct.
Large-scale spatiotemporal photonic reservoir computer for image classification
Venue: arXiv cs.NE 2020
Year: 2020

We propose a scalable photonic architecture for implementation of feedforward and recurrent neural networks to perform the classification of handwritten digits from the MNIST database. Our experiment exploits off-the-shelf optical and electronic components to currently achieve a network size of 16,384 nodes. Both network types are designed within the the reservoir computing paradigm with randomly weighted input and hidden layers. Using various feature extraction techniques (e.g. histograms of oriented gradients, zoning, Gabor filters) and a simple training procedure consisting of linear regression and winner-takes-all decision strategy, we demonstrate numerically and experimentally that a feedforward network allows for classification error rate of 1%, which is at the state-of-the-art for experimental implementations and remains competitive with more advanced algorithmic approaches. We also investigate recurrent networks in numerical simulations by explicitly activating the temporal dynamics, and predict a performance improvement over the feedforward configuration.
DySky: Dynamic Skyline Queries on Uncertain Graphs
Venue: arXiv cs.DB 2020
Year: 2020

Given a graph, and a set of query vertices (subset of the vertices), the dynamic skyline query problem returns a subset of data vertices (other than query vertices) which are not dominated by other data vertices based on certain distance measure. In this paper, we study the dynamic skyline query problem on uncertain graphs (DySky). The input to this problem is an uncertain graph, a subset of its nodes as query vertices, and the goal here is to return all the data vertices which are not dominated by others. We employ two distance measures in uncertain graphs, namely, \emph{Majority Distance}, and \emph{Expected Distance}. Our approach is broadly divided into three steps: \emph{Pruning}, \emph{Distance Computation}, and \emph{Skyline Vertex Set Generation}. We implement the proposed methodology with three publicly available datasets and observe that it can find out skyline vertex set without taking much time even for million sized graphs if expected distance is concerned. Particularly, the pruning strategy reduces the computational time significantly.
Dynamic Ridesharing in Peak Travel Periods
Venue: arXiv cs.DB 2020
Year: 2020

In this paper, we study a variant of the dynamic ridesharing problem with a specific focus on peak hours: Given a set of drivers and rider requests, we aim to match drivers to each rider request by achieving two objectives: maximizing the served rate and minimizing the total additional distance, subject to a series of spatio-temporal constraints. Our problem can be distinguished from existing work in three aspects: (1) Previous work did not fully explore the impact of peak travel periods where the number of rider requests is much greater than the number of available drivers. (2) Existing solutions usually rely on single objective optimization techniques, such as minimizing the total travel cost. (3) When evaluating the overall system performance, the runtime spent on updating drivers' trip schedules as per incoming rider requests should be incorporated, while it is excluded by most existing solutions. We propose an index structure together with a set of pruning rules and an efficient algorithm to include new riders into drivers' existing trip schedule. To answer new rider requests effectively, we propose two algorithms that match drivers with rider requests. Finally, we perform extensive experiments on a large-scale test collection to validate the proposed methods.
Non-invasive modelling methodology for the diagnosis of Coronary Artery Disease using Fuzzy Cognitive Maps
Venue: arXiv cs.AI 2020
Year: 2020

Cardiovascular Diseases (CVD) and strokes produce immense health and economic burdens globally. Coronary Artery Disease (CAD) is the most common type of cardiovascular disease. Coronary Angiography, which is an invasive treatment, is also the standard procedure for diagnosing CAD. In this work, we illustrate a Medical Decision Support System for the prediction of Coronary Artery Disease (CAD) utilizing Fuzzy Cognitive Maps (FCMs). FCMs are a promising modeling methodology, based on human knowledge, capable of dealing with ambiguity and uncertainty, and learning how to adapt to the unknown or changing environment. The newly proposed MDSS is developed using the basic notions of Fuzzy Logic and Fuzzy Cognitive Maps, with some adjustments to improve the results. The proposed model, tested on a labelled CAD dataset of 303 patients, obtains an accuracy of 78.2% outmatching several state-of-the-art classification algorithms.
Multi-Agent Deep Stochastic Policy Gradient for Event Based Dynamic Spectrum Access
Venue: arXiv cs.IT 2020
Year: 2020

We consider the dynamic spectrum access (DSA) problem where $K$ Internet of Things (IoT) devices compete for $T$ time slots constituting a frame. Devices collectively monitor $M$ events where each event could be monitored by multiple IoT devices. Each device, when at least one of its monitored events is active, picks an event and a time slot to transmit the corresponding active event information. In the case where multiple devices select the same time slot, a collision occurs and all transmitted packets are discarded. In order to capture the fact that devices observing the same event may transmit redundant information, we consider the maximization of the average sum event rate of the system instead of the classical frame throughput. We propose a multi-agent reinforcement learning approach based on a stochastic version of Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to access the frame by exploiting device-level correlation and time correlation of events. Through numerical simulations, we show that the proposed approach is able to efficiently exploit the aforementioned correlations and outperforms benchmark solutions such as standard multiple access protocols and the widely used Independent Deep Q-Network (IDQN) algorithm.
Evaluating Models' Local Decision Boundaries via Contrast Sets
Venue: arXiv cs.CL 2020
Year: 2020

Standard test sets for supervised learning evaluate in-distribution generalization. Unfortunately, when a dataset has systematic gaps (e.g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities. We propose a new annotation paradigm for NLP that helps to close systematic gaps in the test data. In particular, after a dataset is constructed, we recommend that the dataset authors manually perturb the test instances in small but meaningful ways that (typically) change the gold label, creating contrast sets. Contrast sets provide a local view of a model's decision boundary, which can be used to more accurately evaluate a model's true linguistic capabilities. We demonstrate the efficacy of contrast sets by creating them for 10 diverse NLP datasets (e.g., DROP reading comprehension, UD parsing, IMDb sentiment analysis). Although our contrast sets are not explicitly adversarial, model performance is significantly lower on them than on the original test sets---up to 25\% in some cases. We release our contrast sets as new evaluation benchmarks and encourage future dataset construction efforts to follow similar annotation processes.
Meta-Learning for Few-Shot NMT Adaptation
Venue: arXiv cs.CL 2020
Year: 2020

We present META-MT, a meta-learning approach to adapt Neural Machine Translation (NMT) systems in a few-shot setting. META-MT provides a new approach to make NMT models easily adaptable to many target domains with the minimal amount of in-domain data. We frame the adaptation of NMT systems as a meta-learning problem, where we learn to adapt to new unseen domains based on simulated offline meta-training domain adaptation tasks. We evaluate the proposed meta-learning strategy on ten domains with general large scale NMT systems. We show that META-MT significantly outperforms classical domain adaptation when very few in-domain examples are available. Our experiments shows that META-MT can outperform classical fine-tuning by up to 2.5 BLEU points after seeing only 4, 000 translated words (300 parallel sentences).