OpenRaiser/Intern-Atlas huggingface dataset:
Towards non-toxic landscapes: Automatic toxic comment detection using DNN
Venue: arXiv cs.LG 2020
Year: 2020

The spectacular expansion of the Internet has led to the development of a new research problem in the field of natural language processing: automatic toxic comment detection, since many countries prohibit hate speech in public media. There is no clear and formal definition of hate, offensive, toxic and abusive speeches. In this article, we put all these terms under the umbrella of "toxic" speech. The contribution of this paper is the design of binary classification and regression-based approaches aiming to predict whether a comment is toxic or not. We compare different unsupervised word representations and different DNN based classifiers. Moreover, we study the robustness of the proposed approaches to adversarial attacks by adding one (healthy or toxic) word. We evaluate the proposed methodology on the English Wikipedia Detox corpus. Our experiments show that using BERT fine-tuning outperforms feature-based BERT, Mikolov's and fastText representations with different DNN classifiers.
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
Venue: arXiv cs.CL 2020
Year: 2020

We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functions. We perform experiments on the standard LibriSpeech dataset, and leverage additional unlabeled data from LibriVox through pseudo-labeling. We show that while Transformer-based acoustic models have superior performance with the supervised dataset alone, semi-supervision improves all models across architectures and loss functions and bridges much of the performance gaps between them. In doing so, we reach a new state-of-the-art for end-to-end acoustic models decoded with an external language model in the standard supervised learning setting, and a new absolute state-of-the-art with semi-supervised training. Finally, we study the effect of leveraging different amounts of unlabeled audio, propose several ways of evaluating the characteristics of unlabeled audio which improve acoustic modeling, and show that acoustic models trained with more audio rely less on external language models.
Talking datasets: Understanding data sensemaking behaviours
Venue: arXiv cs.HC 2020
Year: 2020

The sharing and reuse of data are seen as critical to solving the most complex problems of today. Despite this potential, relatively little is known about a key step in data reuse: people's behaviours involved in data-centric sensemaking. We aim to address this gap by presenting a mixed-methods study combining in-depth interviews, a think-aloud task and a screen recording analysis with 31 researchers as they summarised and interacted with both familiar and unfamiliar data. We use our findings to identify and detail common activity patterns and necessary data attributes across three clusters of sensemaking activities: inspecting data, engaging with content, and placing data within broader contexts. We conclude by proposing design recommendations for tools and documentation practices which can be used to facilitate sensemaking and subsequent data reuse.
A CNN-RNN Framework for Crop Yield Prediction
Venue: arXiv cs.LG 2020
Year: 2020

Crop yield prediction is extremely challenging due to its dependence on multiple factors such as crop genotype, environmental factors, management practices, and their interactions. This paper presents a deep learning framework using convolutional neural networks (CNN) and recurrent neural networks (RNN) for crop yield prediction based on environmental data and management practices. The proposed CNN-RNN model, along with other popular methods such as random forest (RF), deep fully-connected neural networks (DFNN), and LASSO, was used to forecast corn and soybean yield across the entire Corn Belt (including 13 states) in the United States for years 2016, 2017, and 2018 using historical data. The new model achieved a root-mean-square-error (RMSE) 9% and 8% of their respective average yields, substantially outperforming all other methods that were tested. The CNN-RNN have three salient features that make it a potentially useful method for other crop yield prediction studies. (1) The CNN-RNN model was designed to capture the time dependencies of environmental factors and the genetic improvement of seeds over time without having their genotype information. (2) The model demonstrated the capability to generalize the yield prediction to untested environments without significant drop in the prediction accuracy. (3) Coupled with the backpropagation method, the model could reveal the extent to which weather conditions, accuracy of weather predictions, soil conditions, and management practices were able to explain the variation in the crop yields.
The politics of deceptive borders: 'biomarkers of deceit' and the case of iBorderCtrl
Venue: arXiv cs.CY 2020
Year: 2020

This paper critically examines a recently developed proposal for a border control system called iBorderCtrl, designed to detect deception based on facial recognition technology and the measurement of micro-expressions, termed 'biomarkers of deceit'. Funded under the European Commission's Horizon 2020 programme, we situate our analysis in the wider political economy of 'emotional AI' and the history of deception detection technologies. We then move on to interrogate the design of iBorderCtrl using publicly available documents and assess the assumptions and scientific validation underpinning the project design. Finally, drawing on a Bayesian analysis we outline statistical fallacies in the foundational premise of mass screening and argue that it is very unlikely that the model that iBorderCtrl provides for deception detection would work in practice. By interrogating actual systems in this way, we argue that we can begin to question the very premise of the development of data-driven systems, and emotional AI and deception detection in particular, pushing back on the assumption that these systems are fulfilling the tasks they claim to be attending to and instead ask what function such projects carry out in the creation of subjects and management of populations. This function is not merely technical but, rather, we argue, distinctly political and forms part of a mode of governance increasingly shaping life opportunities and fundamental rights.
Game theoretical analysis of Kidney Exchange Programs
Venue: arXiv cs.GT 2020
Year: 2020

The goal of a kidney exchange program (KEP) is to maximize number of transplants within a pool of incompatible patient-donor pairs by exchanging donors. A KEP can be modelled as a maximum matching problem in a graph. A KEP between incompatible patient-donor from pools of several hospitals, regions or countries has the potential to increase the number of transplants. These entities aim is to maximize the transplant benefit for their patients, which can lead to strategic behaviours. Recently, this was formulated as a non-cooperative two-player game and the game solutions (equilibria) were characterized when the entities objective function is the number of their patients receiving a kidney. In this paper, we generalize these results for $N$-players and discuss the impact in the game solutions when transplant information quality is introduced. Furthermore, the game theory model is analyzed through computational experiments on instances generated through the Canada Kidney Paired Donation Program. These experiments highlighting the importance of using the concept of Nash equilibrium, as well as, the anticipation of the necessity to further research for supporting police makers once measures on transplant quality are available.
Gliding vertex on the horizontal bounding box for multi-oriented object detection
Venue: arXiv cs.CV 2020
Year: 2020

Object detection has recently experienced substantial progress. Yet, the widely adopted horizontal bounding box representation is not appropriate for ubiquitous oriented objects such as objects in aerial images and scene texts. In this paper, we propose a simple yet effective framework to detect multi-oriented objects. Instead of directly regressing the four vertices, we glide the vertex of the horizontal bounding box on each corresponding side to accurately describe a multi-oriented object. Specifically, We regress four length ratios characterizing the relative gliding offset on each corresponding side. This may facilitate the offset learning and avoid the confusion issue of sequential label points for oriented objects. To further remedy the confusion issue for nearly horizontal objects, we also introduce an obliquity factor based on area ratio between the object and its horizontal bounding box, guiding the selection of horizontal or oriented detection for each object. We add these five extra target variables to the regression head of faster R-CNN, which requires ignorable extra computation time. Extensive experimental results demonstrate that without bells and whistles, the proposed method achieves superior performances on multiple multi-oriented object detection benchmarks including object detection in aerial images, scene text detection, pedestrian detection in fisheye images.
Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problem
Venue: arXiv cs.AI 2020
Year: 2020

Learning how to automatically solve optimization problems has the potential to provide the next big leap in optimization technology. The performance of automatically learned heuristics on routing problems has been steadily improving in recent years, but approaches based purely on machine learning are still outperformed by state-of-the-art optimization methods. To close this performance gap, we propose a novel large neighborhood search (LNS) framework for vehicle routing that integrates learned heuristics for generating new solutions. The learning mechanism is based on a deep neural network with an attention mechanism and has been especially designed to be integrated into an LNS search setting. We evaluate our approach on the capacitated vehicle routing problem (CVRP) and the split delivery vehicle routing problem (SDVRP). On CVRP instances with up to 297 customers, our approach significantly outperforms an LNS that uses only handcrafted heuristics and a well-known heuristic from the literature. Furthermore, we show for the CVRP and the SDVRP that our approach surpasses the performance of existing machine learning approaches and comes close to the performance of state-of-the-art optimization approaches.
Generalizing Information to the Evolution of Rational Belief
Venue: arXiv cs.IT 2020
Year: 2020

Information theory provides a mathematical foundation to measure uncertainty in belief. Belief is represented by a probability distribution that captures our understanding of an outcome's plausibility. Information measures based on Shannon's concept of entropy include realization information, Kullback-Leibler divergence, Lindley's information in experiment, cross entropy, and mutual information. We derive a general theory of information from first principles that accounts for evolving belief and recovers all of these measures. Rather than simply gauging uncertainty, information is understood in this theory to measure change in belief. We may then regard entropy as the information we expect to gain upon realization of a discrete latent random variable. This theory of information is compatible with the Bayesian paradigm in which rational belief is updated as evidence becomes available. Furthermore, this theory admits novel measures of information with well-defined properties, which we explore in both analysis and experiment. This view of information illuminates the study of machine learning by allowing us to quantify information captured by a predictive model and distinguish it from residual information contained in training data. We gain related insights regarding feature selection, anomaly detection, and novel Bayesian approaches.
ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring
Venue: arXiv cs.LG 2020
Year: 2020

We improve the recently-proposed "MixMatch" semi-supervised learning algorithm by introducing two new techniques: distribution alignment and augmentation anchoring. Distribution alignment encourages the marginal distribution of predictions on unlabeled data to be close to the marginal distribution of ground-truth labels. Augmentation anchoring feeds multiple strongly augmented versions of an input into the model and encourages each output to be close to the prediction for a weakly-augmented version of the same input. To produce strong augmentations, we propose a variant of AutoAugment which learns the augmentation policy while the model is being trained. Our new algorithm, dubbed ReMixMatch, is significantly more data-efficient than prior work, requiring between $5\times$ and $16\times$ less data to reach the same accuracy. For example, on CIFAR-10 with 250 labeled examples we reach $93.73\%$ accuracy (compared to MixMatch's accuracy of $93.58\%$ with $4{,}000$ examples) and a median accuracy of $84.92\%$ with just four labels per class. We make our code and data open-source at https://github.com/google-research/remixmatch.
The JDDC Corpus: A Large-Scale Multi-Turn Chinese Dialogue Dataset for E-commerce Customer Service
Venue: arXiv cs.CL 2020
Year: 2020

Human conversations are complicated and building a human-like dialogue agent is an extremely challenging task. With the rapid development of deep learning techniques, data-driven models become more and more prevalent which need a huge amount of real conversation data. In this paper, we construct a large-scale real scenario Chinese E-commerce conversation corpus, JDDC, with more than 1 million multi-turn dialogues, 20 million utterances, and 150 million words. The dataset reflects several characteristics of human-human conversations, e.g., goal-driven, and long-term dependency among the context. It also covers various dialogue types including task-oriented, chitchat and question-answering. Extra intent information and three well-annotated challenge sets are also provided. Then, we evaluate several retrieval-based and generative models to provide basic benchmark performance on the JDDC corpus. And we hope JDDC can serve as an effective testbed and benefit the development of fundamental research in dialogue task
Artificial neural networks in action for an automated cell-type classification of biological neural networks
Venue: arXiv cs.NE 2020
Year: 2020

Identification of different neuronal cell types is critical for understanding their contribution to brain functions. Yet, automated and reliable classification of neurons remains a challenge, primarily because of their biological complexity. Typical approaches include laborious and expensive immunohistochemical analysis while feature extraction algorithms based on cellular characteristics have recently been proposed. The former rely on molecular markers, which are often expressed in many cell types, while the latter suffer from similar issues: finding features that are distinctive for each class has proven to be equally challenging. Moreover, both approaches are time consuming and demand a lot of human intervention. In this work we establish the first, automated cell-type classification method that relies on neuronal activity rather than molecular or cellular features. We test our method on a real-world dataset comprising of raw calcium activity signals for four neuronal types. We compare the performance of three different deep learning models and demonstrate that our method can achieve automated classification of neuronal cell types with unprecedented accuracy.
Asymmetric entanglement-assisted quantum error-correcting codes and BCH codes
Venue: arXiv cs.IT 2020
Year: 2020

The concept of asymmetric entanglement-assisted quantum error-correcting code (asymmetric EAQECC) is introduced in this article. Codes of this type take advantage of the asymmetry in quantum errors since phase-shift errors are more probable than qudit-flip errors. Moreover, they use pre-shared entanglement between encoder and decoder to simplify the theory of quantum error correction and increase the communication capacity. Thus, asymmetric EAQECCs can be constructed from any pair of classical linear codes over an arbitrary field. Their parameters are described and a Gilbert-Varshamov bound is presented. Explicit parameters of asymmetric EAQECCs from BCH codes are computed and examples exceeding the introduced Gilbert-Varshamov bound are shown.
ptype: Probabilistic Type Inference
Venue: arXiv cs.LG 2020
Year: 2020

Type inference refers to the task of inferring the data type of a given column of data. Current approaches often fail when data contains missing data and anomalies, which are found commonly in real-world data sets. In this paper, we propose ptype, a probabilistic robust type inference method that allows us to detect such entries, and infer data types. We further show that the proposed method outperforms the existing methods.
Implementation of Optical Deep Neural Networks using the Fabry-Perot Interferometer
Venue: arXiv cs.ET 2020
Year: 2020

Future developments in deep learning applications requiring large datasets will be limited by power and speed limitations of silicon based Von-Neumann computing architectures. Optical architectures provide a low power and high speed hardware alternative. Recent publications have suggested promising implementations of optical neural networks (ONNs), showing huge orders of magnitude efficiency and speed gains over current state of the art hardware alternatives. In this work, the transmission of the Fabry-Perot Interferometer (FPI) is proposed as a low power, low footprint activation function unit. Numerical simulations of optical CNNs using the FPI based activation functions show accuracies of 98% on the MNIST dataset. An investigation of possible physical implementation of the network shows that an ONN based on current tunable FPIs could be slowed by actuation delays, but rapidly developing optical hardware fabrication techniques could make an integrated approach using the proposed FPI setups a powerful solution for previously inaccessible deep learning applications.
Secure Sketch for All Noisy Sources (Noisy)
Venue: arXiv cs.IT 2020
Year: 2020

Secure sketch produces public information of its input $w$ without revealing it, yet, allows the exact recovery of $w$ given another value $w'$ that is close to $w$. Therefore, it can be used to reliably reproduce any error-prone secret (i.e., biometrics) stored in secret storage. However, some sources have lower entropy compared to the error itself, formally called "more error than entropy", a standard secure sketch cannot show its security promise perfectly to these kind of sources. This paper focuses on secure sketch. We propose a concrete construction for secure sketch. We show security to all noisy sources, including the trivial source with zero min-entropy. In addition, our construction comes with efficient recovery algorithm operates in polynomial time in the sketch size, which can tolerate high number of error rate arbitrary close to 1/2. Above result acts in conjunction to our derivation on the solution to two NP-complete coding problems, suggesting P=NP.
A Transformer-based approach to Irony and Sarcasm detection
Venue: arXiv cs.CL 2020
Year: 2020

Figurative Language (FL) seems ubiquitous in all social-media discussion forums and chats, posing extra challenges to sentiment analysis endeavors. Identification of FL schemas in short texts remains largely an unresolved issue in the broader field of Natural Language Processing (NLP), mainly due to their contradictory and metaphorical meaning content. The main FL expression forms are sarcasm, irony and metaphor. In the present paper we employ advanced Deep Learning (DL) methodologies to tackle the problem of identifying the aforementioned FL forms. Significantly extending our previous work [71], we propose a neural network methodology that builds on a recently proposed pre-trained transformer-based network architecture which, is further enhanced with the employment and devise of a recurrent convolutional neural network (RCNN). With this set-up, data preprocessing is kept in minimum. The performance of the devised hybrid neural architecture is tested on four benchmark datasets, and contrasted with other relevant state of the art methodologies and systems. Results demonstrate that the proposed methodology achieves state of the art performance under all benchmark datasets, outperforming, even by a large margin, all other methodologies and published studies.
An Iterative Security Game for Computing Robust and Adaptive Network Flows
Venue: arXiv cs.GT 2020
Year: 2020

The recent advancement in real-world critical infrastructure networks has led to an exponential growth in the use of automated devices which in turn has created new security challenges. In this paper, we study the robust and adaptive maximum flow problem in an uncertain environment where the network parameters (e.g., capacities) are known and deterministic, but the network structure (e.g., edges) is vulnerable to adversarial attacks or failures. We propose a robust and sustainable network flow model to effectively and proactively counter plausible attacking behaviors of an adversary operating under a budget constraint. Specifically, we introduce a novel scenario generation approach based on an iterative two-player game between a defender and an adversary. We assume that the adversary always takes a best myopic response (out of some feasible attacks) against the current flow scenario prepared by the defender. On the other hand, we assume that the defender considers all the attacking behaviors revealed by the adversary in previous iterations in order to generate a new conservative flow strategy that is robust (maximin) against all those attacks. This iterative game continues until the objectives of the adversary and the administrator both converge. We show that the robust network flow problem to be solved by the defender is NP-hard and that the complexity of the adversary's decision problem grows exponentially with the network size and the adversary's budget value. We propose two principled heuristic approaches for solving the adversary's problem at the scale of a large urban network. Extensive computational results on multiple synthetic and real-world data sets demonstrate that the solution provided by the defender's problem significantly increases the amount of flow pushed through the network and reduces the expected lost flow over four state-of-the-art benchmark approaches.