OpenRaiser/Intern-Atlas huggingface dataset:
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
Venue: arXiv cs.CL 2020
Year: 2020

In this paper we present the first model for directly synthesizing fluent, natural-sounding spoken audio captions for images that does not require natural language text as an intermediate representation or source of supervision. Instead, we connect the image captioning module and the speech synthesis module with a set of discrete, sub-word speech units that are discovered with a self-supervised visual grounding task. We conduct experiments on the Flickr8k spoken caption dataset in addition to a novel corpus of spoken audio captions collected for the popular MSCOCO dataset, demonstrating that our generated captions also capture diverse visual semantics of the images they describe. We investigate several different intermediate speech representations, and empirically find that the representation must satisfy several important properties to serve as drop-in replacements for text.
Incremental Embedding Learning via Zero-Shot Translation
Venue: arXiv cs.CV 2020
Year: 2020

Modern deep learning methods have achieved great success in machine learning and computer vision fields by learning a set of pre-defined datasets. Howerver, these methods perform unsatisfactorily when applied into real-world situations. The reason of this phenomenon is that learning new tasks leads the trained model quickly forget the knowledge of old tasks, which is referred to as catastrophic forgetting. Current state-of-the-art incremental learning methods tackle catastrophic forgetting problem in traditional classification networks and ignore the problem existing in embedding networks, which are the basic networks for image retrieval, face recognition, zero-shot learning, etc. Different from traditional incremental classification networks, the semantic gap between the embedding spaces of two adjacent tasks is the main challenge for embedding networks under incremental learning setting. Thus, we propose a novel class-incremental method for embedding network, named as zero-shot translation class-incremental method (ZSTCI), which leverages zero-shot translation to estimate and compensate the semantic gap without any exemplars. Then, we try to learn a unified representation for two adjacent tasks in sequential learning process, which captures the relationships of previous classes and current classes precisely. In addition, ZSTCI can easily be combined with existing regularization-based incremental learning methods to further improve performance of embedding networks. We conduct extensive experiments on CUB-200-2011 and CIFAR100, and the experiment results prove the effectiveness of our method. The code of our method has been released.
Neural Machine Translation: A Review of Methods, Resources, and Tools
Venue: arXiv cs.CL 2020
Year: 2020

Machine translation (MT) is an important sub-field of natural language processing that aims to translate natural languages using computers. In recent years, end-to-end neural machine translation (NMT) has achieved great success and has become the new mainstream method in practical MT systems. In this article, we first provide a broad review of the methods for NMT and focus on methods relating to architectures, decoding, and data augmentation. Then we summarize the resources and tools that are useful for researchers. Finally, we conclude with a discussion of possible future research directions.
On the importance of functions in data modeling
Venue: arXiv cs.DB 2020
Year: 2020

In this paper we argue that representing entity properties by tuple attributes, as evangelized in most set-oriented data models, is a controversial method conflicting with the principle of tuple immutability. As a principled solution to this problem of tuple immutability on one hand and the need to modify tuple attributes on the other hand, we propose to use mathematical functions for representing entity properties. In this approach, immutable tuples are intended for representing the existence of entities while mutable functions (mappings between sets) are used for representing entity properties. In this model, called the concept-oriented model (COM), functions are made first-class elements along with sets, and both functions and sets are used to represent and process data in a simpler and more natural way in comparison to purely set-oriented models.
A Deep Retinal Image Quality Assessment Network with Salient Structure Priors
Venue: arXiv cs.CV 2020
Year: 2020

Retinal image quality assessment is an essential prerequisite for diagnosis of retinal diseases. Its goal is to identify retinal images in which anatomic structures and lesions attracting ophthalmologists' attention most are exhibited clearly and definitely while reject poor quality fundus images. Motivated by this, we mimic the way that ophthalmologists assess the quality of retinal images and propose a method termed SalStructuIQA. First, two salient structures for automated retinal quality assessment. One is the large-size salient structures including optic disc region and exudates in large-size. The other is the tiny-size salient structures which mainly include vessels. Then we incorporate the proposed two salient structure priors with deep convolutional neural network (CNN) to shift the focus of CNN to salient structures. Accordingly, we develop two CNN architectures: Dual-branch SalStructIQA and Single-branch SalStructIQA. Dual-branch SalStructIQA contains two CNN branches and one is guided by large-size salient structures while the other is guided by tiny-size salient structures. Single-branch SalStructIQA contains one CNN branch, which is guided by the concatenation of salient structures in both large-size and tiny-size. Experimental results on Eye-Quality dataset show that our proposed Dual-branch SalStructIQA outperforms the state-of-the-art methods for retinal image quality assessment and Single-branch SalStructIQA is much light-weight comparing with state-of-the-art deep retinal image quality assessment methods and still achieves competitive performances.
PFL-MoE: Personalized Federated Learning Based on Mixture of Experts
Venue: arXiv cs.LG 2020
Year: 2020

Federated learning (FL) is an emerging distributed machine learning paradigm that avoids data sharing among training nodes so as to protect data privacy. Under coordination of the FL server, each client conducts model training using its own computing resource and private data set. The global model can be created by aggregating the training results of clients. To cope with highly non-IID data distributions, personalized federated learning (PFL) has been proposed to improve overall performance by allowing each client to learn a personalized model. However, one major drawback of a personalized model is the loss of generalization. To achieve model personalization while maintaining generalization, in this paper, we propose a new approach, named PFL-MoE, which mixes outputs of the personalized model and global model via the MoE architecture. PFL-MoE is a generic approach and can be instantiated by integrating existing PFL algorithms. Particularly, we propose the PFL-MF algorithm which is an instance of PFL-MoE based on the freeze-base PFL algorithm. We further improve PFL-MF by enhancing the decision-making ability of MoE gating network and propose a variant algorithm PFL-MFE. We demonstrate the effectiveness of PFL-MoE by training the LeNet-5 and VGG-16 models on the Fashion-MNIST and CIFAR-10 datasets with non-IID partitions.
TexSmart: A Text Understanding System for Fine-Grained NER and Enhanced Semantic Analysis
Venue: arXiv cs.CL 2020
Year: 2020

This technique report introduces TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities. Compared to most previous publicly available text understanding systems and tools, TexSmart holds some unique features. First, the NER function of TexSmart supports over 1,000 entity types, while most other public tools typically support several to (at most) dozens of entity types. Second, TexSmart introduces new semantic analysis functions like semantic expansion and deep semantic representation, that are absent in most previous systems. Third, a spectrum of algorithms (from very fast algorithms to those that are relatively slow but more accurate) are implemented for one function in TexSmart, to fulfill the requirements of different academic and industrial applications. The adoption of unsupervised or weakly-supervised algorithms is especially emphasized, with the goal of easily updating our models to include fresh data with less human annotation efforts. The main contents of this report include major functions of TexSmart, algorithms for achieving these functions, how to use the TexSmart toolkit and Web APIs, and evaluation results of some key algorithms.
Investigating Memorability of Dynamic Media
Venue: arXiv cs.MM 2020
Year: 2020

The Predicting Media Memorability task in MediaEval'20 has some challenging aspects compared to previous years. In this paper we identify the high-dynamic content in videos and dataset of limited size as the core challenges for the task, we propose directions to overcome some of these challenges and we present our initial result in these directions.
Multi-Channel Sequential Behavior Networks for User Modeling in Online Advertising
Venue: arXiv cs.IR 2020
Year: 2020

Multiple content providers rely on native advertisement for revenue by placing ads within the organic content of their pages. We refer to this setting as ``queryless'' to differentiate from search advertisement where a user submits a search query and gets back related ads. Understanding user intent is critical because relevant ads improve user experience and increase the likelihood of delivering clicks that have value to our advertisers. This paper presents Multi-Channel Sequential Behavior Network (MC-SBN), a deep learning approach for embedding users and ads in a semantic space in which relevance can be evaluated. Our proposed user encoder architecture summarizes user activities from multiple input channels--such as previous search queries, visited pages, or clicked ads--into a user vector. It uses multiple RNNs to encode sequences of event sessions from the different channels and then applies an attention mechanism to create the user representation. A key property of our approach is that user vectors can be maintained and updated incrementally, which makes it feasible to be deployed for large-scale serving. We conduct extensive experiments on real-world datasets. The results demonstrate that MC-SBN can improve the ranking of relevant ads and boost the performance of both click prediction and conversion prediction in the queryless native advertising setting.
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
Venue: arXiv cs.CL 2020
Year: 2020

In social settings, much of human behavior is governed by unspoken rules of conduct. For artificial systems to be fully integrated into social environments, adherence to such norms is a central prerequisite. We investigate whether contemporary NLG models can function as behavioral priors for systems deployed in social settings by generating action hypotheses that achieve predefined goals under moral constraints. Moreover, we examine if models can anticipate likely consequences of (im)moral actions, or explain why certain actions are preferable by generating relevant norms. For this purpose, we introduce 'Moral Stories', a crowd-sourced dataset of structured, branching narratives for the study of grounded, goal-oriented social reasoning. Finally, we propose decoding strategies that effectively combine multiple expert models to significantly improve the quality of generated actions, consequences, and norms compared to strong baselines, e.g. though abductive reasoning.
LookHops: light multi-order convolution and pooling for graph classification
Venue: arXiv cs.LG 2020
Year: 2020

Convolution and pooling are the key operations to learn hierarchical representation for graph classification, where more expressive $k$-order($k>1$) method requires more computation cost, limiting the further applications. In this paper, we investigate the strategy of selecting $k$ via neighborhood information gain and propose light $k$-order convolution and pooling requiring fewer parameters while improving the performance. Comprehensive and fair experiments through six graph classification benchmarks show: 1) the performance improvement is consistent to the $k$-order information gain. 2) the proposed convolution requires fewer parameters while providing competitive results. 3) the proposed pooling outperforms SOTA algorithms in terms of efficiency and performance.
A simple combinatorial algorithm for restricted 2-matchings in subcubic graphs -- via half-edges
Venue: arXiv cs.DS 2020
Year: 2020

We consider three variants of the problem of finding a maximum weight restricted $2$-matching in a subcubic graph $G$. (A $2$-matching is any subset of the edges such that each vertex is incident to at most two of its edges.) Depending on the variant a restricted $2$-matching means a $2$-matching that is either triangle-free or square-free or both triangle- and square-free. While there exist polynomial time algorithms for the first two types of $2$-matchings, they are quite complicated or use advanced methodology. For each of the three problems we present a simple reduction to the computation of a maximum weight $b$-matching. The reduction is conducted with the aid of half-edges. A half-edge of edge $e$ is, informally speaking, a half of $e$ containing exactly one of its endpoints. For a subset of triangles of $G$, we replace each edge of such a triangle with two half-edges. Two half-edges of one edge $e$ of weight $w(e)$ may get different weights, not necessarily equal to $\frac{1}{2}w(e)$. In the metric setting when the edge weights satisfy the triangle inequality, this has a geometric interpretation connected to how an incircle partitions the edges of a triangle. Our algorithms are additionally faster than those known before. The running time of each of them is $O(n^2\log{n})$, where $n$ denotes the number of vertices in the graph.
Explainability Matters: Backdoor Attacks on Medical Imaging
Venue: arXiv cs.CR 2020
Year: 2020

Deep neural networks have been shown to be vulnerable to backdoor attacks, which could be easily introduced to the training set prior to model training. Recent work has focused on investigating backdoor attacks on natural images or toy datasets. Consequently, the exact impact of backdoors is not yet fully understood in complex real-world applications, such as in medical imaging where misdiagnosis can be very costly. In this paper, we explore the impact of backdoor attacks on a multi-label disease classification task using chest radiography, with the assumption that the attacker can manipulate the training dataset to execute the attack. Extensive evaluation of a state-of-the-art architecture demonstrates that by introducing images with few-pixel perturbations into the training set, an attacker can execute the backdoor successfully without having to be involved with the training procedure. A simple 3$\times$3 pixel trigger can achieve up to 1.00 Area Under the Receiver Operating Characteristic (AUROC) curve on the set of infected images. In the set of clean images, the backdoored neural network could still achieve up to 0.85 AUROC, highlighting the stealthiness of the attack. As the use of deep learning based diagnostic systems proliferates in clinical practice, we also show how explainability is indispensable in this context, as it can identify spatially localized backdoors in inference time.
Analysis of a new chaotic system, electronic realization and use in navigation of differential drive mobile robot
Venue: arXiv cs.RO 2020
Year: 2020

This paper presents a new chaotic system having four attractors, including two fixed point attractors and two symmetrical chaotic strange attractors. Dynamical properties of the system, viz. sensitive dependence on initial conditions, Lyapunov spectrum, strangeness measure, attraction basin, including the class and size of it, existence of strange attractor, bifurcation analysis, multistability, electronic circuit design, and hardware implementation, are rigorously treated. Numerical computations are used to compute the basin of attraction and show that the system has a far-reaching composite basin of attraction. Such a basin of attraction is vital for engineering applications. Moreover, a circuit model of the system is realized using analog electronic components. A procedure is detailed for converting the system parameters into corresponding electronic component values such as the circuital resistances while ensuring the dynamic ranges are bounded. Besides, the system is used as the source of control inputs for independent navigation of a differential drive mobile robot, which is subject to the Pfaffian velocity constraint. Due to the properties of sensitivity on initial conditions and topological mixing, the robot's path becomes unpredictable and guaranteed to scan the workspace, respectively.
Scene Text Detection for Augmented Reality -- Character Bigram Approach to reduce False Positive Rate
Venue: arXiv cs.CV 2020
Year: 2020

Natural scene text detection is an important aspect of scene understanding and could be a useful tool in building engaging augmented reality applications. In this work, we address the problem of false positives in text spotting. We propose improving the performace of sliding window text spotters by looking for character pairs (bigrams) rather than single characters. An efficient convolutional neural network is designed and trained to detect bigrams. The proposed detector reduces false positive rate by 28.16% on the ICDAR 2015 dataset. We demonstrate that detecting bigrams is a computationally inexpensive way to improve sliding window text spotters.
Advanced Machine Learning Techniques for Fake News (Online Disinformation) Detection: A Systematic Mapping Study
Venue: arXiv cs.CL 2020
Year: 2020

Fake news has now grown into a big problem for societies and also a major challenge for people fighting disinformation. This phenomenon plagues democratic elections, reputations of individual persons or organizations, and has negatively impacted citizens, (e.g., during the COVID-19 pandemic in the US or Brazil). Hence, developing effective tools to fight this phenomenon by employing advanced Machine Learning (ML) methods poses a significant challenge. The following paper displays the present body of knowledge on the application of such intelligent tools in the fight against disinformation. It starts by showing the historical perspective and the current role of fake news in the information war. Proposed solutions based solely on the work of experts are analysed and the most important directions of the application of intelligent systems in the detection of misinformation sources are pointed out. Additionally, the paper presents some useful resources (mainly datasets useful when assessing ML solutions for fake news detection) and provides a short overview of the most important R&D projects related to this subject. The main purpose of this work is to analyse the current state of knowledge in detecting fake news; on the one hand to show possible solutions, and on the other hand to identify the main challenges and methodological gaps to motivate future research.
An Ecosystem Approach to Ethical AI and Data Use: Experimental Reflections
Venue: arXiv cs.CY 2020
Year: 2020

While we have witnessed a rapid growth of ethics documents meant to guide AI development, the promotion of AI ethics has nonetheless proceeded with little input from AI practitioners themselves. Given the proliferation of AI for Social Good initiatives, this is an emerging gap that needs to be addressed in order to develop more meaningful ethical approaches to AI use and development. This paper offers a methodology, a shared fairness approach, aimed at identifying the needs of AI practitioners when it comes to confronting and resolving ethical challenges and to find a third space where their operational language can be married with that of the more abstract principles that presently remain at the periphery of their work experiences. We offer a grassroots approach to operational ethics based on dialog and mutualised responsibility. This methodology is centred around conversations intended to elicit practitioners perceived ethical attribution and distribution over key value laden operational decisions, to identify when these decisions arise and what ethical challenges they confront, and to engage in a language of ethics and responsibility which enables practitioners to internalise ethical responsibility. The methodology bridges responsibility imbalances that rest in structural decision making power and elite technical knowledge, by commencing with personal, facilitated conversations, returning the ethical discourse to those meant to give it meaning at the sharp end of the ecosystem. Our primary contribution is to add to the recent literature seeking to bring AI practitioners' experiences to the fore by offering a methodology for understanding how ethics manifests as a relational and interdependent sociotechnical practice in their work.
Predicting Individual Substance Abuse Vulnerability using Machine Learning Techniques
Venue: arXiv cs.LG 2020
Year: 2020

Substance abuse is the unrestrained and detrimental use of psychoactive chemical substances, unauthorized drugs, and alcohol. Continuous use of these substances can ultimately lead a human to disastrous consequences. As patients display a high rate of relapse, prevention at an early stage can be an effective restraint. We therefore propose a binary classifier to identify any individual's present vulnerability towards substance abuse by analyzing subjects' socio-economic environment. We have collected data by a questionnaire which is created after carefully assessing the commonly involved factors behind substance abuse. Pearson's chi-squared test of independence is used to identify key feature variables influencing substance abuse. Later we build the predictive classifiers using machine learning classification algorithms on those variables. Logistic regression classifier trained with 18 features can predict individual vulnerability with the best accuracy.
Neural Network Coding
Venue: arXiv cs.IT 2020
Year: 2020

In this paper we introduce Neural Network Coding(NNC), a data-driven approach to joint source and network coding. In NNC, the encoders at each source and intermediate node, as well as the decoder at each destination node, are neural networks which are all trained jointly for the task of communicating correlated sources through a network of noisy point-to-point links. The NNC scheme is application-specific and makes use of a training set of data, instead of making assumptions on the source statistics. In addition, it can adapt to any arbitrary network topology and power constraint. We show empirically that, for the task of transmitting MNIST images over a network, the NNC scheme shows improvement over baseline schemes, especially in the low-SNR regime.
Approximations with deep neural networks in Sobolev time-space
Venue: arXiv cs.LG 2020
Year: 2020

Solutions of evolution equation generally lies in certain Bochner-Sobolev spaces, in which the solution may has regularity and integrability properties for the time variable that can be different for the space variables. Therefore, in this paper, we develop a framework shows that deep neural networks can approximate Sobolev-regular functions with respect to Bochner-Sobolev spaces. In our work we use the so-called Rectified Cubic Unit (ReCU) as an activation function in our networks, which allows us to deduce approximation results of the neural networks while avoiding issues caused by the non regularity of the most commonly used Rectivied Linear Unit (ReLU) activation function.