OpenRaiser/Intern-Atlas huggingface dataset:
Improving Model Accuracy for Imbalanced Image Classification Tasks by Adding a Final Batch Normalization Layer: An Empirical Study
Venue: arXiv cs.CV 2020
Year: 2020

Some real-world domains, such as Agriculture and Healthcare, comprise early-stage disease indications whose recording constitutes a rare event, and yet, whose precise detection at that stage is critical. In this type of highly imbalanced classification problems, which encompass complex features, deep learning (DL) is much needed because of its strong detection capabilities. At the same time, DL is observed in practice to favor majority over minority classes and consequently suffer from inaccurate detection of the targeted early-stage indications. To simulate such scenarios, we artificially generate skewness (99% vs. 1%) for certain plant types out of the PlantVillage dataset as a basis for classification of scarce visual cues through transfer learning. By randomly and unevenly picking healthy and unhealthy samples from certain plant types to form a training set, we consider a base experiment as fine-tuning ResNet34 and VGG19 architectures and then testing the model performance on a balanced dataset of healthy and unhealthy images. We empirically observe that the initial F1 test score jumps from 0.29 to 0.95 for the minority class upon adding a final Batch Normalization (BN) layer just before the output layer in VGG19. We demonstrate that utilizing an additional BN layer before the output layer in modern CNN architectures has a considerable impact in terms of minimizing the training time and testing error for minority classes in highly imbalanced data sets. Moreover, when the final BN is employed, minimizing the loss function may not be the best way to assure a high F1 test score for minority classes in such problems. That is, the network might perform better even if it is not confident enough while making a prediction; leading to another discussion about why softmax output is not a good uncertainty measure for DL models.
SigmaLaw-ABSA: Dataset for Aspect-Based Sentiment Analysis in Legal Opinion Texts
Venue: arXiv cs.CL 2020
Year: 2020

Aspect-Based Sentiment Analysis (ABSA) has been prominent and ongoing research over many different domains, but it is not widely discussed in the legal domain. A number of publicly available datasets for a wide range of domains usually fulfill the needs of researchers to perform their studies in the field of ABSA. To the best of our knowledge, there is no publicly available dataset for the Aspect (Party) Based Sentiment Analysis for legal opinion texts. Therefore, creating a publicly available dataset for the research of ABSA for the legal domain can be considered as a task with significant importance. In this study, we introduce a manually annotated legal opinion text dataset (SigmaLaw-ABSA) intended towards facilitating researchers for ABSA tasks in the legal domain. SigmaLaw-ABSA consists of legal opinion texts in the English language which have been annotated by human judges. This study discusses the sub-tasks of ABSA relevant to the legal domain and how to use the dataset to perform them. This paper also describes the statistics of the dataset and as a baseline, we present some results on the performance of some existing deep learning based systems on the SigmaLaw-ABSA dataset.
Identifying influential nodes in complex networks: Effective distance gravity model
Venue: arXiv cs.SI 2020
Year: 2020

The identification of important nodes in complex networks is an area of exciting growth due to its applications across various disciplines like disease controlling, community finding, data mining, network system controlling, just to name a few. Many measures have thus been proposed to date, and these measures are either based on the locality of nodes or the global nature of the network. These measures typically use distance based on the concept of traditional Euclidean Distance, which only focus on the local static geographic distance between nodes but ignore the interaction between nodes in real-world networks. However, a variety of factors should be considered for the purpose of identifying influential nodes, such as degree, edge, direction and weight. Some methods based on evidence theory have also been proposed. In this paper, we have proposed an original and novel gravity model with effective distance for identifying influential nodes based on information fusion and multi-level processing. Our method is able to comprehensively consider the global and local information of the complex network, and also utilize the effective distance to replace the Euclidean Distance. This allows us to fully consider the complex topological structure of the real network, as well as the dynamic interaction information between nodes. In order to validate the effectiveness of our proposed method, we have utilized the susceptible infected (SI) model to carry out a variety of simulations on eight different real-world networks using six existing well-known methods. The experimental results indicate the reasonableness and effectiveness of our proposed method.
Customizing Trusted AI Accelerators for Efficient Privacy-Preserving Machine Learning
Venue: arXiv cs.AR 2020
Year: 2020

The use of trusted hardware has become a promising solution to enable privacy-preserving machine learning. In particular, users can upload their private data and models to a hardware-enforced trusted execution environment (e.g. an enclave in Intel SGX-enabled CPUs) and run machine learning tasks in it with confidentiality and integrity guaranteed. To improve performance, AI accelerators have been widely employed for modern machine learning tasks. However, how to protect privacy on an AI accelerator remains an open question. To address this question, we propose a solution for efficient privacy-preserving machine learning based on an unmodified trusted CPU and a customized trusted AI accelerator. We carefully leverage cryptographic primitives to establish trust and protect the channel between the CPU and the accelerator. As a case study, we demonstrate our solution based on the open-source versatile tensor accelerator. The result of evaluation shows that the proposed solution provides efficient privacy-preserving machine learning at a small design cost and moderate performance overhead.
Fast robust peg-in-hole insertion with continuous visual servoing
Venue: arXiv cs.RO 2020
Year: 2020

This paper demonstrates a visual servoing method which is robust towards uncertainties related to system calibration and grasping, while significantly reducing the peg-in-hole time compared to classical methods and recent attempts based on deep learning. The proposed visual servoing method is based on peg and hole point estimates from a deep neural network in a multi-cam setup, where the model is trained on purely synthetic data. Empirical results show that the learnt model generalizes to the real world, allowing for higher success rates and lower cycle times than existing approaches.
Same Object, Different Grasps: Data and Semantic Knowledge for Task-Oriented Grasping
Venue: arXiv cs.RO 2020
Year: 2020

Despite the enormous progress and generalization in robotic grasping in recent years, existing methods have yet to scale and generalize task-oriented grasping to the same extent. This is largely due to the scale of the datasets both in terms of the number of objects and tasks studied. We address these concerns with the TaskGrasp dataset which is more diverse both in terms of objects and tasks, and an order of magnitude larger than previous datasets. The dataset contains 250K task-oriented grasps for 56 tasks and 191 objects along with their RGB-D information. We take advantage of this new breadth and diversity in the data and present the GCNGrasp framework which uses the semantic knowledge of objects and tasks encoded in a knowledge graph to generalize to new object instances, classes and even new tasks. Our framework shows a significant improvement of around 12% on held-out settings compared to baseline methods which do not use semantics. We demonstrate that our dataset and model are applicable for the real world by executing task-oriented grasps on a real robot on unknown objects. Code, data and supplementary video could be found at https://sites.google.com/view/taskgrasp
Communication Efficient Coresets for Maximum Matching
Venue: arXiv cs.DS 2020
Year: 2020

In this paper we revisit the problem of constructing randomized composable coresets for bipartite matching. In this problem the input graph is randomly partitioned across $k$ players, each of which sends a single message to a coordinator, who then must output a good approximation to the maximum matching in the input graph. Assadi and Khanna gave the first such coreset, achieving a $1/9$-approximation by having every player send a maximum matching, i.e. at most $n/2$ words per player. The approximation factor was improved to $1/3$ by Bernstein et al. In this paper, we show that the matching skeleton construction of Goel, Kapralov and Khanna, which is a carefully chosen (fractional) matching, is a randomized composable coreset that achieves a $1/2-o(1)$ approximation using at most $n-1$ words of communication per player. We also show an upper bound of $2/3+o(1)$ on the approximation ratio achieved by this coreset.
On the Performance of Convolutional Neural Networks under High and Low Frequency Information
Venue: arXiv cs.CV 2020
Year: 2020

Convolutional neural networks (CNNs) have shown very promising performance in recent years for different problems, including object recognition, face recognition, medical image analysis, etc. However, generally the trained CNN models are tested over the test set which is very similar to the trained set. The generalizability and robustness of the CNN models are very important aspects to make it to work for the unseen data. In this letter, we study the performance of CNN models over the high and low frequency information of the images. We observe that the trained CNN fails to generalize over the high and low frequency images. In order to make the CNN robust against high and low frequency images, we propose the stochastic filtering based data augmentation during training. A satisfactory performance improvement has been observed in terms of the high and low frequency generalization and robustness with the proposed stochastic filtering based data augmentation approach. The experimentations are performed using ResNet50 model over the CIFAR-10 dataset and ResNet101 model over Tiny-ImageNet dataset.
Quality4.0 -- Transparent product quality supervision in the age of Industry 4.0
Venue: arXiv cs.AI 2020
Year: 2020

Progressive digitalization is changing the game of many industrial sectors. Focus-ing on product quality the main profitability driver of this so-called Industry 4.0 will be the horizontal integration of information over the complete supply chain. Therefore, the European RFCS project 'Quality4.0' aims in developing an adap-tive platform, which releases decisions on product quality and provides tailored information of high reliability that can be individually exchanged with customers. In this context Machine Learning will be used to detect outliers in the quality data. This paper discusses the intermediate project results and the concepts developed so far for this horizontal integration of quality information.
Linear Dilation-Erosion Perceptron Trained Using a Convex-Concave Procedure
Venue: arXiv cs.LG 2020
Year: 2020

Mathematical morphology (MM) is a theory of non-linear operators used for the processing and analysis of images. Morphological neural networks (MNNs) are neural networks whose neurons compute morphological operators. Dilations and erosions are the elementary operators of MM. From an algebraic point of view, a dilation and an erosion are operators that commute respectively with the supremum and infimum operations. In this paper, we present the \textit{linear dilation-erosion perceptron} ($\ell$-DEP), which is given by applying linear transformations before computing a dilation and an erosion. The decision function of the $\ell$-DEP model is defined by adding a dilation and an erosion. Furthermore, training a $\ell$-DEP can be formulated as a convex-concave optimization problem. We compare the performance of the $\ell$-DEP model with other machine learning techniques using several classification problems. The computational experiments support the potential application of the proposed $\ell$-DEP model for binary classification tasks.
Self-supervised reinforcement learning for speaker localisation with the iCub humanoid robot
Venue: arXiv cs.RO 2020
Year: 2020

In the future robots will interact more and more with humans and will have to communicate naturally and efficiently. Automatic speech recognition systems (ASR) will play an important role in creating natural interactions and making robots better companions. Humans excel in speech recognition in noisy environments and are able to filter out noise. Looking at a person's face is one of the mechanisms that humans rely on when it comes to filtering speech in such noisy environments. Having a robot that can look toward a speaker could benefit ASR performance in challenging environments. To this aims, we propose a self-supervised reinforcement learning-based framework inspired by the early development of humans to allow the robot to autonomously create a dataset that is later used to learn to localize speakers with a deep learning network.
Noncoherent Multiuser Chirp Spread Spectrum: Performance with Doppler and Asynchronism
Venue: arXiv cs.IT 2020
Year: 2020

In this paper, we investigate multi user chirp spread spectrum with noncoherent detection as a continuation of our work on coherent detection in [1]. We derive the analytical bit error ratio (BER) expression for binary chirp spread spectrum (BCSS) in the presence of multiple access interference (MAI) caused by correlation with other user signals because of either asynchronism or Doppler shifts, or both, and validate with simulations. To achieve this we analyze the signal cross correlations, and compare traditional linear chirps with our recently-proposed nonlinear chirps introduced in [1] and with other nonlinear chirps from the literature. In doing so we illustrate the superior performance of our new nonlinear chirp designs in these practical conditions, for the noncoherent counterpart of [1].
Functorial Semantics for Partial Theories
Venue: arXiv cs.LO 2020
Year: 2020

We provide a Lawvere-style definition for partial theories, extending the classical notion of equational theory by allowing partially defined operations. As in the classical case, our definition is syntactic: we use an appropriate class of string diagrams as terms. This allows for equational reasoning about the class of models defined by a partial theory. We demonstrate the expressivity of such equational theories by considering a number of examples, including partial combinatory algebras and cartesian closed categories. Moreover, despite the increase in expressivity of the syntax we retain a well-behaved notion of semantics: we show that our categories of models are precisely locally finitely presentable categories, and that free models exist.
Utilizing Ensemble Learning for Performance and Power Modeling and Improvement of Parallel Cancer Deep Learning CANDLE Benchmarks
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning (ML) continues to grow in importance across nearly all domains and is a natural tool in modeling to learn from data. Often a tradeoff exists between a model's ability to minimize bias and variance. In this paper, we utilize ensemble learning to combine linear, nonlinear, and tree-/rule-based ML methods to cope with the bias-variance tradeoff and result in more accurate models. Hardware performance counter values are correlated with properties of applications that impact performance and power on the underlying system. We use the datasets collected for two parallel cancer deep learning CANDLE benchmarks, NT3 (weak scaling) and P1B2 (strong scaling), to build performance and power models based on hardware performance counters using single-object and multiple-objects ensemble learning to identify the most important counters for improvement. Based on the insights from these models, we improve the performance and energy of P1B2 and NT3 by optimizing the deep learning environments TensorFlow, Keras, Horovod, and Python under the huge page size of 8 MB on the Cray XC40 Theta at Argonne National Laboratory. Experimental results show that ensemble learning not only produces more accurate models but also provides more robust performance counter ranking. We achieve up to 61.15% performance improvement and up to 62.58% energy saving for P1B2 and up to 55.81% performance improvement and up to 52.60% energy saving for NT3 on up to 24,576 cores.
Performance and Power Modeling and Prediction Using MuMMI and Ten Machine Learning Methods
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we use modeling and prediction tool MuMMI (Multiple Metrics Modeling Infrastructure) and ten machine learning methods to model and predict performance and power and compare their prediction error rates. We use a fault-tolerant linear algebra code and a fault-tolerant heat distribution code to conduct our modeling and prediction study on the Cray XC40 Theta and IBM BG/Q Mira at Argonne National Laboratory and the Intel Haswell cluster Shepard at Sandia National Laboratories. Our experiment results show that the prediction error rates in performance and power using MuMMI are less than 10% for most cases. Based on the models for runtime, node power, CPU power, and memory power, we identify the most significant performance counters for potential optimization efforts associated with the application characteristics and the target architectures, and we predict theoretical outcomes of the potential optimizations. When we compare the prediction accuracy using MuMMI with that using 10 machine learning methods, we observe that MuMMI not only results in more accurate prediction in both performance and power but also presents how performance counters impact the performance and power models. This provides some insights about how to fine-tune the applications and/or systems for energy efficiency.
Communication complexity of Nash equilibrium in potential games
Venue: arXiv cs.GT 2020
Year: 2020

We prove communication complexity lower bounds for (possibly mixed) Nash equilibrium in potential games. In particular, we show that finding a Nash equilibrium requires $poly(N)$ communication in two-player $N \times N$ potential games, and $2^{poly(n)}$ communication in $n$-player two-action games. To the best of our knowledge, these are the first results to demonstrate hardness in any model of (possibly mixed) Nash equilibrium in potential games.
Active Reinforcement Learning: Observing Rewards at a Cost
Venue: arXiv cs.LG 2020
Year: 2020

Active reinforcement learning (ARL) is a variant on reinforcement learning where the agent does not observe the reward unless it chooses to pay a query cost c > 0. The central question of ARL is how to quantify the long-term value of reward information. Even in multi-armed bandits, computing the value of this information is intractable and we have to rely on heuristics. We propose and evaluate several heuristic approaches for ARL in multi-armed bandits and (tabular) Markov decision processes, and discuss and illustrate some challenging aspects of the ARL problem.
iHorology: Lowering the Barrier to Microsecond-level Internet Time
Venue: arXiv cs.NI 2020
Year: 2020

High precision, synchronized clocks are essential to a growing number of Internet applications. Standard protocols and their associated server infrastructure have been shown to typically enable client clocks to synchronize on the order of tens of milliseconds. We address one of the key challenges to high precision Internet timekeeping - the intrinsic contribution to clock error of path asymmetry between client and time server, a fundamental barrier to microsecond level accuracy. We first exploit results of a measurement study to quantify asymmetry and its effect on timing. We then describe three approaches to addressing the path asymmetry problem: LBBE, SBBE and K-SBBE, each based on timestamp exchange with multiple servers, with the goal of tightening bounds on asymmetry for each client. We explore their capabilities and limitations through simulation and argument. We show that substantial improvements are possible, and discuss whether, and how, the goal of microsecond accuracy might be attained.
A Survey on Recent Advances in Sequence Labeling from Deep Learning Models
Venue: arXiv cs.CL 2020
Year: 2020

Sequence labeling (SL) is a fundamental research problem encompassing a variety of tasks, e.g., part-of-speech (POS) tagging, named entity recognition (NER), text chunking, etc. Though prevalent and effective in many downstream applications (e.g., information retrieval, question answering, and knowledge graph embedding), conventional sequence labeling approaches heavily rely on hand-crafted or language-specific features. Recently, deep learning has been employed for sequence labeling tasks due to its powerful capability in automatically learning complex features of instances and effectively yielding the stat-of-the-art performances. In this paper, we aim to present a comprehensive review of existing deep learning-based sequence labeling models, which consists of three related tasks, e.g., part-of-speech tagging, named entity recognition, and text chunking. Then, we systematically present the existing approaches base on a scientific taxonomy, as well as the widely-used experimental datasets and popularly-adopted evaluation metrics in the SL domain. Furthermore, we also present an in-depth analysis of different SL models on the factors that may affect the performance and future directions in the SL domain.
Investigating Learning in Deep Neural Networks using Layer-Wise Weight Change
Venue: arXiv cs.LG 2020
Year: 2020

Understanding the per-layer learning dynamics of deep neural networks is of significant interest as it may provide insights into how neural networks learn and the potential for better training regimens. We investigate learning in Deep Convolutional Neural Networks (CNNs) by measuring the relative weight change of layers while training. Several interesting trends emerge in a variety of CNN architectures across various computer vision classification tasks, including the overall increase in relative weight change of later layers as compared to earlier ones.
Metric-Free Individual Fairness with Cooperative Contextual Bandits
Venue: arXiv cs.LG 2020
Year: 2020

Data mining algorithms are increasingly used in automated decision making across all walks of daily life. Unfortunately, as reported in several studies these algorithms inject bias from data and environment leading to inequitable and unfair solutions. To mitigate bias in machine learning, different formalizations of fairness have been proposed that can be categorized into group fairness and individual fairness. Group fairness requires that different groups should be treated similarly which might be unfair to some individuals within a group. On the other hand, individual fairness requires that similar individuals be treated similarly. However, individual fairness remains understudied due to its reliance on problem-specific similarity metrics. We propose a metric-free individual fairness and a cooperative contextual bandits (CCB) algorithm. The CCB algorithm utilizes fairness as a reward and attempts to maximize it. The advantage of treating fairness as a reward is that the fairness criterion does not need to be differentiable. The proposed algorithm is tested on multiple real-world benchmark datasets. The results show the effectiveness of the proposed algorithm at mitigating bias and at achieving both individual and group fairness.