OpenRaiser/Intern-Atlas huggingface dataset:
pCAMP: Performance Comparison of Machine Learning Packages on the Edges
Venue: arXiv cs.DC 2019
Year: 2019

Machine learning has changed the computing paradigm. Products today are built with machine intelligence as a central attribute, and consumers are beginning to expect near-human interaction with the appliances they use. However, much of the deep learning revolution has been limited to the cloud. Recently, several machine learning packages based on edge devices have been announced which aim to offload the computing to the edges. However, little research has been done to evaluate these packages on the edges, making it difficult for end users to select an appropriate pair of software and hardware. In this paper, we make a performance comparison of several state-of-the-art machine learning packages on the edges, including TensorFlow, Caffe2, MXNet, PyTorch, and TensorFlow Lite. We focus on evaluating the latency, memory footprint, and energy of these tools with two popular types of neural networks on different edge devices. This evaluation not only provides a reference to select appropriate combinations of hardware and software packages for end users but also points out possible future directions to optimize packages for developers.
AI-Skin : Skin Disease Recognition based on Self-learning and Wide Data Collection through a Closed Loop Framework
Venue: arXiv cs.CV 2019
Year: 2019

There are a lot of hidden dangers in the change of human skin conditions, such as the sunburn caused by long-time exposure to ultraviolet radiation, which not only has aesthetic impact causing psychological depression and lack of self-confidence, but also may even be life-threatening due to skin canceration. Current skin disease researches adopt the auto-classification system for improving the accuracy rate of skin disease classification. However, the excessive dependence on the image sample database is unable to provide individualized diagnosis service for different population groups. To overcome this problem, a medical AI framework based on data width evolution and self-learning is put forward in this paper to provide skin disease medical service meeting the requirement of real time, extendibility and individualization. First, the wide collection of data in the close-loop information flow of user and remote medical data center is discussed. Next, a data set filter algorithm based on information entropy is given, to lighten the load of edge node and meanwhile improve the learning ability of remote cloud analysis model. In addition, the framework provides an external algorithm load module, which can be compatible with the application requirements according to the model selected. Three kinds of deep learning model, i.e. LeNet-5, AlexNet and VGG16, are loaded and compared, which have verified the universality of the algorithm load module. The experiment platform for the proposed real-time, individualized and extensible skin disease recognition system is built. And the system's computation and communication delay under the interaction scenario between tester and remote data center are analyzed. It is demonstrated that the system we put forward is reliable and effective.
Corn leaf detection using Region based convolutional neural network
Venue: arXiv cs.CV 2019
Year: 2019

The field of machine learning has become an increasingly budding area of research as more efficient methods are needed in the quest to handle more complex image detection challenges. To solve the problems of agriculture is more and more important because food is the fundamental of life. However, the detection accuracy in recent corn field systems are still far away from the demands in practice due to a number of different weeds. This paper presents a model to handle the problem of corn leaf detection in given digital images collected from farm field. Based on results of experiments conducted with several state-of-the-art models adopted by CNN, a region-based method has been proposed as a faster and more accurate method of corn leaf detection. Being motivated with such unique attributes of ResNet, we combine it with region based network (such as faster rcnn), which is able to automatically detect corn leaf in heavy weeds occlusion. The method is evaluated on the dataset from farm and we make an annotation ourselves. Our proposed method achieves significantly outperform in corn detection system.
Machine Learning and System Identification for Estimation in Physical Systems
Venue: arXiv cs.LG 2019
Year: 2019

In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear modeling power while keeping data requirements practical. The thesis covers a wide range of applications, many inspired by applications within robotics, but also extending outside this already wide field. Usage of the proposed methods and algorithms are in many cases illustrated in the real-world applications that motivated the research. Topics covered include dynamics modeling and estimation, model-based reinforcement learning, spectral estimation, friction modeling and state estimation and calibration in robotic machining. In the work on modeling and identification of dynamics, we develop regularization strategies that allow us to incorporate prior domain knowledge into flexible, overparameterized models. We make use of classical control theory to gain insight into training and regularization while using flexible tools from modern deep learning. A particular focus of the work is to allow use of modern methods in scenarios where gathering data is associated with a high cost. In the robotics-inspired parts of the thesis, we develop methods that are practically motivated and ensure that they are implementable also outside the research setting. We demonstrate this by performing experiments in realistic settings and providing open-source implementations of all proposed methods and algorithms.
Visual Confusion Label Tree For Image Classification
Venue: arXiv cs.CV 2019
Year: 2019

Convolution neural network models are widely used in image classification tasks. However, the running time of such models is so long that it is not the conforming to the strict real-time requirement of mobile devices. In order to optimize models and meet the requirement mentioned above, we propose a method that replaces the fully-connected layers of convolution neural network models with a tree classifier. Specifically, we construct a Visual Confusion Label Tree based on the output of the convolution neural network models, and use a multi-kernel SVM plus classifier with hierarchical constraints to train the tree classifier. Focusing on those confusion subsets instead of the entire set of categories makes the tree classifier more discriminative and the replacement of the fully-connected layers reduces the original running time. Experiments show that our tree classifier obtains a significant improvement over the state-of-the-art tree classifier by 4.3% and 2.4% in terms of top-1 accuracy on CIFAR-100 and ImageNet datasets respectively. Additionally, our method achieves 124x and 115x speedup ratio compared with fully-connected layers on AlexNet and VGG16 without accuracy decline.
KAS-term: Extracting Slovene Terms from Doctoral Theses via Supervised Machine Learning
Venue: arXiv cs.CL 2019
Year: 2019

This paper presents a dataset and supervised learning experiments for term extraction from Slovene academic texts. Term candidates in the dataset were extracted via morphosyntactic patterns and annotated for their termness by four annotators. Experiments on the dataset show that most co-occurrence statistics, applied after morphosyntactic patterns and a frequency threshold, perform close to random and that the results can be significantly improved by combining, with supervised machine learning, all the seven statistic measures included in the dataset. On multi-word terms the model using all statistics obtains an AUC of 0.736 while the best single statistic produces only AUC 0.590. Among many additional candidate features, only adding multi-word morphosyntactic pattern information and length of the single-word term candidates achieves further improvements of the results.
On Low Complexity RLL Code for Visible Light Communication
Venue: arXiv cs.IT 2019
Year: 2019

Run length limited (RLL) codes are used in visible light communication (VLC) to avoid flicker and to support different dimming ranges such that communication is not affected by the variation in light intensity. In this paper, we propose a low complexity split phase code as RLL code in serial concatenation with the convolutional code as a forward error correcting code (FEC) for VLC. The split phase code can be represented by a two-state trellis-like code which can provide an iterative gain in FEC-RLL serial concatenation. We also use the extrinsic information transfer (EXIT) chart to explain the iterative decoding behavior of the proposed serial concatenated scheme. Furthermore, we use puncturing and compensation symbols to support various dimming range in VLC.
Artificial Intelligence in Clinical Health Care Applications: Viewpoint
Venue: arXiv cs.CY 2019
Year: 2019

The idea of Artificial Intelligence (AI) has a long history. It turned out, however, that reaching intelligence at human levels is more complicated than originally anticipated. Currently we are experiencing a renewed interest in AI, fueled by an enormous increase in computing power and an even larger increase in data, in combination with improved AI technologies like deep learning. Healthcare is considered the next domain to be revolutionized by Artificial Intelligence. While AI approaches are excellently suited to develop certain algorithms, for biomedical applications there are specific challenges. We propose recommendations to improve AI projects in the biomedical space and especially clinical healthcare.
Extractive Summarization via Weighted Dissimilarity and Importance Aligned Key Iterative Algorithm
Venue: arXiv cs.CL 2019
Year: 2019

We present importance aligned key iterative algorithm for extractive summarization that is faster than conventional algorithms keeping its accuracy. The computational complexity of our algorithm is O($SNlogN$) to summarize original $N$ sentences into final $S$ sentences. Our algorithm maximizes the weighted dissimilarity defined by the product of importance and cosine dissimilarity so that the summary represents the document and at the same time the sentences of the summary are not similar to each other. The weighted dissimilarity is heuristically maximized by iterative greedy search and binary search to the sentences ordered by importance. We finally show a benchmark score based on summarization of customer reviews of products, which highlights the quality of our algorithm comparable to human and existing algorithms. We provide the source code of our algorithm on github https://github.com/qhapaq-49/imakita .
Theme-aware generation model for chinese lyrics
Venue: arXiv cs.CL 2019
Year: 2019

With rapid development of neural networks, deep-learning has been extended to various natural language generation fields, such as machine translation, dialogue generation and even literature creation. In this paper, we propose a theme-aware language generation model for Chinese music lyrics, which improves the theme-connectivity and coherence of generated paragraphs greatly. A multi-channel sequence-to-sequence (seq2seq) model encodes themes and previous sentences as global and local contextual information. Moreover, attention mechanism is incorporated for sequence decoding, enabling to fuse context into predicted next texts. To prepare appropriate train corpus, LDA (Latent Dirichlet Allocation) is applied for theme extraction. Generated lyrics is grammatically correct and semantically coherent with selected themes, which offers a valuable modelling method in other fields including multi-turn chatbots, long paragraph generation and etc.
Towards Business Partnership Recommendation Using User Opinion on Facebook
Venue: arXiv cs.SI 2019
Year: 2019

The identification of strategic business partnerships can potentially provide competitive advantages for businesses; however, due to the dynamics and uncertainty present in business environments, this task could be challenging. To help businesses in this task, this study presents a similarity model between businesses that consider the opinions of users on content shared by businesses on social media. Thus, this model captures significant virtual relationships among businesses that are generated by users in the virtual world. Besides, we propose an algorithm for detecting business communities in the considered model. We also propose an algorithm to identify possible business outliers in the detected communities, which could represent an automatic way to identify non-obvious relations that might deserve particular attention of business owners. By exploring approximately 280 million user reactions on Facebook, we show that our results could favor the development of, for example, a new strategic business partnership recommendation service.
Non-Linear Estimation of Convolutionally Encoded Sequences
Venue: arXiv cs.IT 2019
Year: 2019

Suppose that a convolutionally encoded sequence is transmitted symbol by symbol over an AWGN channel using BPSK modulation. In this case, pairs of the signal (i.e., code symbol) and observation are not jointly Gaussian and therefore, a linear estimation method cannot be applied. Hence, in this paper, non-linear estimation of convolutionally encoded sequences is discussed. First a probability measure (denoted Q), whose Radon-Nikodym derivative with respect to the underlying probability measure P is an exponential martingale, is constructed. It is shown that with respect to Q, the observations are mutually independent Gaussian random vectors with zero mean and identity covariance matrix. We see that the relationship between observation noises (with respect to P) and observations (with respect to Q) has a close relation to the Girsanov theorem in continuous case. Next, using the probability measure Q, we calculate the conditional probability of an event related to any encoded symbol conditioned by the observations. Moreover, we transform it into a recursive form. In the process of derivation, the metric associated with an encoded sequence comes out in a natural way. Finally, it is shown that maximum a posteriori probability (MAP) decoding of convolutional codes is realized using the derived conditional probability.
Comprehensive Personalized Ranking Using One-Bit Comparison Data
Venue: arXiv cs.IR 2019
Year: 2019

The task of a personalization system is to recommend items or a set of items according to the users' taste, and thus predicting their future needs. In this paper, we address such personalized recommendation problems for which one-bit comparison data of user preferences for different items as well as the different user inclinations toward an item are available. We devise a comprehensive personalized ranking (CPR) system by employing a Bayesian treatment. We also provide a connection to the learning method with respect to the CPR optimization criterion to learn the underlying low-rank structure of the rating matrix based on the well-established matrix factorization method. Numerical results are provided to verify the performance of our algorithm.
Extreme Points Derived Confidence Map as a Cue For Class-Agnostic Segmentation Using Deep Neural Network
Venue: arXiv cs.CV 2019
Year: 2019

To automate the process of segmenting an anatomy of interest, we can learn a model from previously annotated data. The learning-based approach uses annotations to train a model that tries to emulate the expert labeling on a new data set. While tremendous progress has been made using such approaches, labeling of medical images remains a time-consuming and expensive task. In this paper, we evaluate the utility of extreme points in learning to segment. Specifically, we propose a novel approach to compute a confidence map from extreme points that quantitatively encodes the priors derived from extreme points. We use the confidence map as a cue to train a deep neural network based on ResNet-101 and PSP module to develop a class-agnostic segmentation model that outperforms state-of-the-art method that employs extreme points as a cue. Further, we evaluate a realistic use-case by using our model to generate training data for supervised learning (U-Net) and observed that U-Net performs comparably when trained with either the generated data or the ground truth data. These findings suggest that models trained using cues can be used to generate reliable training data.
Nonconvex Approach for Sparse and Low-Rank Constrained Models with Dual Momentum
Venue: arXiv cs.LG 2019
Year: 2019

In this manuscript, we research on the behaviors of surrogates for the rank function on different image processing problems and their optimization algorithms. We first propose a novel nonconvex rank surrogate on the general rank minimization problem and apply this to the corrupted image completion problem. Then, we propose that nonconvex rank surrogates can be introduced into two well-known sparse and low-rank models: Robust Principal Component Analysis (RPCA) and Low-Rank Representation (LRR). For optimization, we use alternating direction method of multipliers (ADMM) and propose a trick, which is called the dual momentum. We add the difference of the dual variable between the current and the last iteration with a weight. This trick can avoid the local minimum problem and make the algorithm converge to a solution with smaller recovery error in the nonconvex optimization problem. Also, it can boost the convergence when the variable updates too slowly. We also give a severe proof and verify that the proposed algorithms are convergent. Then, several experiments are conducted, including image completion, denoising, and spectral clustering with outlier detection. These experiments show that the proposed methods are effective in image and signal processing applications, and have the best performance compared with state-of-the-art methods.
Primal-Dual Block Frank-Wolfe
Venue: arXiv cs.LG 2019
Year: 2019

We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems. Our formulation includes Elastic Net, regularized SVMs and phase retrieval as special cases. The proposed Primal-Dual Block Frank-Wolfe algorithm reduces the per-iteration cost while maintaining linear convergence rate. The per iteration cost of our method depends on the structural complexity of the solution (i.e. sparsity/low-rank) instead of the ambient dimension. We empirically show that our algorithm outperforms the state-of-the-art methods on (multi-class) classification tasks.
Joint Optimization of Cooperative Communication and Computation in Two-Way Relay MEC Systems
Venue: arXiv cs.IT 2019
Year: 2019

Considering two users exchange computational results through a two-way relay equipped with a mobile-edge computing server, we investigate the joint optimization problem of cooperative communication and computation, whose objective is the total energy consumption minimization subject to the delay constraint. To derive the low-complexity optimal solution, by assuming that the transmit power of the relay is given, we theoretically derive the optimal values of computation task partition factors. Thus, the optimal solution to the total energy consumption minimization problem can be found by one-dimensional search over the transmit power of the relay. Simulation results show that our proposed scheme performs better than that where all computational tasks are allocated to the relay or users.
Towards navigation without precise localization: Weakly supervised learning of goal-directed navigation cost map
Venue: arXiv cs.RO 2019
Year: 2019

Autonomous navigation based on precise localization has been widely developed in both academic research and practical applications. The high demand for localization accuracy has been essential for safe robot planing and navigation while it makes the current geometric solutions less robust to environmental changes. Recent research on end-to-end methods handle raw sensory data with forms of navigation instructions and directly output the command for robot control. However, the lack of intermediate semantics makes the system more rigid and unstable for practical use. To explore these issues, this paper proposes an innovate navigation framework based on the GPS-level localization, which takes the raw perception data with publicly accessible navigation maps to produce an intermediate navigation cost map that allows subsequent flexible motion planning. A deterministic conditional adversarial network is adopted in our method to generate visual goal-directed paths under diverse navigation conditions. The adversarial loss avoids the pixel-level annotation and enables a weakly supervised training strategy to implicitly learn both of the traffic semantics in image perceptions and the planning intentions in navigation instructions. The navigation cost map is then rendered from the goal-directed path and the concurrently collected laser data, indicating the way towards the destination. Comprehensive experiments have been conducted with a real vehicle running in our campus and the results have verified the robustness to localization error of the proposed navigation system.
The route to chaos in routing games: When is Price of Anarchy too optimistic?
Venue: arXiv cs.GT 2019
Year: 2019

Routing games are amongst the most studied classes of games. Their two most well-known properties are that learning dynamics converge to equilibria and that all equilibria are approximately optimal. In this work, we perform a stress test for these classic results by studying the ubiquitous dynamics, Multiplicative Weights Update, in different classes of congestion games, uncovering intricate non-equilibrium phenomena. As the system demand increases, the learning dynamics go through period-doubling bifurcations, leading to instabilities, chaos and large inefficiencies even in the simplest case of non-atomic routing games with two paths of linear cost where the Price of Anarchy is equal to one. Starting with this simple class, we show that every system has a carrying capacity, above which it becomes unstable. If the equilibrium flow is a symmetric $50-50\%$ split, the system exhibits one period-doubling bifurcation. A single periodic attractor of period two replaces the attracting fixed point. Although the Price of Anarchy is equal to one, in the large population limit the time-average social cost for all but a zero measure set of initial conditions converges to its worst possible value. For asymmetric equilibrium flows, increasing the demand eventually forces the system into Li-Yorke chaos with positive topological entropy and periodic orbits of all possible periods. Remarkably, in all non-equilibrating regimes, the time-average flows on the paths converge exactly to the equilibrium flows, a property akin to no-regret learning in zero-sum games. These results are robust. We extend them to routing games with arbitrarily many strategies, polynomial cost functions, non-atomic as well as atomic routing games and heteregenous users. Our results are also applicable to any sequence of shrinking learning rates, e.g., $1/\sqrt{T}$, by allowing for a dynamically increasing population size.