OpenRaiser/Intern-Atlas huggingface dataset:
An Improved Deep Belief Network Model for Road Safety Analyses
Venue: arXiv cs.LG 2018
Year: 2018

Crash prediction is a critical component of road safety analyses. A widely adopted approach to crash prediction is application of regression based techniques. The underlying calibration process is often time-consuming, requiring significant domain knowledge and expertise and cannot be easily automated. This paper introduces a new machine learning (ML) based approach as an alternative to the traditional techniques. The proposed ML model is called regularized deep belief network, which is a deep neural network with two training steps: it is first trained using an unsupervised learning algorithm and then fine-tuned by initializing a Bayesian neural network with the trained weights from the first step. The resulting model is expected to have improved prediction power and reduced need for the time-consuming human intervention. In this paper, we attempt to demonstrate the potential of this new model for crash prediction through two case studies including a collision data set from 800 km stretch of Highway 401 and other highways in Ontario, Canada. Our intention is to show the performance of this ML approach in comparison to various traditional models including negative binomial (NB) model, kernel regression (KR), and Bayesian neural network (Bayesian NN). We also attempt to address other related issues such as effect of training data size and training parameters.
Efficient Autotuning of Hyperparameters in Approximate Nearest Neighbor Search
Venue: arXiv cs.DS 2018
Year: 2018

Approximate nearest neighbor algorithms are used to speed up nearest neighbor search in a wide array of applications. However, current indexing methods feature several hyperparameters that need to be tuned to reach an acceptable accuracy--speed trade-off. A grid search in the parameter space is often impractically slow due to a time-consuming index-building procedure. Therefore, we propose an algorithm for automatically tuning the hyperparameters of indexing methods based on randomized space-partitioning trees. In particular, we present results using randomized k-d trees, random projection trees and randomized PCA trees. The tuning algorithm adds minimal overhead to the index-building process but is able to find the optimal hyperparameters accurately. We demonstrate that the algorithm is significantly faster than existing approaches, and that the indexing methods used are competitive with the state-of-the-art methods in query time while being faster to build.
Uniform Convergence Bounds for Codec Selection
Venue: arXiv cs.SD 2018
Year: 2018

We frame the problem of selecting an optimal audio encoding scheme as a supervised learning task. Through uniform convergence theory, we guarantee approximately optimal codec selection while controlling for selection bias. We present rigorous statistical guarantees for the codec selection problem that hold for arbitrary distributions over audio sequences and for arbitrary quality metrics. Our techniques can thus balance sound quality and compression ratio, and use audio samples from the distribution to select a codec that performs well on that particular type of data. The applications of our technique are immense, as it can be used to optimize for quality and bandwidth usage of streaming and other digital media, while significantly outperforming approaches that apply a fixed codec to all data sources.
Deep Transfer Learning for Static Malware Classification
Venue: arXiv cs.LG 2018
Year: 2018

We propose to apply deep transfer learning from computer vision to static malware classification. In the transfer learning scheme, we borrow knowledge from natural images or objects and apply to the target domain of static malware detection. As a result, training time of deep neural networks is accelerated while high classification performance is still maintained. We demonstrate the effectiveness of our approach on three experiments and show that our proposed method outperforms other classical machine learning methods measured in accuracy, false positive rate, true positive rate and $F_1$ score (in binary classification). We instrument an interpretation component to the algorithm and provide interpretable explanations to enhance security practitioners' trust to the model. We further discuss a convex combination scheme of transfer learning and training from scratch for enhanced malware detection, and provide insights of the algorithmic interpretation of vision-based malware classification techniques.
Integrating Runtime Values with Source Code to Facilitate Program Comprehension
Venue: arXiv cs.SE 2018
Year: 2018

An inherently abstract nature of source code makes programs difficult to understand. In our research, we designed three techniques utilizing concrete values of variables and other expressions during program execution. RuntimeSearch is a debugger extension searching for a given string in all expressions at runtime. DynamiDoc generates documentation sentences containing examples of arguments, return values and state changes. RuntimeSamp augments source code lines in the IDE (integrated development environment) with sample variable values. In this post-doctoral article, we briefly describe these three approaches and related motivational studies, surveys and evaluations. We also reflect on the PhD study, providing advice for current students. Finally, short-term and long-term future work is described.
Training on the test set? An analysis of Spampinato et al. [31]
Venue: arXiv cs.CV 2018
Year: 2018

A recent paper [31] claims to classify brain processing evoked in subjects watching ImageNet stimuli as measured with EEG and to use a representation derived from this processing to create a novel object classifier. That paper, together with a series of subsequent papers [8, 15, 17, 20, 21, 30, 35], claims to revolutionize the field by achieving extremely successful results on several computer-vision tasks, including object classification, transfer learning, and generation of images depicting human perception and thought using brain-derived representations measured through EEG. Our novel experiments and analyses demonstrate that their results crucially depend on the block design that they use, where all stimuli of a given class are presented together, and fail with a rapid-event design, where stimuli of different classes are randomly intermixed. The block design leads to classification of arbitrary brain states based on block-level temporal correlations that tend to exist in all EEG data, rather than stimulus-related activity. Because every trial in their test sets comes from the same block as many trials in the corresponding training sets, their block design thus leads to surreptitiously training on the test set. This invalidates all subsequent analyses performed on this data in multiple published papers and calls into question all of the purported results. We further show that a novel object classifier constructed with a random codebook performs as well as or better than a novel object classifier constructed with the representation extracted from EEG data, suggesting that the performance of their classifier constructed with a representation extracted from EEG data does not benefit at all from the brain-derived representation. Our results calibrate the underlying difficulty of the tasks involved and caution against sensational and overly optimistic, but false, claims to the contrary.
Training on Art Composition Attributes to Influence CycleGAN Art Generation
Venue: arXiv cs.LG 2018
Year: 2018

I consider how to influence CycleGAN, image-to-image translation, by using additional constraints from a neural network trained on art composition attributes. I show how I trained the the Art Composition Attributes Network (ACAN) by incorporating domain knowledge based on the rules of art evaluation and the result of applying each art composition attribute to apple2orange image translation.
Optimality Clue for Graph Coloring Problem
Venue: arXiv cs.DM 2018
Year: 2018

In this paper, we present a new approach which qualifies or not a solution found by a heuristic as a potential optimal solution. Our approach is based on the following observation: for a minimization problem, the number of admissible solutions decreases with the value of the objective function. For the Graph Coloring Problem (GCP), we confirm this observation and present a new way to prove optimality. This proof is based on the counting of the number of different k-colorings and the number of independent sets of a given graph G. Exact solutions counting problems are difficult problems (\#P-complete). However, we show that, using only randomized heuristics, it is possible to define an estimation of the upper bound of the number of k-colorings. This estimate has been calibrated on a large benchmark of graph instances for which the exact number of optimal k-colorings is known. Our approach, called optimality clue, build a sample of k-colorings of a given graph by running many times one randomized heuristic on the same graph instance. We use the evolutionary algorithm HEAD [Moalic et Gondran, 2018], which is one of the most efficient heuristic for GCP. Optimality clue matches with the standard definition of optimality on a wide number of instances of DIMACS and RBCII benchmarks where the optimality is known. Then, we show the clue of optimality for another set of graph instances. Optimality Metaheuristics Near-optimal.
Discriminative analysis of the human cortex using spherical CNNs - a study on Alzheimer's disease diagnosis
Venue: arXiv cs.CV 2018
Year: 2018

In neuroimaging studies, the human cortex is commonly modeled as a sphere to preserve the topological structure of the cortical surface. Cortical neuroimaging measures hence can be modeled in spherical representation. In this work, we explore analyzing the human cortex using spherical CNNs in an Alzheimer's disease (AD) classification task using cortical morphometric measures derived from structural MRI. Our results show superior performance in classifying AD versus cognitively normal and in predicting MCI progression within two years, using structural MRI information only. This work demonstrates for the first time the potential of the spherical CNNs framework in the discriminative analysis of the human cortex and could be extended to other modalities and other neurological diseases.
"Woman-Metal-White vs Man-Dress-Shorts": Combining Social, Temporal and Image Signals to Understand Popularity of Pinterest Fashion Boards
Venue: arXiv cs.SI 2018
Year: 2018

Pinterest is a popular photo sharing website. Fashion is one the most popular and content generating category on this platform. Most of the popular fashion brands and designers use boards on Pinterest for showcasing their products. However, the characteristics of popular fashion boards are not well-known. These characteristics can be used for predicting popularity of a nascent board. Further, newly formed boards can organize their content in a way similar to the popular fashion boards to garner enhanced popularity. What properties on these fashion boards determine their popularity? Can these properties be systematically quantified? In this paper, we show how social, temporal and image signals can together help in characterizing the popular fashion boards. In particular, we study the sharing/borrowing behavior of pins and the image content characteristics of the fashion boards. We analyze the sharing behavior using social and temporal signals, and propose six novel yet simple metrics: originality score, retention coefficients, production coefficients, inter-copying time, duration of sharing and speed coefficients. We further study the image based content properties by extracting fashion, color and gender terms embedded in the pin images. We observe significant differences across the popular (highly followed or highly ranked by the experts) and the unpopular (less followed) boards. We then use these characteristic features to early predict the popularity of a board and achieve a high correlation of 0.874 with low RMSE value. Our key observation is that likes and repin retention coefficients are the most discriminatory factors of a board's popularity apart from the usage of various color, gender and fashion terms.
Learning On-Road Visual Control for Self-Driving Vehicles with Auxiliary Tasks
Venue: arXiv cs.CV 2018
Year: 2018

A safe and robust on-road navigation system is a crucial component of achieving fully automated vehicles. NVIDIA recently proposed an End-to-End algorithm that can directly learn steering commands from raw pixels of a front camera by using one convolutional neural network. In this paper, we leverage auxiliary information aside from raw images and design a novel network structure, called Auxiliary Task Network (ATN), to help boost the driving performance while maintaining the advantage of minimal training data and an End-to-End training method. In this network, we introduce human prior knowledge into vehicle navigation by transferring features from image recognition tasks. Image semantic segmentation is applied as an auxiliary task for navigation. We consider temporal information by introducing an LSTM module and optical flow to the network. Finally, we combine vehicle kinematics with a sensor fusion step. We discuss the benefits of our method over state-of-the-art visual navigation methods both in the Udacity simulation environment and on the real-world Comma.ai dataset.
Assessing technical and cost efficiency of research activities: A case study of the Italian university system
Venue: arXiv cs.DL 2018
Year: 2018

This paper employs data envelopment analysis (DEA) to assess both technical and cost efficiency of research activities of the Italian university system. Differently from both peer review and the top-down discipline-invariant bibliographic approaches used elsewhere, a bottom-up bibliometric methodology is applied. Publications are assigned first to authors and then to one of nine scientific and technical university disciplinary areas. Inputs are specified in terms of the numbers of full, associate and assistant professors and outputs as the number of publications, contributions to publications and their scientific impact as variously measured across the disciplines included. DEA is undertaken cross-sectionally using the averages of these inputs and outputs over the period 2001-2003. The results typically show much variation in the rankings of the disciplinary areas within and across universities, depending on the efficiency indicator employed.
Interference-aware User Grouping Strategy in NOMA Systems with QoS Constraints
Venue: arXiv cs.NI 2018
Year: 2018

To meet the performance and complexity requirements from practical deployment of non-orthogonal multiple access (NOMA) systems, several users are grouped together for NOMA transmission while orthogonal resources are allocated among groups. User grouping strategies have significant impact on the power consumption and system performance. However, existing related studies divide users into groups based on channel conditions, where diverse quality of service (QoS) and interference have not been considered. In this paper, we focus on the interference-aware user grouping strategy in NOMA systems, aiming at minimizing power consumption with QoS constraints. We define a power consumption and externality (PCE) function for each user to represent the power consumption involved by this user to satisfy its QoS requirement as well as interference that this user brings to others in the same group. Then, we extend the definition of PCE to multi-user scenarios and convert the user grouping problem into the problem of searching for specific negative loops in the graph. Bellman-Ford algorithm is extended to find these negative loops. Furthermore, a greedy suboptimal algorithm is proposed to approach the solution within polynomial time. Simulation results show that the proposed algorithms can considerably reduce the total power consumption compared with existing strategies.
Physical Attribute Prediction Using Deep Residual Neural Networks
Venue: arXiv cs.CV 2018
Year: 2018

Images taken from the Internet have been used alongside Deep Learning for many different tasks such as: smile detection, ethnicity, hair style, hair colour, gender and age prediction. After witnessing these usages, we were wondering what other attributes can be predicted from facial images available on the Internet. In this paper we tackle the prediction of physical attributes from face images using Convolutional Neural Networks trained on our dataset named FIRW. We crawled around 61, 000 images from the web, then use face detection to crop faces from these real world images. We choose ResNet-50 as our base network architecture. This network was pretrained for the task of face recognition by using the VGG-Face dataset, and we finetune it by using our own dataset to predict physical attributes. Separate networks are trained for the prediction of body type, ethnicity, gender, height and weight; our models achieve the following accuracies for theses tasks, respectively: 84.58%, 87.34%, 97.97%, 70.51%, 63.99%. To validate our choice of ResNet-50 as the base architecture, we also tackle the famous CelebA dataset. Our models achieve an averagy accuracy of 91.19% on CelebA, which is comparable to state-of-the-art approaches.
Self-Attention: A Better Building Block for Sentiment Analysis Neural Network Classifiers
Venue: arXiv cs.CL 2018
Year: 2018

Sentiment Analysis has seen much progress in the past two decades. For the past few years, neural network approaches, primarily RNNs and CNNs, have been the most successful for this task. Recently, a new category of neural networks, self-attention networks (SANs), have been created which utilizes the attention mechanism as the basic building block. Self-attention networks have been shown to be effective for sequence modeling tasks, while having no recurrence or convolutions. In this work we explore the effectiveness of the SANs for sentiment analysis. We demonstrate that SANs are superior in performance to their RNN and CNN counterparts by comparing their classification accuracy on six datasets as well as their model characteristics such as training speed and memory consumption. Finally, we explore the effects of various SAN modifications such as multi-head attention as well as two methods of incorporating sequence position information into SANs.
Modelling and Fast Terminal Sliding Mode Control for Mirror-based Pointing Systems
Venue: arXiv cs.SY 2018
Year: 2018

In this paper, we present a new discrete-time Fast Terminal Sliding Mode (FTSM) controller for mirror-based pointing systems. We first derive the decoupled model of those systems and then estimate the parameters using a nonlinear least-square identification method. Based on the derived model, we design a FTSM sliding manifold in the continuous domain. We then exploit the Euler discretization on the designed FTSM sliding surfaces to synthesize a discrete-time controller. Furthermore, we improve the transient dynamics of the sliding surface by adding a linear term. Finally, we prove the stability of the proposed controller based on the Sarpturk reaching condition. Extensive simulations, followed by comparisons with the Terminal Sliding Mode (TSM) and Model Predictive Control (MPC) have been carried out to evaluate the effectiveness of the proposed approach. A comparative study with data obtained from a real-time experiment was also conducted. The results indicate the advantage of the proposed method over the other techniques.
Dynamic Programming Approach to Template-based OCR
Venue: arXiv cs.CV 2018
Year: 2018

In this paper we propose a dynamic programming solution to the template-based recognition task in OCR case. We formulate a problem of optimal position search for complex objects consisting of parts forming a sequence. We limit the distance between every two adjacent elements with predefined upper and lower thresholds. We choose the sum of penalties for each part in given position as a function to be minimized. We show that such a choice of restrictions allows a faster algorithm to be used than the one for the general form of deformation penalties. We named this algorithm Dynamic Squeezeboxes Packing (DSP) and applied it to solve the two OCR problems: text fields extraction from an image of document Visual Inspection Zone (VIZ) and license plate segmentation. The quality and the performance of resulting solutions were experimentally proved to meet the requirements of the state-of-the-art industrial recognition systems.
Lexical analysis of automated accounts on Twitter
Venue: arXiv cs.SI 2018
Year: 2018

In recent years, social bots have been using increasingly more sophisticated, challenging detection strategies. While many approaches and features have been proposed, social bots evade detection and interact much like humans making it difficult to distinguish real human accounts from bot accounts. For detection systems, various features under the broader categories of account profile, tweet content, network and temporal pattern have been utilised. The use of tweet content features is limited to analysis of basic terms such as URLs, hashtags, name entities and sentiment. Given a set of tweet contents with no obvious pattern can we distinguish contents produced by social bots from that of humans? We aim to answer this question by analysing the lexical richness of tweets produced by the respective accounts using large collections of different datasets. Our results show a clear margin between the two classes in lexical diversity, lexical sophistication and distribution of emoticons. We found that the proposed lexical features significantly improve the performance of classifying both account types. These features are useful for training a standard machine learning classifier for effective detection of social bot accounts. A new dataset is made freely available for further exploration.
A Reinforcement Learning Hyper-Heuristic in Multi-Objective Single Point Search with Application to Structural Fault Identification
Venue: arXiv cs.LG 2018
Year: 2018

Multi-objective optimizations are frequently encountered in engineering practices. The solution techniques and parametric selections however are usually problem-specific. In this study we formulate a reinforcement learning hyper-heuristic scheme, and propose four low-level heuristics which can work coherently with the single point search algorithm MOSA/R (Multi-Objective Simulated Annealing Algorithm based on Re-seed) towards multi-objective optimization problems of general applications. Making use of the domination amount, crowding distance and hypervolume calculations, the proposed hyper-heuristic scheme can meet various optimization requirements adaptively and autonomously. The approach developed not only exhibits improved and more robust performance compared to AMOSA, NSGA-II and MOEA/D when applied to benchmark test cases, but also shows promising results when applied to a generic structural fault identification problem. The outcome of this research can be extended to a variety of design and manufacturing optimization applications.
A Review of Meta-Reinforcement Learning for Deep Neural Networks Architecture Search
Venue: arXiv cs.LG 2018
Year: 2018

Deep Neural networks are efficient and flexible models that perform well for a variety of tasks such as image, speech recognition and natural language understanding. In particular, convolutional neural networks (CNN) generate a keen interest among researchers in computer vision and more specifically in classification tasks. CNN architecture and related hyperparameters are generally correlated to the nature of the processed task as the network extracts complex and relevant characteristics allowing the optimal convergence. Designing such architectures requires significant human expertise, substantial computation time and doesn't always lead to the optimal network. Model configuration topic has been extensively studied in machine learning without leading to a standard automatic method. This survey focuses on reviewing and discussing the current progress in automating CNN architecture search.