OpenRaiser/Intern-Atlas huggingface dataset:
A comparison of encodings for cardinality constraints in a SAT solver
Venue: arXiv cs.LO 2018
Year: 2018

Cardinality constraints are important in many Sat problems; previous studies provide contradictory conclusions about the best encoding to use. Here, three encodings are compared: Sinz's sequential-counter, Bailleux and Boufkhad's tree-based, and Ab\'ıo and coworkers' sort-based approaches. The sequential-counter approach is found to be the fastest of these for a range of related, combinatorial test cases. All encodings permit multiple solutions in the auxiliary variables for a single solution to the main variables; the numbers of multiple solutions can be very large, and might impede a Sat solver. Variants of the encodings are developed, where extra clauses reduce the numbers of multiple solutions. These variants are found to have remarkably little effect on solution time, even when the number of clauses is approximately doubled. The results accentuate the well-known observation that clause count and other measures of encoding size are not reliable indicators of the difficulty of a Sat problem.
Adaptive Extreme Learning Machine for Recurrent Beta-basis Function Neural Network Training
Venue: arXiv cs.LG 2018
Year: 2018

Beta Basis Function Neural Network (BBFNN) is a special kind of kernel basis neural networks. It is a feedforward network typified by the use of beta function as a hidden activation function. Beta is a flexible transfer function representing richer forms than the common existing functions. As in every network, the architecture setting as well as the learning method are two main gauntlets faced by BBFNN. In this paper, new architecture and training algorithm are proposed for the BBFNN. An Extreme Learning Machine (ELM) is used as a training approach of BBFNN with the aim of quickening the training process. The peculiarity of ELM is permitting a certain decrement of the computing time and complexity regarding the already used BBFNN learning algorithms such as backpropagation, OLS, etc. For the architectural design, a recurrent structure is added to the common BBFNN architecture in order to make it more able to deal with complex, non linear and time varying problems. Throughout this paper, the conceived recurrent ELM-trained BBFNN is tested on a number of tasks related to time series prediction, classification and regression. Experimental results show noticeable achievements of the proposed network compared to common feedforward and recurrent networks trained by ELM and using hyperbolic tangent as activation function. These achievements are in terms of accuracy and robustness against data breakdowns such as noise signals.
Structure Learning of Deep Neural Networks with Q-Learning
Venue: arXiv cs.LG 2018
Year: 2018

Recently, with convolutional neural networks gaining significant achievements in many challenging machine learning fields, hand-crafted neural networks no longer satisfy our requirements as designing a network will cost a lot, and automatically generating architectures has attracted increasingly more attention and focus. Some research on auto-generated networks has achieved promising results. However, they mainly aim at picking a series of single layers such as convolution or pooling layers one by one. There are many elegant and creative designs in the carefully hand-crafted neural networks, such as Inception-block in GoogLeNet, residual block in residual network and dense block in dense convolutional network. Based on reinforcement learning and taking advantages of the superiority of these networks, we propose a novel automatic process to design a multi-block neural network, whose architecture contains multiple types of blocks mentioned above, with the purpose to do structure learning of deep neural networks and explore the possibility whether different blocks can be composed together to form a well-behaved neural network. The optimal network is created by the Q-learning agent who is trained to sequentially pick different types of blocks. To verify the validity of our proposed method, we use the auto-generated multi-block neural network to conduct experiments on image benchmark datasets MNIST, SVHN and CIFAR-10 image classification task with restricted computational resources. The results demonstrate that our method is very effective, achieving comparable or better performance than hand-crafted networks and advanced auto-generated neural networks.
Analysis of Interference in Wireless Networks
Venue: arXiv cs.NI 2018
Year: 2018

As wireless systems grow rapidly worldwide, one of the most important things, wireless systems designers and service providers faces is interference. Interference decreases coverage, capacity [1], and limits the effectiveness of both new and existing systems. It is very difficult to avoid because wireless communications systems must exist together in extremely complex signal environments. These environments are consisting of multiple operating wireless networks [2]. At the same instant, new technologies and signal sources in Wireless Local Area Networks (WLANs) and digital video broadcasting are jeopardized to wireless communications service. This article provides a survey and analysis of interference in Wireless Network and provides a taxonomy.
Multi-Layers Supply chain modelling based on Multi-Agent Approach
Venue: arXiv cs.MA 2018
Year: 2018

This paper proposes a strategic multi layers model based on multi agents approach for supply chain system. It introduces a formulation and a solution methodology for the problem of supply chain design and modeling. In this paper we describe and analyze the relationships among main entities of a supply chain, such as suppliers, producers, and distribution centers, in the aim to design the agents and define their behavior. We also study, how these relationships can be formulated in a multi layer model. Finally, a generic multi agent model is illustrated.
A speech-based driver assisting module for Intelligent Transport System
Venue: arXiv cs.HC 2018
Year: 2018

Aim of this research is to transform images of roadside traffic panels to speech to assist the vehicle driver, which is a new approach in the state-of-the-art of the advanced driver assistance systems. The designed system comprises of three modules, where the first module is used to capture and detect the text area in traffic panels, second module is responsible for converting the image of the detected text area to editable text and the last module is responsible for transforming the text to speech. Additionally, during experiments, we developed a corpus of 250 images of traffic panels for two Indian languages.
Career advancement and scientific performance in universities
Venue: arXiv cs.DL 2018
Year: 2018

Many governments have placed priority on excellence in higher education as part of their policy agendas. Processes for recruitment and career advancement in universities thus have a critical role. The efficiency of faculty selection processes can be evaluated by comparing the subsequent performance of competition winners against that of the losers and the pre-existing staff of equal academic rank. Our study presents an empirical analysis concerning the recruitment procedures for associate professors in the Italian university system. The results of a bibliometric analysis of the hard science areas reveal that new associate professors are on average more productive than the incumbents. However a number of crucial concerns emerge, in particular concerning occurrence of non-winner candidates that are more productive than the winners over the subsequent triennium, and cases of winners that are completely unproductive. Beyond the implications for the Italian case, the analysis offers considerations for all decision-makers regarding the ex post evaluation of the efficiency of the recruitment process and the desirability of providing selection committees with bibliometric indicators in support of evaluation (i.e. informed peer review).
Selection committees for academic recruitment: does gender matter?
Venue: arXiv cs.DL 2018
Year: 2018

Underrepresentation of women in the academic system is a problem common to many countries, often associated with gender discrimination. In the Italian academic context in particular, favoritism is recognized as a diffuse phenomenon affecting hiring and career advancement. One of the questions that naturally arises is whether women who do assume decisional roles, having witnessed other phenomena of discrimination, would practice less favoritism than men in similar positions. Our analysis refers to the particular case of favoritism in the work of university selection committees responsible for career advancement. We observe a moderate positive association between competitions with expected outcomes and the fact the committee president is a woman. Although committees presided by women give more weight to scientific merit than those presided by men, favoritism still occurs. In fact, in the case the committee president is a woman, the single most important factor for the success of a candidate is joint research with the president; while in the case of male presidents, it is the years together in the same university.
C2TCP: A Flexible Cellular TCP to Meet Stringent Delay Requirements
Venue: arXiv cs.NI 2018
Year: 2018

Since current widely available network protocols/systems are mainly throughput-oriented designs, meeting stringent delay requirements of new applications such as virtual reality and vehicle-to-vehicle communications on cellular network requires new network protocol/system designs. C2TCP is an effort toward that new design direction. C2TCP is inspired by in-network active queue management (AQM) designs such as RED and CoDel and motivated by lack of a flexible end-to-end (e2e) approach which can adapt itself to different applications' QoS requirements without modifying any network devices. It copes with unique challenges in cellular networks for achieving ultra-low latency (including highly variable channels, deep per-user buffers, self-inflicted queuing delays, radio uplink/downlink scheduling delays) and intends to satisfy stringent delay requirements of different applications while maximizing the throughput. C2TCP works on top of classic throughput-oriented TCP and accommodates various target delays without requiring any channel prediction, network state profiling, or complicated rate adjustment mechanisms. We have evaluated C2TCP in both real-world environment and extensive trace-based emulations and compared its performance with different TCP variants and state-of-the-art schemes including PCC-Vivace, Google's BBR, Verus, Sprout, TCP Westwood, and Cubic. Results show that C2TCP outperforms all these schemes and achieves lower average delay, jitter, and 95th percentile delay for packets.
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation
Venue: arXiv cs.LG 2018
Year: 2018

The convergence rate and final performance of common deep learning models have significantly benefited from heuristics such as learning rate schedules, knowledge distillation, skip connections, and normalization layers. In the absence of theoretical underpinnings, controlled experiments aimed at explaining these strategies can aid our understanding of deep learning landscapes and the training dynamics. Existing approaches for empirical analysis rely on tools of linear interpolation and visualizations with dimensionality reduction, each with their limitations. Instead, we revisit such analysis of heuristics through the lens of recently proposed methods for loss surface and representation analysis, viz., mode connectivity and canonical correlation analysis (CCA), and hypothesize reasons for the success of the heuristics. In particular, we explore knowledge distillation and learning rate heuristics of (cosine) restarts and warmup using mode connectivity and CCA. Our empirical analysis suggests that: (a) the reasons often quoted for the success of cosine annealing are not evidenced in practice; (b) that the effect of learning rate warmup is to prevent the deeper layers from creating training instability; and (c) that the latent knowledge shared by the teacher is primarily disbursed to the deeper layers.
The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences
Venue: arXiv cs.DL 2018
Year: 2018

Accurate measurement of research productivity should take account of both the number of co-authors of every scientific work and of the different contributions of the individuals. For researchers in the life sciences, common practice is to indicate such contributions through position in the authors list. In this work, we measure the distortion introduced to bibliometric ranking lists for scientific productivity when the number of co-authors or their position in the list is ignored. The field of observation consists of all Italian university professors working in the life sciences, with scientific production examined over the period 2004-2008. The outcomes of the study lead to a recommendation against using indicators or evaluation methods that ignore the different authors' contributions to the research results.
The spin-off of elite universities in non-competitive, undifferentiated higher education systems: an empirical simulation in Italy
Venue: arXiv cs.DL 2018
Year: 2018

Higher education systems featuring intense competition have developed world-class universities, capable of attracting top professors and students and considerable public-private funding. This does not occur in non-competitive systems, where highly-talented faculty and students are dispersed across all institutions. In such systems, the authors propose the budding of spin-off universities, staffed by migration of top scientists from the entire public research system. This work illustrate the proposal through an example: the spin-off of a new university in Rome-Italy staffed with the best professors from the three current public city universities. Such a faculty would offer top national research productivity, a magnet to attract the other critical ingredients of a world-class university: talented students, abundant resources and visionary governance.
Meeting the traffic requirements of residential users in the next decade with current FTTH standards: how much? how long?
Venue: arXiv cs.NI 2018
Year: 2018

Traffic demand in the access has grown in the last years, and service providers need to upgrade their infrastructure to the latest access standards. While fiber has become the preferred technology of choice in access networks, there are many fibre access technologies available in the market. This poses a challenging question to operators not always easy to answer: how to upgrade? what technology and for how long it will cope with the demands? In this paper we model the traffic forecast in the access for the next decade and analyze possible upgrade scenarios of fibre access networks, concluding which of the NG-PON flavors could better fit the demand.
Unsupervised Identification of Disease Marker Candidates in Retinal OCT Imaging Data
Venue: arXiv cs.CV 2018
Year: 2018

The identification and quantification of markers in medical images is critical for diagnosis, prognosis, and disease management. Supervised machine learning enables the detection and exploitation of findings that are known a priori after annotation of training examples by experts. However, supervision does not scale well, due to the amount of necessary training examples, and the limitation of the marker vocabulary to known entities. In this proof-of-concept study, we propose unsupervised identification of anomalies as candidates for markers in retinal Optical Coherence Tomography (OCT) imaging data without a constraint to a priori definitions. We identify and categorize marker candidates occurring frequently in the data, and demonstrate that these markers show predictive value in the task of detecting disease. A careful qualitative analysis of the identified data driven markers reveals how their quantifiable occurrence aligns with our current understanding of disease course, in early- and late age-related macular degeneration (AMD) patients. A multi-scale deep denoising autoencoder is trained on healthy images, and a one-class support vector machine identifies anomalies in new data. Clustering in the anomalies identifies stable categories. Using these markers to classify healthy-, early AMD- and late AMD cases yields an accuracy of 81.40%. In a second binary classification experiment on a publicly available data set (healthy vs. intermediate AMD) the model achieves an area under the ROC curve of 0.944.
On the Persistence of Clustering Solutions and True Number of Clusters in a Dataset
Venue: arXiv cs.LG 2018
Year: 2018

Typically clustering algorithms provide clustering solutions with prespecified number of clusters. The lack of a priori knowledge on the true number of underlying clusters in the dataset makes it important to have a metric to compare the clustering solutions with different number of clusters. This article quantifies a notion of persistence of clustering solutions that enables comparing solutions with different number of clusters. The persistence relates to the range of data-resolution scales over which a clustering solution persists; it is quantified in terms of the maximum over two-norms of all the associated cluster-covariance matrices. Thus we associate a persistence value for each element in a set of clustering solutions with different number of clusters. We show that the datasets where natural clusters are a priori known, the clustering solutions that identify the natural clusters are most persistent - in this way, this notion can be used to identify solutions with true number of clusters. Detailed experiments on a variety of standard and synthetic datasets demonstrate that the proposed persistence-based indicator outperforms the existing approaches, such as, gap-statistic method, $X$-means, $G$-means, $PG$-means, dip-means algorithms and information-theoretic method, in accurately identifying the clustering solutions with true number of clusters. Interestingly, our method can be explained in terms of the phase-transition phenomenon in the deterministic annealing algorithm, where the number of distinct cluster centers changes (bifurcates) with respect to an annealing parameter.
Neural Belief Propagation Decoding of CRC-Polar Concatenated Codes
Venue: arXiv cs.IT 2018
Year: 2018

Polar codes are the first class of error correcting codes that provably achieve the channel capacity at infinite code length. They were selected for use in the fifth generation of cellular mobile communications (5G). In practical scenarios such as 5G, a cyclic redundancy check (CRC) is concatenated with polar codes to improve their finite length performance. This is mostly beneficial for sequential successive-cancellation list decoders. However, for parallel iterative belief propagation (BP) decoders, CRC is only used as an early stopping criterion with incremental error-correction performance improvement. In this paper, we first propose a CRC-polar BP (CPBP) decoder by exchanging the extrinsic information between the factor graph of the polar code and that of the CRC. We then propose a neural CPBP (NCPBP) algorithm which improves the CPBP decoder by introducing trainable normalizing weights on the concatenated factor graph. Our results on a 5G polar code of length 128 show that at the frame error rate of 10^(-5) and with a maximum of 30 iterations, the error-correction performance of CPBP and NCPBP are approximately 0.25 dB and 0.5 dB better than that of the conventional CRC-aided BP decoder, respectively, while introducing almost no latency overhead.
Dirichlet Variational Autoencoder for Text Modeling
Venue: arXiv cs.CL 2018
Year: 2018

We introduce an improved variational autoencoder (VAE) for text modeling with topic information explicitly modeled as a Dirichlet latent variable. By providing the proposed model topic awareness, it is more superior at reconstructing input texts. Furthermore, due to the inherent interactions between the newly introduced Dirichlet variable and the conventional multivariate Gaussian variable, the model is less prone to KL divergence vanishing. We derive the variational lower bound for the new model and conduct experiments on four different data sets. The results show that the proposed model is superior at text reconstruction across the latent space and classifications on learned representations have higher test accuracies.
Democratizing Production-Scale Distributed Deep Learning
Venue: arXiv cs.CV 2018
Year: 2018

The interest and demand for training deep neural networks have been experiencing rapid growth, spanning a wide range of applications in both academia and industry. However, training them distributed and at scale remains difficult due to the complex ecosystem of tools and hardware involved. One consequence is that the responsibility of orchestrating these complex components is often left to one-off scripts and glue code customized for specific problems. To address these restrictions, we introduce \emph{Alchemist} - an internal service built at Apple from the ground up for \emph{easy}, \emph{fast}, and \emph{scalable} distributed training. We discuss its design, implementation, and examples of running different flavors of distributed training. We also present case studies of its internal adoption in the development of autonomous systems, where training times have been reduced by 10x to keep up with the ever-growing data collection.
Online Learning Algorithms for Statistical Arbitrage
Venue: arXiv cs.LG 2018
Year: 2018

Statistical arbitrage is a class of financial trading strategies using mean reversion models. The corresponding techniques rely on a number of assumptions which may not hold for general non-stationary stochastic processes. This paper presents an alternative technique for statistical arbitrage based on online learning which does not require such assumptions and which benefits from strong learning guarantees.
Balanced Sparsity for Efficient DNN Inference on GPU
Venue: arXiv cs.CV 2018
Year: 2018

In trained deep neural networks, unstructured pruning can reduce redundant weights to lower storage cost. However, it requires the customization of hardwares to speed up practical inference. Another trend accelerates sparse model inference on general-purpose hardwares by adopting coarse-grained sparsity to prune or regularize consecutive weights for efficient computation. But this method often sacrifices model accuracy. In this paper, we propose a novel fine-grained sparsity approach, balanced sparsity, to achieve high model accuracy with commercial hardwares efficiently. Our approach adapts to high parallelism property of GPU, showing incredible potential for sparsity in the widely deployment of deep learning services. Experiment results show that balanced sparsity achieves up to 3.1x practical speedup for model inference on GPU, while retains the same high model accuracy as fine-grained sparsity.
Language-Independent Representor for Neural Machine Translation
Venue: arXiv cs.CL 2018
Year: 2018

Current Neural Machine Translation (NMT) employs a language-specific encoder to represent the source sentence and adopts a language-specific decoder to generate target translation. This language-dependent design leads to large-scale network parameters and makes the duality of the parallel data underutilized. To address the problem, we propose in this paper a language-independent representor to replace the encoder and decoder by using weight sharing. This shared representor can not only reduce large portion of network parameters, but also facilitate us to fully explore the language duality by jointly training source-to-target, target-to-source, left-to-right and right-to-left translations within a multi-task learning framework. Experiments show that our proposed framework can obtain significant improvements over conventional NMT models on resource-rich and low-resource translation tasks with only a quarter of parameters.