OpenRaiser/Intern-Atlas huggingface dataset:
All-Transfer Learning for Deep Neural Networks and its Application to Sepsis Classification
Venue: arXiv cs.CV 2017
Year: 2017

In this article, we propose a transfer learning method for deep neural networks (DNNs). Deep learning has been widely used in many applications. However, applying deep learning is problematic when a large amount of training data are not available. One of the conventional methods for solving this problem is transfer learning for DNNs. In the field of image recognition, state-of-the-art transfer learning methods for DNNs re-use parameters trained on source domain data except for the output layer. However, this method may result in poor classification performance when the amount of target domain data is significantly small. To address this problem, we propose a method called All-Transfer Deep Learning, which enables the transfer of all parameters of a DNN. With this method, we can compute the relationship between the source and target labels by the source domain knowledge. We applied our method to actual two-dimensional electrophoresis image~(2-DE image) classification for determining if an individual suffers from sepsis; the first attempt to apply a classification approach to 2-DE images for proteomics, which has attracted considerable attention as an extension beyond genomics. The results suggest that our proposed method outperforms conventional transfer learning methods for DNNs.
Information Design for Strategic Coordination of Autonomous Devices with Non-Aligned Utilities
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, we investigate the coordination of autonomous devices with non-aligned utility functions. Both encoder and decoder are considered as players, that choose the encoding and the decoding in order to maximize their long-run utility functions. The topology of the point-to-point network under investigation, suggests that the decoder implements a strategy, knowing in advance the strategy of the encoder. We characterize the encoding and decoding functions that form an equilibrium, by using empirical coordination. The equilibrium solution is related to an auxiliary game in which both players choose some conditional distributions in order to maximize their expected utilities. This problem is closely related to the literature on "Information Design" in Game Theory. We also characterize the set of posterior distributions that are compatible with a rate-limited channel between the encoder and the decoder. Finally, we provide an example of non-aligned utility functions corresponding to parallel fading multiple access channels.
Linear-Time Algorithms for Maximum-Weight Induced Matchings and Minimum Chain Covers in Convex Bipartite Graphs
Venue: arXiv cs.DS 2017
Year: 2017

A bipartite graph $G=(U,V,E)$ is convex if the vertices in $V$ can be linearly ordered such that for each vertex $u\in U$, the neighbors of $u$ are consecutive in the ordering of $V$. An induced matching $H$ of $G$ is a matching such that no edge of $E$ connects endpoints of two different edges of $H$. We show that in a convex bipartite graph with $n$ vertices and $m$ weighted edges, an induced matching of maximum total weight can be computed in $O(n+m)$ time. An unweighted convex bipartite graph has a representation of size $O(n)$ that records for each vertex $u\in U$ the first and last neighbor in the ordering of $V$. Given such a compact representation, we compute an induced matching of maximum cardinality in $O(n)$ time. In convex bipartite graphs, maximum-cardinality induced matchings are dual to minimum chain covers. A chain cover is a covering of the edge set by chain subgraphs, that is, subgraphs that do not contain induced matchings of more than one edge. Given a compact representation, we compute a representation of a minimum chain cover in $O(n)$ time. If no compact representation is given, the cover can be computed in $O(n+m)$ time. All of our algorithms achieve optimal running time for the respective problem and model. Previous algorithms considered only the unweighted case, and the best algorithm for computing a maximum-cardinality induced matching or a minimum chain cover in a convex bipartite graph had a running time of $O(n^2)$.
Consensus Halving is PPA-Complete
Venue: arXiv cs.CC 2017
Year: 2017

We show that the computational problem CONSENSUS-HALVING is PPA-complete, the first PPA-completeness result for a problem whose definition does not involve an explicit circuit. We also show that an approximate version of this problem is polynomial-time equivalent to NECKLACE SPLITTING, which establishes PPAD-hardness for NECKLACE SPLITTING, and suggests that it is also PPA-complete.
Towards a Cloud-Based Service for Maintaining and Analyzing Data About Scientific Events
Venue: arXiv cs.DL 2017
Year: 2017

We propose the new cloud-based service OpenResearch for managing and analyzing data about scientific events such as conferences and workshops in a persistent and reliable way. This includes data about scientific articles, participants, acceptance rates, submission numbers, impact values as well as organizational details such as program committees, chairs, fees and sponsors. OpenResearch is a centralized repository for scientific events and supports researchers in collecting, organizing, sharing and disseminating information about scientific events in a structured way. An additional feature currently under development is the possibility to archive web pages along with the extracted semantic data in order to lift the burden of maintaining new and old conference web sites from public research institutions. However, the main advantage is that this cloud-based repository enables a comprehensive analysis of conference data. Based on extracted semantic data, it is possible to determine quality estimations, scientific communities, research trends as well the development of acceptance rates, fees, and number of participants in a continuous way complemented by projections into the future. Furthermore, data about research articles can be systematically explored using a content-based analysis as well as citation linkage. All data maintained in this crowd-sourcing platform is made freely available through an open SPARQL endpoint, which allows for analytical queries in a flexible and user-defined way.
Vertebral body segmentation with GrowCut: Initial experience, workflow and practical application
Venue: arXiv cs.CV 2017
Year: 2017

In this contribution, we used the GrowCut segmentation algorithm publicly available in three-dimensional Slicer for three-dimensional segmentation of vertebral bodies. To the best of our knowledge, this is the first time that the GrowCut method has been studied for the usage of vertebral body segmentation. In brief, we found that the GrowCut segmentation times were consistently less than the manual segmentation times. Hence, GrowCut provides an alternative to a manual slice-by-slice segmentation process.
Not all bytes are equal: Neural byte sieve for fuzzing
Venue: arXiv cs.SE 2017
Year: 2017

Fuzzing is a popular dynamic program analysis technique used to find vulnerabilities in complex software. Fuzzing involves presenting a target program with crafted malicious input designed to cause crashes, buffer overflows, memory errors, and exceptions. Crafting malicious inputs in an efficient manner is a difficult open problem and often the best approach to generating such inputs is through applying uniform random mutations to pre-existing valid inputs (seed files). We present a learning technique that uses neural networks to learn patterns in the input files from past fuzzing explorations to guide future fuzzing explorations. In particular, the neural models learn a function to predict good (and bad) locations in input files to perform fuzzing mutations based on the past mutations and corresponding code coverage information. We implement several neural models including LSTMs and sequence-to-sequence models that can encode variable length input files. We incorporate our models in the state-of-the-art AFL (American Fuzzy Lop) fuzzer and show significant improvements in terms of code coverage, unique code paths, and crashes for various input formats including ELF, PNG, PDF, and XML.
Text Mining Descriptions Of Dreams: aesthetic and clinical efforts
Venue: arXiv cs.CY 2017
Year: 2017

Dreams are highly valued in both Freudian psychoanalysis and less conservative clinical traditions. Text mining enables the extraction of meaning from writings in powerful and unexpected ways. In this work, we report methods, uses and results obtained by mining descriptions of dreams. The texts were collected as part of a course in Schizoanalysis (Clinical Psychology) from dozens of participants. They were subsequently mined using various techniques for the achievement of poems and summaries, which were then used in clinical sessions by means of music and declamation. The results were found aesthetically appealing and effective to engage the audience. The expansion of the corpus, mining methods and strategies for using the derivatives for art and therapy are considered for future work.
Impartial redistricting: a Markov chain approach to the "Gerrymandering problem"
Venue: arXiv cs.CY 2017
Year: 2017

After every U.S. national census, a state legislature is required to redraw the boundaries of congressional districts in order to account for changes in population. At the moment this is done in a highly partisan way, with districting done in order to maximize the benefits to the party in power. This is a threat to U.S's democracy. There have been proposals to take the re-districting out of the hands of political parties and give to an "independent" commission. Independence is hard to come by and in this thesis we want to explore the possibility of computer generated districts that as far as possible to avoid partisan "gerrymandering". The idea we have is to treat every possible redistricting as a state in a Markov Chain: every state is obtained by its former state in random way. With some technical conditions, we will get a near uniform member of the states after running sufficiently long time (the mixing time). Then we can say the uniform member is an impartial distribution. Based on the geographical and statistical data of Pennsylvania, I have achieved the Markov Chain algorithm with several constraints, done optimization experiments and a web interface is going to be made to show the results.
Tensor Decompositions for Modeling Inverse Dynamics
Venue: arXiv cs.LG 2017
Year: 2017

Modeling inverse dynamics is crucial for accurate feedforward robot control. The model computes the necessary joint torques, to perform a desired movement. The highly non-linear inverse function of the dynamical system can be approximated using regression techniques. We propose as regression method a tensor decomposition model that exploits the inherent three-way interaction of positions x velocities x accelerations. Most work in tensor factorization has addressed the decomposition of dense tensors. In this paper, we build upon the decomposition of sparse tensors, with only small amounts of nonzero entries. The decomposition of sparse tensors has successfully been used in relational learning, e.g., the modeling of large knowledge graphs. Recently, the approach has been extended to multi-class classification with discrete input variables. Representing the data in high dimensional sparse tensors enables the approximation of complex highly non-linear functions. In this paper we show how the decomposition of sparse tensors can be applied to regression problems. Furthermore, we extend the method to continuous inputs, by learning a mapping from the continuous inputs to the latent representations of the tensor decomposition, using basis functions. We evaluate our proposed model on a dataset with trajectories from a seven degrees of freedom SARCOS robot arm. Our experimental results show superior performance of the proposed functional tensor model, compared to challenging state-of-the art methods.
Detecting Near Duplicates in Software Documentation
Venue: arXiv cs.SE 2017
Year: 2017

Contemporary software documentation is as complicated as the software itself. During its lifecycle, the documentation accumulates a lot of near duplicate fragments, i.e. chunks of text that were copied from a single source and were later modified in different ways. Such near duplicates decrease documentation quality and thus hamper its further utilization. At the same time, they are hard to detect manually due to their fuzzy nature. In this paper we give a formal definition of near duplicates and present an algorithm for their detection in software documents. This algorithm is based on the exact software clone detection approach: the software clone detection tool Clone Miner was adapted to detect exact duplicates in documents. Then, our algorithm uses these exact duplicates to construct near ones. We evaluate the proposed algorithm using the documentation of 19 open source and commercial projects. Our evaluation is very comprehensive - it covers various documentation types: design and requirement specifications, programming guides and API documentation, user manuals. Overall, the evaluation shows that all kinds of software documentation contain a significant number of both exact and near duplicates. Next, we report on the performed manual analysis of the detected near duplicates for the Linux Kernel Documentation. We present both quantative and qualitative results of this analysis, demonstrate algorithm strengths and weaknesses, and discuss the benefits of duplicate management in software documents.
"Found in Translation": Predicting Outcomes of Complex Organic Chemistry Reactions using Neural Sequence-to-Sequence Models
Venue: arXiv cs.LG 2017
Year: 2017

There is an intuitive analogy of an organic chemist's understanding of a compound and a language speaker's understanding of a word. Consequently, it is possible to introduce the basic concepts and analyze potential impacts of linguistic analysis to the world of organic chemistry. In this work, we cast the reaction prediction task as a translation problem by introducing a template-free sequence-to-sequence model, trained end-to-end and fully data-driven. We propose a novel way of tokenization, which is arbitrarily extensible with reaction information. With this approach, we demonstrate results superior to the state-of-the-art solution by a significant margin on the top-1 accuracy. Specifically, our approach achieves an accuracy of 80.1% without relying on auxiliary knowledge such as reaction templates. Also, 66.4% accuracy is reached on a larger and noisier dataset.
Accelerating HPC codes on Intel(R) Omni-Path Architecture networks: From particle physics to Machine Learning
Venue: arXiv cs.DC 2017
Year: 2017

We discuss practical methods to ensure near wirespeed performance from clusters with either one or two Intel(R) Omni-Path host fabric interfaces (HFI) per node, and Intel(R) Xeon Phi(TM) 72xx (Knight's Landing) processors, and using the Linux operating system. The study evaluates the performance improvements achievable and the required programming approaches in two distinct example problems: firstly in Cartesian communicator halo exchange problems, appropriate for structured grid PDE solvers that arise in quantum chromodynamics simulations of particle physics, and secondly in gradient reduction appropriate to synchronous stochastic gradient descent for machine learning. As an example, we accelerate a published Baidu Research reduction code and obtain a factor of ten speedup over the original code using the techniques discussed in this paper. This displays how a factor of ten speedup in strongly scaled distributed machine learning could be achieved when synchronous stochastic gradient descent is massively parallelised with a fixed mini-batch size. We find a significant improvement in performance robustness when memory is obtained using carefully allocated 2MB "huge" virtual memory pages, implying that either non-standard allocation routines should be used for communication buffers. These can be accessed via a LD\_PRELOAD override in the manner suggested by libhugetlbfs. We make use of a the Intel(R) MPI 2019 library "Technology Preview" and underlying software to enable thread concurrency throughout the communication software stake via multiple PSM2 endpoints per process and use of multiple independent MPI communicators. When using a single MPI process per node, we find that this greatly accelerates delivered bandwidth in many core Intel(R) Xeon Phi processors.
pyLEMMINGS: Large Margin Multiple Instance Classification and Ranking for Bioinformatics Applications
Venue: arXiv cs.LG 2017
Year: 2017

Motivation: A major challenge in the development of machine learning based methods in computational biology is that data may not be accurately labeled due to the time and resources required for experimentally annotating properties of proteins and DNA sequences. Standard supervised learning algorithms assume accurate instance-level labeling of training data. Multiple instance learning is a paradigm for handling such labeling ambiguities. However, the widely used large-margin classification methods for multiple instance learning are heuristic in nature with high computational requirements. In this paper, we present stochastic sub-gradient optimization large margin algorithms for multiple instance classification and ranking, and provide them in a software suite called pyLEMMINGS. Results: We have tested pyLEMMINGS on a number of bioinformatics problems as well as benchmark datasets. pyLEMMINGS has successfully been able to identify functionally important segments of proteins: binding sites in Calmodulin binding proteins, prion forming regions, and amyloid cores. pyLEMMINGS achieves state-of-the-art performance in all these tasks, demonstrating the value of multiple instance learning. Furthermore, our method has shown more than 100-fold improvement in terms of running time as compared to heuristic solutions with improved accuracy over benchmark datasets. Availability and Implementation: pyLEMMINGS python package is available for download at: http://faculty.pieas.edu.pk/fayyaz/software.html#pylemmings.
Multiple-Source Adaptation for Regression Problems
Venue: arXiv cs.LG 2017
Year: 2017

We present a detailed theoretical analysis of the problem of multiple-source adaptation in the general stochastic scenario, extending known results that assume a single target labeling function. Our results cover a more realistic scenario and show the existence of a single robust predictor accurate for \emph{any} target mixture of the source distributions. Moreover, we present an efficient and practical optimization solution to determine the robust predictor in the important case of squared loss, by casting the problem as an instance of DC-programming. We report the results of experiments with both an artificial task and a sentiment analysis task. We find that our algorithm outperforms competing approaches by producing a single robust model that performs well on any target mixture distribution.
Achieve Sustainable Ultra-Dense Heterogeneous Networks for 5G
Venue: arXiv cs.NI 2017
Year: 2017

Due to the exponentially increased demands of mobile data traffic, e.g., a 1000-fold increase in traffic demand from 4G to 5G, network densification is considered as a key mechanism in the evolution of cellular networks, and ultra-dense heterogeneous network (UDHN) is a promising technique to meet the requirements of explosive data traffic in 5G networks. In the UDHN, base station is brought closer and closer to users through densely deploying small cells, which would result in extremely high spectral efficiency and energy efficiency. In this article, we first present a potential network architecture for the UDHN, and then propose a generalized orthogonal/non-orthogonal random access scheme to improve the network efficiency while reducing the signaling overhead. Simulation results demonstrate the effectiveness of the proposed scheme. Finally, we present some of the key challenges of the UDHN.
Robust Keyframe-based Dense SLAM with an RGB-D Camera
Venue: arXiv cs.CV 2017
Year: 2017

In this paper, we present RKD-SLAM, a robust keyframe-based dense SLAM approach for an RGB-D camera that can robustly handle fast motion and dense loop closure, and run without time limitation in a moderate size scene. It not only can be used to scan high-quality 3D models, but also can satisfy the demand of VR and AR applications. First, we combine color and depth information to construct a very fast keyframe-based tracking method on a CPU, which can work robustly in challenging cases (e.g.~fast camera motion and complex loops). For reducing accumulation error, we also introduce a very efficient incremental bundle adjustment (BA) algorithm, which can greatly save unnecessary computation and perform local and global BA in a unified optimization framework. An efficient keyframe-based depth representation and fusion method is proposed to generate and timely update the dense 3D surface with online correction according to the refined camera poses of keyframes through BA. The experimental results and comparisons on a variety of challenging datasets and TUM RGB-D benchmark demonstrate the effectiveness of the proposed system.
False Positive and Cross-relation Signals in Distant Supervision Data
Venue: arXiv cs.CL 2017
Year: 2017

Distant supervision (DS) is a well-established method for relation extraction from text, based on the assumption that when a knowledge-base contains a relation between a term pair, then sentences that contain that pair are likely to express the relation. In this paper, we use the results of a crowdsourcing relation extraction task to identify two problems with DS data quality: the widely varying degree of false positives across different relations, and the observed causal connection between relations that are not considered by the DS method. The crowdsourcing data aggregation is performed using ambiguity-aware CrowdTruth metrics, that are used to capture and interpret inter-annotator disagreement. We also present preliminary results of using the crowd to enhance DS training data for a relation classification model, without requiring the crowd to annotate the entire set.
Unsupervised patient representations from clinical notes with interpretable classification decisions
Venue: arXiv cs.CL 2017
Year: 2017

We have two main contributions in this work: 1. We explore the usage of a stacked denoising autoencoder, and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. We evaluate these representations by using them as features in multiple supervised setups, and compare their performance with those of sparse representations. 2. To understand and interpret the representations, we explore the best encoded features within the patient representations obtained from the autoencoder model. Further, we calculate the significance of the input features of the trained classifiers when we use these pretrained representations as input.