OpenRaiser/Intern-Atlas huggingface dataset:
Squeeze-SegNet: A new fast Deep Convolutional Neural Network for Semantic Segmentation
Venue: arXiv cs.CV 2017
Year: 2017

The recent researches in Deep Convolutional Neural Network have focused their attention on improving accuracy that provide significant advances. However, if they were limited to classification tasks, nowadays with contributions from Scientific Communities who are embarking in this field, they have become very useful in higher level tasks such as object detection and pixel-wise semantic segmentation. Thus, brilliant ideas in the field of semantic segmentation with deep learning have completed the state of the art of accuracy, however this architectures become very difficult to apply in embedded systems as is the case for autonomous driving. We present a new Deep fully Convolutional Neural Network for pixel-wise semantic segmentation which we call Squeeze-SegNet. The architecture is based on Encoder-Decoder style. We use a SqueezeNet-like encoder and a decoder formed by our proposed squeeze-decoder module and upsample layer using downsample indices like in SegNet and we add a deconvolution layer to provide final multi-channel feature map. On datasets like Camvid or City-states, our net gets SegNet-level accuracy with less than 10 times fewer parameters than SegNet.
A Generally Applicable, Highly Scalable Measurement Computation and Optimization Approach to Sequential Model-Based Diagnosis
Venue: arXiv cs.AI 2017
Year: 2017

Model-Based Diagnosis deals with the identification of the real cause of a system's malfunction based on a formal system model and observations of the system behavior. When a malfunction is detected, there is usually not enough information available to pinpoint the real cause and one needs to discriminate between multiple fault hypotheses (called diagnoses). To this end, Sequential Diagnosis approaches ask an oracle for additional system measurements. This work presents strategies for (optimal) measurement selection in model-based sequential diagnosis. In particular, assuming a set of leading diagnoses being given, we show how queries (sets of measurements) can be computed and optimized along two dimensions: expected number of queries and cost per query. By means of a suitable decoupling of two optimizations and a clever search space reduction the computations are done without any inference engine calls. For the full search space, we give a method requiring only a polynomial number of inferences and show how query properties can be guaranteed which existing methods do not provide. Evaluation results using real-world problems indicate that the new method computes (virtually) optimal queries instantly independently of the size and complexity of the considered diagnosis problems and outperforms equally general methods not exploiting the proposed theory by orders of magnitude.
Dialogue Act Recognition via CRF-Attentive Structured Network
Venue: arXiv cs.CL 2017
Year: 2017

Dialogue Act Recognition (DAR) is a challenging problem in dialogue interpretation, which aims to attach semantic labels to utterances and characterize the speaker's intention. Currently, many existing approaches formulate the DAR problem ranging from multi-classification to structured prediction, which suffer from handcrafted feature extensions and attentive contextual structural dependencies. In this paper, we consider the problem of DAR from the viewpoint of extending richer Conditional Random Field (CRF) structural dependencies without abandoning end-to-end training. We incorporate hierarchical semantic inference with memory mechanism on the utterance modeling. We then extend structured attention network to the linear-chain conditional random field layer which takes into account both contextual utterances and corresponding dialogue acts. The extensive experiments on two major benchmark datasets Switchboard Dialogue Act (SWDA) and Meeting Recorder Dialogue Act (MRDA) datasets show that our method achieves better performance than other state-of-the-art solutions to the problem. It is a remarkable fact that our method is nearly close to the human annotator's performance on SWDA within 2% gap.
Exploiting Layerwise Convexity of Rectifier Networks with Sign Constrained Weights
Venue: arXiv cs.LG 2017
Year: 2017

By introducing sign constraints on the weights, this paper proposes sign constrained rectifier networks (SCRNs), whose training can be solved efficiently by the well known majorization-minimization (MM) algorithms. We prove that the proposed two-hidden-layer SCRNs, which exhibit negative weights in the second hidden layer and negative weights in the output layer, are capable of separating any two (or more) disjoint pattern sets. Furthermore, the proposed two-hidden-layer SCRNs can decompose the patterns of each class into several clusters so that each cluster is convexly separable from all the patterns from the other classes. This provides a means to learn the pattern structures and analyse the discriminant factors between different classes of patterns.
PQSER: A Matlab package for spectral seriation
Venue: arXiv cs.MS 2017
Year: 2017

The seriation problem is an important ordering issue which consists of finding the best ordering of a set of units whose interrelationship is defined by a bipartite graph. It has important applications in, e.g., archaeology, anthropology, psychology, and biology. This paper presents a Matlab implementation of an algorithm for spectral seriation by Atkins et al., based on the use of the Fiedler vector of the Laplacian matrix associated to the problem, which encodes the set of admissible solutions into a PQ-tree. We introduce some numerical technicalities in the original algorithm to improve its performance, and point out that the presence of a multiple Fiedler value may have a substantial influence on the computation of an approximated solution, in the presence of inconsistent data sets. Practical examples and numerical experiments show how to use the toolbox to process data sets deriving from real-world applications.
Fast Predictive Simple Geodesic Regression
Venue: arXiv cs.CV 2017
Year: 2017

Deformable image registration and regression are important tasks in medical image analysis. However, they are computationally expensive, especially when analyzing large-scale datasets that contain thousands of images. Hence, cluster computing is typically used, making the approaches dependent on such computational infrastructure. Even larger computational resources are required as study sizes increase. This limits the use of deformable image registration and regression for clinical applications and as component algorithms for other image analysis approaches. We therefore propose using a fast predictive approach to perform image registrations. In particular, we employ these fast registration predictions to approximate a simplified geodesic regression model to capture longitudinal brain changes. The resulting method is orders of magnitude faster than the standard optimization-based regression model and hence facilitates large-scale analysis on a single graphics processing unit (GPU). We evaluate our results on 3D brain magnetic resonance images (MRI) from the ADNI datasets.
ORBIT: Ordering Based Information Transfer Across Space and Time for Global Surface Water Monitoring
Venue: arXiv cs.LG 2017
Year: 2017

Many earth science applications require data at both high spatial and temporal resolution for effective monitoring of various ecosystem resources. Due to practical limitations in sensor design, there is often a trade-off in different resolutions of spatio-temporal datasets and hence a single sensor alone cannot provide the required information. Various data fusion methods have been proposed in the literature that mainly rely on individual timesteps when both datasets are available to learn a mapping between features values at different resolutions using local relationships between pixels. Earth observation data is often plagued with spatially and temporally correlated noise, outliers and missing data due to atmospheric disturbances which pose a challenge in learning the mapping from a local neighborhood at individual timesteps. In this paper, we aim to exploit time-independent global relationships between pixels for robust transfer of information across different scales. Specifically, we propose a new framework, ORBIT (Ordering Based Information Transfer) that uses relative ordering constraint among pixels to transfer information across both time and scales. The effectiveness of the framework is demonstrated for global surface water monitoring using both synthetic and real-world datasets.
K3, L3, LP, RM3, A3, FDE: How to Make Many-Valued Logics Work for You
Venue: arXiv cs.AI 2017
Year: 2017

We investigate some well-known (and a few not-so-well-known) many-valued logics that have a small number (3 or 4) of truth values. For some of them we complain that they do not have any \emph{logical} use (despite their perhaps having some intuitive semantic interest) and we look at ways to add features so as to make them useful, while retaining their intuitive appeal. At the end, we show some surprising results in the system FDE, and its relationships with features of other logics. We close with some new examples of "synonymous logics." An Appendix contains a natural deduction system for our augmented FDE, and proofs of soundness and completeness.
Understanding the Changing Roles of Scientific Publications via Citation Embeddings
Venue: arXiv cs.DL 2017
Year: 2017

Researchers may describe different aspects of past scientific publications in their publications and the descriptions may keep changing in the evolution of science. The diverse and changing descriptions (i.e., citation context) on a publication characterize the impact and contributions of the past publication. In this article, we aim to provide an approach to understanding the changing and complex roles of a publication characterized by its citation context. We described a method to represent the publications' dynamic roles in science community in different periods as a sequence of vectors by training temporal embedding models. The temporal representations can be used to quantify how much the roles of publications changed and interpret how they changed. Our study in the biomedical domain shows that our metric on the changes of publications' roles is stable over time at the population level but significantly distinguish individuals. We also show the interpretability of our methods by a concrete example.
Knowledge transfer for surgical activity prediction
Venue: arXiv cs.LG 2017
Year: 2017

Lack of training data hinders automatic recognition and prediction of surgical activities necessary for situation-aware operating rooms. We propose using knowledge transfer to compensate for data deficit and improve prediction. We used two approaches to extract and transfer surgical process knowledge. First, we encoded semantic information about surgical terms using word embedding which boosted learning process. Secondly, we passed knowledge between different clinical datasets of neurosurgical procedures using transfer learning. Transfer learning was shown to be more effective than a simple combination of data, especially for less similar procedures. The combination of two methods provided 22% improvement of activity prediction. We also made several pertinent observations about surgical practices.
Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy
Venue: arXiv cs.LG 2017
Year: 2017

Deep learning networks have achieved state-of-the-art accuracies on computer vision workloads like image classification and object detection. The performant systems, however, typically involve big models with numerous parameters. Once trained, a challenging aspect for such top performing models is deployment on resource constrained inference systems - the models (often deep networks or wide networks or both) are compute and memory intensive. Low-precision numerics and model compression using knowledge distillation are popular techniques to lower both the compute requirements and memory footprint of these deployed models. In this paper, we study the combination of these two techniques and show that the performance of low-precision networks can be significantly improved by using knowledge distillation techniques. Our approach, Apprentice, achieves state-of-the-art accuracies using ternary precision and 4-bit precision for variants of ResNet architecture on ImageNet dataset. We present three schemes using which one can apply knowledge distillation techniques to various stages of the train-and-deploy pipeline.
An Optimal and Progressive Approach to Online Search of Top-k Influential Communities
Venue: arXiv cs.DB 2017
Year: 2017

Community search over large graphs is a fundamental problem in graph analysis. Recent studies propose to compute top-k influential communities, where each reported community not only is a cohesive subgraph but also has a high influence value. The existing approaches to the problem of top-k influential community search can be categorized as index-based algorithms and online search algorithms without indexes. The index-based algorithms, although being very efficient in conducting community searches, need to pre-compute a special-purpose index and only work for one built-in vertex weight vector. In this paper, we investigate on-line search approaches and propose an instance-optimal algorithm LocalSearch whose time complexity is linearly proportional to the size of the smallest subgraph that a correct algorithm needs to access without indexes. In addition, we also propose techniques to make LocalSearch progressively compute and report the communities in decreasing influence value order such that k does not need to be specified. Moreover, we extend our framework to the general case of top-k influential community search regarding other cohesiveness measures. Extensive empirical studies on real graphs demonstrate that our algorithms outperform the existing online search algorithms by several orders of magnitude.
A General Neural Network Hardware Architecture on FPGA
Venue: arXiv cs.CV 2017
Year: 2017

Field Programmable Gate Arrays (FPGAs) plays an increasingly important role in data sampling and processing industries due to its highly parallel architecture, low power consumption, and flexibility in custom algorithms. Especially, in the artificial intelligence field, for training and implement the neural networks and machine learning algorithms, high energy efficiency hardware implement and massively parallel computing capacity are heavily demanded. Therefore, many global companies have applied FPGAs into AI and Machine learning fields such as autonomous driving and Automatic Spoken Language Recognition (Baidu) [1] [2] and Bing search (Microsoft) [3]. Considering the FPGAs great potential in these fields, we tend to implement a general neural network hardware architecture on XILINX ZU9CG System On Chip (SOC) platform [4], which contains abundant hardware resource and powerful processing capacity. The general neural network architecture on the FPGA SOC platform can perform forward and backward algorithms in deep neural networks (DNN) with high performance and easily be adjusted according to the type and scale of the neural networks.
On Analyzing Job Hop Behavior and Talent Flow Networks
Venue: arXiv cs.SI 2017
Year: 2017

Analyzing job hopping behavior is important for the understanding of job preference and career progression of working individuals. When analyzed at the workforce population level, job hop analysis helps to gain insights of talent flow and organization competition. Traditionally, surveys are conducted on job seekers and employers to study job behavior. While surveys are good at getting direct user input to specially designed questions, they are often not scalable and timely enough to cope with fast-changing job landscape. In this paper, we present a data science approach to analyze job hops performed by about 490,000 working professionals located in a city using their publicly shared profiles. We develop several metrics to measure how much work experience is needed to take up a job and how recent/established the job is, and then examine how these metrics correlate with the propensity of hopping. We also study how job hop behavior is related to job promotion/demotion. Finally, we perform network analyses at the job and organization levels in order to derive insights on talent flow as well as job and organizational competitiveness.
Optimal Load Balancing in Millimeter Wave Cellular Heterogeneous Networks
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, we propose a novel and effective approach to optimizing the load balancing in a millimeter wave (mmWave) cellular heterogeneous network (HetNet) with a macro-tier and a micro-tier. The unique characteristics of mmWave transmission are incorporated into the network by adopting the Poisson point process (PPP) for base station (BS) location, the line-of-sight (LoS) ball model for mmWave links, the sectored antenna model for key antenna array characteristics, and Nakagami-$m$ fading for wireless channels. To reduce the load of macro-tier BSs, we consider a bias factor $A_{s}$ in the network for offloading user equipments (UEs) to micro-tier BSs. For this network, we first analyze the loads of macro- and micro-tier BSs. Then we derive a new expression for the rate coverage probability of the network, based on which the optimal $A_{s}$ maximizing the rate coverage probability is found. Through numerical results, we demonstrate the correctness of our analysis and the validity of the optimal $A_{s}$. Importantly, the optimal $A_{s}$ can bring a profound improvement in the rate coverage probability relative to a fixed $A_{s}$. Furthermore, we evaluate the impact of various network parameters, e.g., the densities and the beamwidths of BSs, on the rate coverage probability and the optimal $A_{s}$, offering valuable guidelines into practical mmWave HetNet design.
On evolutionary selection of blackjack strategies
Venue: arXiv cs.NE 2017
Year: 2017

We apply the approach of evolutionary programming to the problem of optimization of the blackjack basic strategy. We demonstrate that the population of initially random blackjack strategies evolves and saturates to a profitable performance in about one hundred generations. The resulting strategy resembles the known blackjack basic strategies in the specifics of its prescriptions, and has a similar performance. We also study evolution of the population of strategies initialized to the Thorp's basic strategy.
Remedies against the Vocabulary Gap in Information Retrieval
Venue: arXiv cs.IR 2017
Year: 2017

Search engines rely heavily on term-based approaches that represent queries and documents as bags of words. Text---a document or a query---is represented by a bag of its words that ignores grammar and word order, but retains word frequency counts. When presented with a search query, the engine then ranks documents according to their relevance scores by computing, among other things, the matching degrees between query and document terms. While term-based approaches are intuitive and effective in practice, they are based on the hypothesis that documents that exactly contain the query terms are highly relevant regardless of query semantics. Inversely, term-based approaches assume documents that do not contain query terms as irrelevant. However, it is known that a high matching degree at the term level does not necessarily mean high relevance and, vice versa, documents that match null query terms may still be relevant. Consequently, there exists a vocabulary gap between queries and documents that occurs when both use different words to describe the same concepts. It is the alleviation of the effect brought forward by this vocabulary gap that is the topic of this dissertation. More specifically, we propose (1) methods to formulate an effective query from complex textual structures and (2) latent vector space models that circumvent the vocabulary gap in information retrieval.
Bounding the convergence time of local probabilistic evolution
Venue: arXiv cs.DM 2017
Year: 2017

Isoperimetric inequalities form a very intuitive yet powerful characterization of the connectedness of a state space, that has proven successful in obtaining convergence bounds. Since the seventies they form an essential tool in differential geometry, graph theory and Markov chain analysis. In this paper we use isoperimetric inequalities to construct a bound on the convergence time of any local probabilistic evolution that leaves its limit distribution invariant. We illustrate how this general result leads to new bounds on convergence times beyond the explicit Markovian setting, among others on quantum dynamics.
Analyzing Before Solving: Which Parameters Influence Low-Level Surgical Activity Recognition
Venue: arXiv cs.HC 2017
Year: 2017

Automatic low-level surgical activity recognition is today well-known technical bottleneck for smart and situation-aware assistance for the operating room of the future. Our study sought to discover which sensors and signals could facilitate this recognition. Low-level surgical activity represents semantic information about a surgical procedure that is usually expressed by the following elements: an action verb, surgical instrument, and operated anatomical structure. We hypothesized that activity recognition does not require sensors for all three elements. We conducted a large-scale study using deep learning on semantic data from 154 operations from four different surgeries. The results demonstrated that the instrument and verb encode similar information, meaning only one needs to be tracked, preferably the instrument. The anatomical structure, however, provides some unique cues, and it is thus crucial to recognize it. For all the studied surgeries, a combination of two elements, always including the structure, proved sufficient to confidently recognize the activities. We also found that in the presence of noise, combining the information about the instrument, structure, and historical context produced better results than a simple composition of all three elements. Several relevant observations about surgical practices were also made in this paper. Such findings provide cues for designing a new generation of operating rooms.
Speech Dereverberation with Context-aware Recurrent Neural Networks
Venue: arXiv cs.SD 2017
Year: 2017

In this paper, we propose a model to perform speech dereverberation by estimating its spectral magnitude from the reverberant counterpart. Our models are capable of extracting features that take into account both short and long-term dependencies in the signal through a convolutional encoder (which extracts features from a short, bounded context of frames) and a recurrent neural network for extracting long-term information. Our model outperforms a recently proposed model that uses different context information depending on the reverberation time, without requiring any sort of additional input, yielding improvements of up to 0.4 on PESQ, 0.3 on STOI, and 1.0 on POLQA relative to reverberant speech. We also show our model is able to generalize to real room impulse responses even when only trained with simulated room impulse responses, different speakers, and high reverberation times. Lastly, listening tests show the proposed method outperforming benchmark models in reduction of perceived reverberation.