OpenRaiser/Intern-Atlas huggingface dataset:
OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning
Venue: arXiv cs.CV 2017
Year: 2017

Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification. Yet, this does not naturally enforce intra-class similarity nor inter-class margin of the learned deep representations. To simultaneously achieve these two goals, different solutions have been proposed in the literature, such as the pairwise or triplet losses. However, such solutions carry the extra task of selecting pairs or triplets, and the extra computational burden of computing and learning for many combinations of them. In this paper, we propose a plug-and-play loss term for deep networks that explicitly reduces intra-class variance and enforces inter-class margin simultaneously, in a simple and elegant geometric manner. For each class, the deep features are collapsed into a learned linear subspace, or union of them, and inter-class subspaces are pushed to be as orthogonal as possible. Our proposed Orthogonal Low-rank Embedding (OLÉ) does not require carefully crafting pairs or triplets of samples for training, and works standalone as a classification loss, being the first reported deep metric learning framework of its kind. Because of the improved margin between features of different classes, the resulting deep networks generalize better, are more discriminative, and more robust. We demonstrate improved classification performance in general object recognition, plugging the proposed loss term into existing off-the-shelf architectures. In particular, we show the advantage of the proposed loss in the small data/model scenario, and we significantly advance the state-of-the-art on the Stanford STL-10 benchmark.
One for All: Towards Language Independent Named Entity Linking
Venue: arXiv cs.CL 2017
Year: 2017

Entity linking (EL) is the task of disambiguating mentions in text by associating them with entries in a predefined database of mentions (persons, organizations, etc). Most previous EL research has focused mainly on one language, English, with less attention being paid to other languages, such as Spanish or Chinese. In this paper, we introduce LIEL, a Language Independent Entity Linking system, which provides an EL framework which, once trained on one language, works remarkably well on a number of different languages without change. LIEL makes a joint global prediction over the entire document, employing a discriminative reranking framework with many domain and language-independent feature functions. Experiments on numerous benchmark datasets, show that the proposed system, once trained on one language, English, outperforms several state-of-the-art systems in English (by 4 points) and the trained model also works very well on Spanish (14 points better than a competitor system), demonstrating the viability of the approach.
Analyzing Large-Scale, Distributed and Uncertain Data
Venue: arXiv cs.DB 2017
Year: 2017

The exponential growth of data in current times and the demand to gain information and knowledge from the data present new challenges for database researchers. Known database systems and algorithms are no longer capable of effectively handling such large data sets. MapReduce is a novel programming paradigm for processing distributable problems over large-scale data using a computer cluster. In this work we explore the MapReduce paradigm from three different angles. We begin by examining a well-known problem in the field of data mining: mining closed frequent itemsets over a large dataset. By harnessing the power of MapReduce, we present a novel algorithm for mining closed frequent itemsets that outperforms existing algorithms. Next, we explore one of the fundamental implications of "Big Data": The data is not known with complete certainty. A probabilistic database is a relational database with the addendum that each tuple is associated with a probability of its existence. A natural development of MapReduce is of a distributed relational database management system, where relational calculus has been reduced to a combination of map and reduce function. We take this development a step further by proposing a query optimizer over distributed, probabilistic database. Finally, we analyze the best known implementation of MapReduce called Hadoop, aiming to overcome one of its major drawbacks: it does not directly support the explicit specification of the data repeatedly processed throughout different jobs.Many data-mining algorithms, such as clustering and association-rules require iterative computation: the same data are processed again and again until the computation converges or a stopping condition is satisfied. We propose a modification to Hadoop such that it will support efficient access to the same data in different jobs.
State spaces of convolutional codes, codings and encoders
Venue: arXiv cs.IT 2017
Year: 2017

In this paper we give a compact presentation of the theory of abstract spaces for convolutional codes and convolutional encoders, and show a connection between them that seems to be missing in the literature. We use it for a short proof of two facts: the size of a convolutional encoder of a polynomial matrix is at least its inner degree, and the minimal encoder has the size of the external degree if the matrix is reduced.
How to Learn a Model Checker
Venue: arXiv cs.LG 2017
Year: 2017

We show how machine-learning techniques, particularly neural networks, offer a very effective and highly efficient solution to the approximate model-checking problem for continuous and hybrid systems, a solution where the general-purpose model checker is replaced by a model-specific classifier trained by sampling model trajectories. To the best of our knowledge, we are the first to establish this link from machine learning to model checking. Our method comprises a pipeline of analysis techniques for estimating and obtaining statistical guarantees on the classifier's prediction performance, as well as tuning techniques to improve such performance. Our experimental evaluation considers the time-bounded reachability problem for three well-established benchmarks in the hybrid systems community. On these examples, we achieve an accuracy of 99.82% to 100% and a false-negative rate (incorrectly predicting that unsafe states are not reachable from a given state) of 0.0007 to 0. We believe that this level of accuracy is acceptable in many practical applications and we show how the approximate model checker can be made more conservative by tuning the classifier through further training and selection of the classification threshold.
A Study on the Perception of Researchers about the Application of Agile Software Development Methods in Research
Venue: arXiv cs.SE 2017
Year: 2017

Papers on Agile Software Development methods are often focused on their applicability in commercial projects or organizations. There are no current studies that we know about addressing the application of these methods in research projects. The objective of this work is to describe the perception of researchers on the application of agile software development practices and principles for research projects. A study was conducted by constructing and applying a questionnaire to Brazilian researchers of different affiliations, formation and research areas in order to obtain information about their knowledge and openness to follow agile software development principles and practices.
A Local Analysis of Block Coordinate Descent for Gaussian Phase Retrieval
Venue: arXiv cs.IT 2017
Year: 2017

While convergence of the Alternating Direction Method of Multipliers (ADMM) on convex problems is well studied, convergence on nonconvex problems is only partially understood. In this paper, we consider the Gaussian phase retrieval problem, formulated as a linear constrained optimization problem with a biconvex objective. The particular structure allows for a novel application of the ADMM. It can be shown that the dual variable is zero at the global minimizer. This motivates the analysis of a block coordinate descent algorithm, which is equivalent to the ADMM with the dual variable fixed to be zero. We show that the block coordinate descent algorithm converges to the global minimizer at a linear rate, when starting from a deterministically achievable initialization point.
Exact Algorithms With Worst-case Guarantee For Scheduling: From Theory to Practice
Venue: arXiv cs.CC 2017
Year: 2017

This PhD thesis summarizes research works on the design of exact algorithms that provide a worst-case (time or space) guarantee for NP-hard scheduling problems. Both theoretical and practical aspects are considered with three main results reported. The first one is about a Dynamic Programming algorithm which solves the F3Cmax problem in O*(3^n) time and space. The algorithm is easily generalized to other flowshop problems and single machine scheduling problems. The second contribution is about a search tree method called Branch & Merge which solves the 1||SumTi problem with the time complexity converging to O*(2^n) and in polynomial space. Our third contribution aims to improve the practical efficiency of exact search tree algorithms solving scheduling problems. First we realized that a better way to implement the idea of Branch & Merge is to use a technique called Memorization. By the finding of a new algorithmic paradox and the implementation of a memory cleaning strategy, the method succeeded to solve instances with 300 more jobs with respect to the state-of-the-art algorithm for the 1||SumTi problem. Then the treatment is extended to another three problems 1|ri|SumCi, 1|dtilde|SumwiCi and F2||SumCi. The results of the four problems all together show the power of the Memorization paradigm when applied on sequencing problems. We name it Branch & Memorize to promote a systematic consideration of Memorization as an essential building block in branching algorithms like Branch and Bound. The method can surely also be used to solve other problems, which are not necessarily scheduling problems.
An Efficient Algorithm for Non-Negative Matrix Factorization with Random Projections
Venue: arXiv cs.NA 2017
Year: 2017

Non-negative matrix factorization (NMF) is one of the most popular decomposition techniques for multivariate data. NMF is a core method for many machine-learning related computational problems, such as data compression, feature extraction, word embedding, recommender systems etc. In practice, however, its application is challenging for large datasets. The efficiency of NMF is constrained by long data loading times, by large memory requirements and by limited parallelization capabilities. Here we present a novel and efficient compressed NMF algorithm. Our algorithm applies a random compression scheme to drastically reduce the dimensionality of the problem, preserving well the pairwise distances between data points and inherently limiting the memory and communication load. Our algorithm supersedes existing methods in speed. Nonetheless, it matches the best non-compressed algorithms in reconstruction precision.
Named Entity Sequence Classification
Venue: arXiv cs.NE 2017
Year: 2017

Named Entity Recognition (NER) aims at locating and classifying named entities in text. In some use cases of NER, including cases where detected named entities are used in creating content recommendations, it is crucial to have a reliable confidence level for the detected named entities. In this work we study the problem of finding confidence levels for detected named entities. We refer to this problem as Named Entity Sequence Classification (NESC). We frame NESC as a binary classification problem and we use NER as well as recurrent neural networks to find the probability of candidate named entity is a real named entity. We apply this approach to Tweet texts and we show how we could find named entities with high confidence levels from Tweets.
Identifying Data Noises, User Biases, and System Errors in Geo-tagged Twitter Messages (Tweets)
Venue: arXiv cs.SI 2017
Year: 2017

Many social media researchers and data scientists collected geo-tagged tweets to conduct spatial analysis or identify spatiotemporal patterns of filtered messages for specific topics or events. This paper provides a systematic view to illustrate the characteristics (data noises, user biases, and system errors) of geo-tagged tweets from the Twitter Streaming API. First, we found that a small percentage (1%) of active Twitter users can create a large portion (16%) of geo-tagged tweets. Second, there is a significant amount (57.3%) of geo-tagged tweets located outside the Twitter Streaming API's bounding box in San Diego. Third, we can detect spam, bot, cyborg tweets (data noises) by examining the "source" metadata field. The portion of data noises in geo-tagged tweets is significant (29.42% in San Diego, CA and 53.47% in Columbus, OH) in our case study. Finally, the majority of geo-tagged tweets are not created by the generic Twitter apps in Android or iPhone devices, but by other platforms, such as Instagram and Foursquare. We recommend a multi-step procedure to remove these noises for the future research projects utilizing geo-tagged tweets.
Adversarial Examples that Fool Detectors
Venue: arXiv cs.CV 2017
Year: 2017

An adversarial example is an example that has been adjusted to produce a wrong label when presented to a system at test time. To date, adversarial example constructions have been demonstrated for classifiers, but not for detectors. If adversarial examples that could fool a detector exist, they could be used to (for example) maliciously create security hazards on roads populated with smart vehicles. In this paper, we demonstrate a construction that successfully fools two standard detectors, Faster RCNN and YOLO. The existence of such examples is surprising, as attacking a classifier is very different from attacking a detector, and that the structure of detectors - which must search for their own bounding box, and which cannot estimate that box very accurately - makes it quite likely that adversarial patterns are strongly disrupted. We show that our construction produces adversarial examples that generalize well across sequences digitally, even though large perturbations are needed. We also show that our construction yields physical objects that are adversarial.
Parallel Prefix Algorithms for the Registration of Arbitrarily Long Electron Micrograph Series
Venue: arXiv cs.DC 2017
Year: 2017

Recent advances in the technology of transmission electron microscopy have allowed for a more precise visualization of materials and physical processes, such as metal oxidation. Nevertheless, the quality of information is limited by the damage caused by an electron beam, movement of the specimen or other environmental factors. A novel registration method has been proposed to remove those limitations by acquiring a series of low dose microscopy frames and performing a computational registration on them to understand and visualize the sample. This process can be represented as a prefix sum with a complex and computationally intensive binary operator and a parallelization is necessary to enable processing long series of microscopy images. With our parallelization scheme, the time of registration of results from ten seconds of microscopy acquisition has been decreased from almost thirteen hours to less than seven minutes on 512 Intel IvyBridge cores.
Topics and Label Propagation: Best of Both Worlds for Weakly Supervised Text Classification
Venue: arXiv cs.CL 2017
Year: 2017

We propose a Label Propagation based algorithm for weakly supervised text classification. We construct a graph where each document is represented by a node and edge weights represent similarities among the documents. Additionally, we discover underlying topics using Latent Dirichlet Allocation (LDA) and enrich the document graph by including the topics in the form of additional nodes. The edge weights between a topic and a text document represent level of "affinity" between them. Our approach does not require document level labelling, instead it expects manual labels only for topic nodes. This significantly minimizes the level of supervision needed as only a few topics are observed to be enough for achieving sufficiently high accuracy. The Label Propagation Algorithm is employed on this enriched graph to propagate labels among the nodes. Our approach combines the advantages of Label Propagation (through document-document similarities) and Topic Modelling (for minimal but smart supervision). We demonstrate the effectiveness of our approach on various datasets and compare with state-of-the-art weakly supervised text classification approaches.
A Deep Network Model for Paraphrase Detection in Short Text Messages
Venue: arXiv cs.IR 2017
Year: 2017

This paper is concerned with paraphrase detection. The ability to detect similar sentences written in natural language is crucial for several applications, such as text mining, text summarization, plagiarism detection, authorship authentication and question answering. Given two sentences, the objective is to detect whether they are semantically identical. An important insight from this work is that existing paraphrase systems perform well when applied on clean texts, but they do not necessarily deliver good performance against noisy texts. Challenges with paraphrase detection on user generated short texts, such as Twitter, include language irregularity and noise. To cope with these challenges, we propose a novel deep neural network-based approach that relies on coarse-grained sentence modeling using a convolutional neural network and a long short-term memory model, combined with a specific fine-grained word-level similarity matching model. Our experimental results show that the proposed approach outperforms existing state-of-the-art approaches on user-generated noisy social media data, such as Twitter texts, and achieves highly competitive performance on a cleaner corpus.
End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient
Venue: arXiv cs.AI 2017
Year: 2017

Learning a goal-oriented dialog policy is generally performed offline with supervised learning algorithms or online with reinforcement learning (RL). Additionally, as companies accumulate massive quantities of dialog transcripts between customers and trained human agents, encoder-decoder methods have gained popularity as agent utterances can be directly treated as supervision without the need for utterance-level annotations. However, one potential drawback of such approaches is that they myopically generate the next agent utterance without regard for dialog-level considerations. To resolve this concern, this paper describes an offline RL method for learning from unannotated corpora that can optimize a goal-oriented policy at both the utterance and dialog level. We introduce a novel reward function and use both on-policy and off-policy policy gradient to learn a policy offline without requiring online user interaction or an explicit state space definition.
Columnar Database Techniques for Creating AI Features
Venue: arXiv cs.DB 2017
Year: 2017

Recent advances with in-memory columnar database techniques have increased the performance of analytical queries on very large databases and data warehouses. At the same time, advances in artificial intelligence (AI) algorithms have increased the ability to analyze data. We use the term AI to encompass both Deep Learning (DL or neural network) and Machine Learning (ML aka Big Data analytics). Our exploration of the AI full stack has led us to a cross-stack columnar database innovation that efficiently creates features for AI analytics. The innovation is to create Augmented Dictionary Values (ADVs) to add to existing columnar database dictionaries in order to increase the efficiency of featurization by minimizing data movement and data duplication. We show how various forms of featurization (feature selection, feature extraction, and feature creation) can be efficiently calculated in a columnar database. The full stack AI investigation has also led us to propose an integrated columnar database and AI architecture. This architecture has information flows and feedback loops to improve the whole analytics cycle during multiple iterations of extracting data from the data sources, featurization, and analysis.
Using Intermittent Synchronization to Compensate for Rhythmic Body Motion During Autonomous Surgical Cutting and Debridement
Venue: arXiv cs.RO 2017
Year: 2017

Anatomical structures are rarely static during a surgical procedure due to breathing, heartbeats, and peristaltic movements. Inspired by observing an expert surgeon, we propose an intermittent synchronization with the extrema of the rhythmic motion (i.e., the lowest velocity windows). We performed 2 experiments: (1) pattern cutting, and (2) debridement. In (1), we found that the intermittent synchronization approach, while 1.8x slower than tracking motion, was significantly more robust to noise and control latency, and it reduced the max cutting error by 2.6x In (2), a baseline approach with no synchronization achieves 62% success rate for each removal, while intermittent synchronization achieves 80%.
Online Red Packets: A Large-scale Empirical Study of Gift Giving on WeChat
Venue: arXiv cs.SI 2017
Year: 2017

Gift giving is a ubiquitous social phenomenon, and red packets have been used as monetary gifts in Asian countries for thousands of years. In recent years, online red packets have become widespread in China through the WeChat platform. Exploiting a unique dataset consisting of 61 million group red packets and seven million users, we conduct a large-scale, data-driven study to understand the spread of red packets and the effect of red packets on group activity. We find that the cash flows between provinces are largely consistent with provincial GDP rankings, e.g., red packets are sent from users in the south to those in the north. By distinguishing spontaneous from reciprocal red packets, we reveal the behavioral patterns in sending red packets: males, seniors, and people with more in-group friends are more inclined to spontaneously send red packets, while red packets from females, youths, and people with less in-group friends are more reciprocal. Furthermore, we use propensity score matching to study the external effects of red packets on group dynamics. We show that red packets increase group participation and strengthen in-group relationships, which partly explain the benefits and motivations for sending red packets.
Wireless Aggregation at Nearly Constant Rate
Venue: arXiv cs.NI 2017
Year: 2017

One of the most fundamental tasks in sensor networks is the computation of a (compressible) aggregation function of the input measurements. What rate of computation can be maintained, by properly choosing the aggregation tree, the TDMA schedule of the tree edges, and the transmission powers? This can be viewed as the convergecast capacity of a wireless network. We show here that the optimal rate is effectively a constant. This holds even in \emph{arbitrary} networks, under the physical model of interference. This compares with previous bounds that are logarithmic (e.g., $Ω(1/\log n)$). Namely, we show that a rate of $Ω(1/\log^* Δ)$ is possible, where $Δ$ is the length diversity (ratio between the furthest to the shortest distance between nodes). It also implies that the \emph{scheduling complexity} of wireless connectivity is $O(\log^* Δ)$. This is achieved using the natural minimum spanning tree (MST). Our method crucially depends on choosing the appropriate power assignment for the instance at hand, since without power control, only a trivial linear rate can be guaranteed. We also show that there is a fixed power assignment that allows for a rate of $Ω(1/\log\log Δ)$. Surprisingly, these bounds are best possible. No aggregation network can guarantee a rate better than $O(1/\log\log Δ)$ using fixed power assignment. Also, when using arbitrary power control, there are instances whose MSTs cannot be scheduled in fewer than $Ω(1/\log^* Δ)$ slots.