OpenRaiser/Intern-Atlas huggingface dataset:
A Note on Pointwise Dimensions
Venue: arXiv cs.CC 2017
Year: 2017

This short note describes a connection between algorithmic dimensions of individual points and classical pointwise dimensions of measures.
Distributed Computing with Channel Noise
Venue: arXiv cs.CR 2017
Year: 2017

A group of $n$ users want to run a distributed protocol $π$ over a network where communication occurs via private point-to-point channels. Unfortunately, an adversary, who knows $π$, is able to maliciously flip bits on the channels. Can we efficiently simulate $π$ in the presence of such an adversary? We show that this is possible, even when $L$, the number of bits sent in $π$, and $T$, the number of bits flipped by the adversary are not known in advance. In particular, we show how to create a robust version of $π$ that 1) fails with probability at most $δ$, for any $δ>0$; and 2) sends $\tilde{O}(L + T)$ bits, where the $\tilde{O}$ notation hides a $\log (nL/ δ)$ term multiplying $L$. Additionally, we show how to improve this result when the average message size $α$ is not constant. In particular, we give an algorithm that sends $O( L (1 + (1/α) \log (n L/δ) + T)$ bits. This algorithm is adaptive in that it does not require a priori knowledge of $α$. We note that if $α$ is $Ω\left( \log (n L/δ) \right)$, then this improved algorithm sends only $O(L+T)$ bits, and is therefore within a constant factor of optimal.
Compact Full Duplex MIMO Radios in D2D Underlaid Cellular Networks: From System Design to Prototype Results
Venue: arXiv cs.IT 2017
Year: 2017

This paper considers the implementation and application possibilities of a compact full duplex multiple-input multiple-output (MIMO) architecture where direct communication exists between users, e.g., device-to-device (D2D) and cellular link coexisting on the same spectrum. For the architecture of the compact full duplex radio, we combine an analog self-interference canceler based dual-polarization with high cross-polarization discrimination (XPD) and Long Term Evolution (LTE)-based per-subcarrier digital self-interference canceler. While we consider the compactness and power efficiency of an analog solution, we focus on the digital canceler design with robustness to a frequency-selective channel and high compatibility with a conventional LTE system. For an over-the-air wireless experiment of full duplex testbed with a two-user-pair, we implement a full duplex MIMO physical layer (PHY), supporting 20 MHz bandwidth, on an FPGA-based software-defined radio platform. Further, we propose a novel timing synchronization method to construct a more viable full duplex MIMO link. By having the full duplex link prototype fully operating in real-time, we present the first characterization of the proposed compact full duplex MIMO performance depending on the transmit power of the full duplex node. We also show the link quality between nodes. One of the crucial insights of this work is that the full duplex operation of a user is capable of acquiring the throughput gain if the user has self-interference capability with guaranteed performance.
Boosting Neural Machine Translation
Venue: arXiv cs.CL 2017
Year: 2017

Training efficiency is one of the main problems for Neural Machine Translation (NMT). Deep networks need for very large data as well as many training iterations to achieve state-of-the-art performance. This results in very high computation cost, slowing down research and industrialisation. In this paper, we propose to alleviate this problem with several training methods based on data boosting and bootstrap with no modifications to the neural network. It imitates the learning process of humans, which typically spend more time when learning "difficult" concepts than easier ones. We experiment on an English-French translation task showing accuracy improvements of up to 1.63 BLEU while saving 20% of training time.
A Knowledge-Assisted Visual Malware Analysis System: Design, Validation, and Reflection of KAMAS
Venue: arXiv cs.CR 2017
Year: 2017

IT-security experts engage in behavior-based malware analysis in order to learn about previously unknown samples of malicious software (malware) or malware families. For this, they need to find and categorize suspicious patterns from large collections of execution traces. Currently available systems do not meet the analysts' needs described as: visual access suitable for complex data structures, visual representations appropriate for IT-security experts, provide work flow-specific interaction techniques, and the ability to externalize knowledge in the form of rules to ease analysis and for sharing with colleagues. To close this gap, we designed and developed KAMAS, a knowledge-assisted visualization system for behavior-based malware analysis. KAMAS supports malware analysts with visual analytics and knowledge externalization methods for the analysis process. The paper at hand is a design study that describes the design, implementation, and evaluation of the prototype. We report on the validation of KAMAS by expert reviews, a user study with domain experts, and focus group meetings with analysts from industry. Additionally, we reflect the gained insights of the design study and discuss the advantages and disadvantages of the applied visualization methods.
Radio Frequency Modulated Signaling Interconnect for Memory-to-Processor and Processor-to-Processor Interfaces: An Overview
Venue: arXiv cs.ET 2017
Year: 2017

With the evolution of heterogeneous computing system, such as network-on-chip, high-performance distributed computing, accelerator-rich architectures and cluster computing, high-speed, energy-efficient and low-latency interfaces among memory-to-processor and processor-to-processor become the key technology to enable those technologies. Simultaneously, the scaling of CMOS makes the switching speed of the transistor up to sub-THz. Radio-frequency or even millimeter-wave modulated signaling interconnect has unique features in ultra-low power operation, dynamic allocation of bandwidth and low latency, compared with convention baseband signaling interconnect. In this work, we overview the different generations of radio-frequency interconnect (RF-I) technology, compare them with conventional baseband signaling interconnect technologies. The limitations and potentials are also discussed in the end.
DOA Estimation of Coherent Signals on Coprime Arrays Exploiting Fourth-Order Cumulants
Venue: arXiv cs.IT 2017
Year: 2017

This paper considers the problem of direction-of-arrival (DOA) estimation of coherent signals on passive coprime arrays, where we resort to the fourth-order cumulants of the received signal to explore more information. A fourth-order cumulant matrix (FCM) is introduced for the coprime arrays. The special structure of the FCM is combined with the array configuration to resolve the coherent signals. Since each sparse array of the coprime arrays is uniform, a series of overlapping identical subarrays can be extracted. Using this property, we propose a generalized spatial smoothing scheme applied to the FCM. From the smoothed FCM, the DOAs of both the coherent and independent signals can be successfully estimated on the pseudo-spectrum generated by the fourth-order MUSIC algorithm. To overcome the problem of occasional false peak appearing on the pseudo-spectrum, we use a supplementary sparse array whose inter-sensor spacing is coprime to that of either existing sparse array. From the combined spectrum aided by the supplementary sensors, the false peaks are removed while the true peaks remain. The effectiveness of the proposed methods is demonstrated by simulation examples.
Exploring Different Dimensions of Attention for Uncertainty Detection
Venue: arXiv cs.CL 2017
Year: 2017

Neural networks with attention have proven effective for many natural language processing tasks. In this paper, we develop attention mechanisms for uncertainty detection. In particular, we generalize standardly used attention mechanisms by introducing external attention and sequence-preserving attention. These novel architectures differ from standard approaches in that they use external resources to compute attention weights and preserve sequence information. We compare them to other configurations along different dimensions of attention. Our novel architectures set the new state of the art on a Wikipedia benchmark dataset and perform similar to the state-of-the-art model on a biomedical benchmark which uses a large set of linguistic features.
Two decades of local binary patterns: A survey
Venue: arXiv cs.CV 2017
Year: 2017

Texture is an important characteristic for many types of images. In recent years very discriminative and computationally efficient local texture descriptors based on local binary patterns (LBP) have been developed, which has led to significant progress in applying texture methods to different problems and applications. Due to this progress, the division between texture descriptors and more generic image or video descriptors has been disappearing. A large number of different variants of LBP have been developed to improve its robustness, and to increase its discriminative power and applicability to different types of problems. In this chapter, the most recent and important variants of LBP in 2D, spatiotemporal, 3D, and 4D domains are surveyed. Interesting new developments of LBP in 1D signal analysis are also considered. Finally, some future challenges for research are presented.
Beyond Skip Connections: Top-Down Modulation for Object Detection
Venue: arXiv cs.CV 2017
Year: 2017

In recent years, we have seen tremendous progress in the field of object detection. Most of the recent improvements have been achieved by targeting deeper feedforward networks. However, many hard object categories such as bottle, remote, etc. require representation of fine details and not just coarse, semantic representations. But most of these fine details are lost in the early convolutional layers. What we need is a way to incorporate finer details from lower layers into the detection architecture. Skip connections have been proposed to combine high-level and low-level features, but we argue that selecting the right features from low-level requires top-down contextual information. Inspired by the human visual pathway, in this paper we propose top-down modulations as a way to incorporate fine details into the detection framework. Our approach supplements the standard bottom-up, feedforward ConvNet with a top-down modulation (TDM) network, connected using lateral connections. These connections are responsible for the modulation of lower layer filters, and the top-down network handles the selection and integration of contextual information and low-level features. The proposed TDM architecture provides a significant boost on the COCO testdev benchmark, achieving 28.6 AP for VGG16, 35.2 AP for ResNet101, and 37.3 for InceptionResNetv2 network, without any bells and whistles (e.g., multi-scale, iterative box refinement, etc.).
Boolean kernels for collaborative filtering in top-N item recommendation
Venue: arXiv cs.IR 2017
Year: 2017

In many personalized recommendation problems available data consists only of positive interactions (implicit feedback) between users and items. This problem is also known as One-Class Collaborative Filtering (OC-CF). Linear models usually achieve state-of-the-art performances on OC-CF problems and many efforts have been devoted to build more expressive and complex representations able to improve the recommendations. Recent analysis show that collaborative filtering (CF) datasets have peculiar characteristics such as high sparsity and a long tailed distribution of the ratings. In this paper we propose a boolean kernel, called Disjunctive kernel, which is less expressive than the linear one but it is able to alleviate the sparsity issue in CF contexts. The embedding of this kernel is composed by all the combinations of a certain arity d of the input variables, and these combined features are semantically interpreted as disjunctions of the input variables. Experiments on several CF datasets show the effectiveness and the efficiency of the proposed kernel.
Automatic Identification of Scenedesmus Polymorphic Microalgae from Microscopic Images
Venue: arXiv cs.CV 2017
Year: 2017

Microalgae counting is used to measure biomass quantity. Usually, it is performed in a manual way using a Neubauer chamber and expert criterion, with the risk of a high error rate. This paper addresses the methodology for automatic identification of Scenedesmus microalgae (used in the methane production and food industry) and applies it to images captured by a digital microscope. The use of contrast adaptive histogram equalization for pre-processing, and active contours for segmentation are presented. The calculation of statistical features (Histogram of Oriented Gradients, Hu and Zernike moments) with texture features (Haralick and Local Binary Patterns descriptors) are proposed for algae characterization. Scenedesmus algae can build coenobia consisting of 1, 2, 4 and 8 cells. The amount of algae of each coenobium helps to determine the amount of lipids, proteins, and other substances in a given sample of a algae crop. The knowledge of the quantity of those elements improves the quality of bioprocess applications. Classification of coenobia achieves accuracies of 98.63% and 97.32% with Support Vector Machine (SVM) and Artificial Neural Network (ANN), respectively. According to the results it is possible to consider the proposed methodology as an alternative to the traditional technique for algae counting. The database used in this paper is publicly available for download.
Towards Linear Algebra over Normalized Data
Venue: arXiv cs.DB 2017
Year: 2017

Providing machine learning (ML) over relational data is a mainstream requirement for data analytics systems. While almost all the ML tools require the input data to be presented as a single table, many datasets are multi-table, which forces data scientists to join those tables first, leading to data redundancy and runtime waste. Recent works on "factorized" ML mitigate this issue for a few specific ML algorithms by pushing ML through joins. But their approaches require a manual rewrite of ML implementations. Such piecemeal methods create a massive development overhead when extending such ideas to other ML algorithms. In this paper, we show that it is possible to mitigate this overhead by leveraging a popular formal algebra to represent the computations of many ML algorithms: linear algebra. We introduce a new logical data type to represent normalized data and devise a framework of algebraic rewrite rules to convert a large set of linear algebra operations over denormalized data into operations over normalized data. We show how this enables us to automatically "factorize" several popular ML algorithms, thus unifying and generalizing several prior works. We prototype our framework in the popular ML environment R and an industrial R-over-RDBMS tool. Experiments with both synthetic and real normalized data show that our framework also yields significant speed-ups, up to 36x on real data.
Equilibrium Approximation Quality of Current No-Limit Poker Bots
Venue: arXiv cs.GT 2017
Year: 2017

Approximating a Nash equilibrium is currently the best performing approach for creating poker-playing programs. While for the simplest variants of the game, it is possible to evaluate the quality of the approximation by computing the value of the best response strategy, this is currently not computationally feasible for larger variants of the game, such as heads-up no-limit Texas hold'em. In this paper, we present a simple and computationally inexpensive Local Best Response method for computing an approximate lower bound on the value of the best response strategy. Using this method, we show that existing poker-playing programs, based on solving abstract games, are remarkably poor Nash equilibrium approximations.
Markov-Nash Equilibria in Mean-Field Games with Discounted Cost
Venue: arXiv cs.SY 2017
Year: 2017

In this paper, we consider discrete-time dynamic games of the mean-field type with a finite number $N$ of agents subject to an infinite-horizon discounted-cost optimality criterion. The state space of each agent is a locally compact Polish space. At each time, the agents are coupled through the empirical distribution of their states, which affects both the agents' individual costs and their state transition probabilities. We introduce a new solution concept of the Markov-Nash equilibrium, under which a policy is player-by-player optimal in the class of all Markov policies. Under mild assumptions, we demonstrate the existence of a mean-field equilibrium in the infinite-population limit $N \to \infty$, and then show that the policy obtained from the mean-field equilibrium is approximately Markov-Nash when the number of agents $N$ is sufficiently large.
Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge
Venue: arXiv cs.CV 2017
Year: 2017

Automatic detection of pulmonary nodules in thoracic computed tomography (CT) scans has been an active area of research for the last two decades. However, there have only been few studies that provide a comparative performance evaluation of different systems on a common database. We have therefore set up the LUNA16 challenge, an objective evaluation framework for automatic nodule detection algorithms using the largest publicly available reference database of chest CT scans, the LIDC-IDRI data set. In LUNA16, participants develop their algorithm and upload their predictions on 888 CT scans in one of the two tracks: 1) the complete nodule detection track where a complete CAD system should be developed, or 2) the false positive reduction track where a provided set of nodule candidates should be classified. This paper describes the setup of LUNA16 and presents the results of the challenge so far. Moreover, the impact of combining individual systems on the detection performance was also investigated. It was observed that the leading solutions employed convolutional networks and used the provided set of nodule candidates. The combination of these solutions achieved an excellent sensitivity of over 95% at fewer than 1.0 false positives per scan. This highlights the potential of combining algorithms to improve the detection performance. Our observer study with four expert readers has shown that the best system detects nodules that were missed by expert readers who originally annotated the LIDC-IDRI data. We released this set of additional nodules for further development of CAD systems.
On the Achievable Rate of Bandlimited Continuous-Time AWGN Channels with 1-Bit Output Quantization
Venue: arXiv cs.IT 2017
Year: 2017

We consider a continuous-time bandlimited additive white Gaussian noise channel with 1-bit output quantization. On such a channel the information is carried by the temporal distances of the zero-crossings of the transmit signal. The set of input signals is constrained by the bandwidth of the channel and an average power constraint. We derive a lower bound on the capacity by lower-bounding the mutual information rate for a given set of waveforms with exponentially distributed zero-crossing distances, where we focus on the behavior in the mid to high signal-to-noise ratio regime. We find that in case the input randomness scales appropriately with the available bandwidth, the mutual information rate grows linearly with the channel bandwidth for constant signal-to-noise ratios. Furthermore, for a given bandwidth the lower bound saturates with the signal-to-noise ratio growing to infinity. The ratio between the lower bound on the mutual information rate and the capacity of the additive white Gaussian noise channel without quantization is a constant independent of the channel bandwidth for an appropriately chosen randomness of the channel input and a given signal-to-noise ratio. We complement those findings with an upper bound on the mutual information rate for the specific signaling scheme. We show that both bounds are close in the mid to high SNR domain.
Understanding Neural Networks through Representation Erasure
Venue: arXiv cs.CL 2017
Year: 2017

While neural networks have been successfully applied to many natural language processing tasks, they come at the cost of interpretability. In this paper, we propose a general methodology to analyze and interpret decisions from a neural model by observing the effects on the model of erasing various parts of the representation, such as input word-vector dimensions, intermediate hidden units, or input words. We present several approaches to analyzing the effects of such erasure, from computing the relative difference in evaluation metrics, to using reinforcement learning to erase the minimum set of input words in order to flip a neural model's decision. In a comprehensive analysis of multiple NLP tasks, including linguistic feature classification, sentence-level sentiment analysis, and document level sentiment aspect prediction, we show that the proposed methodology not only offers clear explanations about neural model decisions, but also provides a way to conduct error analysis on neural models.
A Fixed-Point Model for Pancreas Segmentation in Abdominal CT Scans
Venue: arXiv cs.CV 2017
Year: 2017

Deep neural networks have been widely adopted for automatic organ segmentation from abdominal CT scans. However, the segmentation accuracy of some small organs (e.g., the pancreas) is sometimes below satisfaction, arguably because deep networks are easily disrupted by the complex and variable background regions which occupies a large fraction of the input volume. In this paper, we formulate this problem into a fixed-point model which uses a predicted segmentation mask to shrink the input region. This is motivated by the fact that a smaller input region often leads to more accurate segmentation. In the training process, we use the ground-truth annotation to generate accurate input regions and optimize network weights. On the testing stage, we fix the network parameters and update the segmentation results in an iterative manner. We evaluate our approach on the NIH pancreas segmentation dataset, and outperform the state-of-the-art by more than 4%, measured by the average Dice-Sørensen Coefficient (DSC). In addition, we report 62.43% DSC in the worst case, which guarantees the reliability of our approach in clinical applications.
Text Summarization using Deep Learning and Ridge Regression
Venue: arXiv cs.CL 2017
Year: 2017

We develop models and extract relevant features for automatic text summarization and investigate the performance of different models on the DUC 2001 dataset. Two different models were developed, one being a ridge regressor and the other one was a multi-layer perceptron. The hyperparameters were varied and their performance were noted. We segregated the summarization task into 2 main steps, the first being sentence ranking and the second step being sentence selection. In the first step, given a document, we sort the sentences based on their Importance, and in the second step, in order to obtain non-redundant sentences, we weed out the sentences that are have high similarity with the previously selected sentences.