OpenRaiser/Intern-Atlas huggingface dataset:
Convolutional Neural Network with Word Embeddings for Chinese Word Segmentation
Venue: arXiv cs.CL 2017
Year: 2017

Character-based sequence labeling framework is flexible and efficient for Chinese word segmentation (CWS). Recently, many character-based neural models have been applied to CWS. While they obtain good performance, they have two obvious weaknesses. The first is that they heavily rely on manually designed bigram feature, i.e. they are not good at capturing n-gram features automatically. The second is that they make no use of full word information. For the first weakness, we propose a convolutional neural model, which is able to capture rich n-gram features without any feature engineering. For the second one, we propose an effective approach to integrate the proposed model with word embeddings. We evaluate the model on two benchmark datasets: PKU and MSR. Without any feature engineering, the model obtains competitive performance -- 95.7% on PKU and 97.3% on MSR. Armed with word embeddings, the model achieves state-of-the-art performance on both datasets -- 96.5% on PKU and 98.0% on MSR, without using any external labeled resource.
SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning
Venue: arXiv cs.CL 2017
Year: 2017

Synthesizing SQL queries from natural language is a long-standing open problem and has been attracting considerable interest recently. Toward solving the problem, the de facto approach is to employ a sequence-to-sequence-style model. Such an approach will necessarily require the SQL queries to be serialized. Since the same SQL query may have multiple equivalent serializations, training a sequence-to-sequence-style model is sensitive to the choice from one of them. This phenomenon is documented as the "order-matters" problem. Existing state-of-the-art approaches rely on reinforcement learning to reward the decoder when it generates any of the equivalent serializations. However, we observe that the improvement from reinforcement learning is limited. In this paper, we propose a novel approach, i.e., SQLNet, to fundamentally solve this problem by avoiding the sequence-to-sequence structure when the order does not matter. In particular, we employ a sketch-based approach where the sketch contains a dependency graph so that one prediction can be done by taking into consideration only the previous predictions that it depends on. In addition, we propose a sequence-to-set model as well as the column attention mechanism to synthesize the query based on the sketch. By combining all these novel techniques, we show that SQLNet can outperform the prior art by 9% to 13% on the WikiSQL task.
Learning Abduction under Partial Observability
Venue: arXiv cs.AI 2017
Year: 2017

Juba recently proposed a formulation of learning abductive reasoning from examples, in which both the relative plausibility of various explanations, as well as which explanations are valid, are learned directly from data. The main shortcoming of this formulation of the task is that it assumes access to full-information (i.e., fully specified) examples; relatedly, it offers no role for declarative background knowledge, as such knowledge is rendered redundant in the abduction task by complete information. In this work, we extend the formulation to utilize such partially specified examples, along with declarative background knowledge about the missing data. We show that it is possible to use implicitly learned rules together with the explicitly given declarative knowledge to support hypotheses in the course of abduction. We observe that when a small explanation exists, it is possible to obtain a much-improved guarantee in the challenging exception-tolerant setting. Such small, human-understandable explanations are of particular interest for potential applications of the task.
Word, Subword or Character? An Empirical Study of Granularity in Chinese-English NMT
Venue: arXiv cs.CL 2017
Year: 2017

Neural machine translation (NMT), a new approach to machine translation, has been proved to outperform conventional statistical machine translation (SMT) across a variety of language pairs. Translation is an open-vocabulary problem, but most existing NMT systems operate with a fixed vocabulary, which causes the incapability of translating rare words. This problem can be alleviated by using different translation granularities, such as character, subword and hybrid word-character. Translation involving Chinese is one of the most difficult tasks in machine translation, however, to the best of our knowledge, there has not been any other work exploring which translation granularity is most suitable for Chinese in NMT. In this paper, we conduct an extensive comparison using Chinese-English NMT as a case study. Furthermore, we discuss the advantages and disadvantages of various translation granularities in detail. Our experiments show that subword model performs best for Chinese-to-English translation with the vocabulary which is not so big while hybrid word-character model is most suitable for English-to-Chinese translation. Moreover, experiments of different granularities show that Hybrid_BPE method can achieve best result on Chinese-to-English translation task.
Tight Cell Probe Bounds for Succinct Boolean Matrix-Vector Multiplication
Venue: arXiv cs.DS 2017
Year: 2017

The conjectured hardness of Boolean matrix-vector multiplication has been used with great success to prove conditional lower bounds for numerous important data structure problems, see Henzinger et al. [STOC'15]. In recent work, Larsen and Williams [SODA'17] attacked the problem from the upper bound side and gave a surprising cell probe data structure (that is, we only charge for memory accesses, while computation is free). Their cell probe data structure answers queries in $\tilde{O}(n^{7/4})$ time and is succinct in the sense that it stores the input matrix in read-only memory, plus an additional $\tilde{O}(n^{7/4})$ bits on the side. In this paper, we essentially settle the cell probe complexity of succinct Boolean matrix-vector multiplication. We present a new cell probe data structure with query time $\tilde{O}(n^{3/2})$ storing just $\tilde{O}(n^{3/2})$ bits on the side. We then complement our data structure with a lower bound showing that any data structure storing $r$ bits on the side, with $n < r < n^2$ must have query time $t$ satisfying $t r = \tildeΩ(n^3)$. For $r \leq n$, any data structure must have $t = \tildeΩ(n^2)$. Since lower bounds in the cell probe model also apply to classic word-RAM data structures, the lower bounds naturally carry over. We also prove similar lower bounds for matrix-vector multiplication over $\mathbb{F}_2$.
An Automatic Diagnosis Method of Facial Acne Vulgaris Based on Convolutional Neural Network
Venue: arXiv cs.CV 2017
Year: 2017

In this paper, we present a new automatic diagnosis method of facial acne vulgaris based on convolutional neural network. This method is proposed to overcome the shortcoming of classification types in previous methods. The core of our method is to extract features of images based on convolutional neural network and achieve classification by classifier. We design a binary classifier of skin-and-non-skin to detect skin area and a seven-classifier to achieve the classification of facial acne vulgaris and healthy skin. In the experiment, we compared the effectiveness of our convolutional neural network and the pre-trained VGG16 neural network on the ImageNet dataset. And we use the ROC curve and normal confusion matrix to evaluate the performance of the binary classifier and the seven-classifier. The results of our experiment show that the pre-trained VGG16 neural network is more effective in extracting image features. The classifiers based on the pre-trained VGG16 neural network achieve the skin detection and acne classification and have good robustness.
Conditional Random Field and Deep Feature Learning for Hyperspectral Image Segmentation
Venue: arXiv cs.CV 2017
Year: 2017

Image segmentation is considered to be one of the critical tasks in hyperspectral remote sensing image processing. Recently, convolutional neural network (CNN) has established itself as a powerful model in segmentation and classification by demonstrating excellent performances. The use of a graphical model such as a conditional random field (CRF) contributes further in capturing contextual information and thus improving the segmentation performance. In this paper, we propose a method to segment hyperspectral images by considering both spectral and spatial information via a combined framework consisting of CNN and CRF. We use multiple spectral cubes to learn deep features using CNN, and then formulate deep CRF with CNN-based unary and pairwise potential functions to effectively extract the semantic correlations between patches consisting of three-dimensional data cubes. Effective piecewise training is applied in order to avoid the computationally expensive iterative CRF inference. Furthermore, we introduce a deep deconvolution network that improves the segmentation masks. We also introduce a new dataset and experimented our proposed method on it along with several widely adopted benchmark datasets to evaluate the effectiveness of our method. By comparing our results with those from several state-of-the-art models, we show the promising potential of our method.
Preserving Reliability to Heterogeneous Ultra-Dense Distributed Networks in Unlicensed Spectrum
Venue: arXiv cs.IT 2017
Year: 2017

This article investigates the prominent dilemma between capacity and reliability in heterogeneous ultra-dense distributed networks, and advocates a new measure of effective capacity to quantify the maximum sustainable data rate of a link while preserving the quality-of-service (QoS) of the link in such networks. Recent breakthroughs are brought forth in developing the theory of the effective capacity in heterogeneous ultra-dense distributed networks. Potential applications of the effective capacity are demonstrated on the admission control, power control and resource allocation of such networks, with substantial gains revealed over existing technologies. This new measure is of particular interest to ultra-dense deployment of the emerging fifth-generation (5G) wireless networks in the unlicensed spectrum, leveraging the capacity gain brought by the use of the unlicensed band and the stringent reliability sustained by 5G in future heterogeneous network environments.
Continuum Deformation of a Multiple Quadcopter Payload Delivery Team without Inter-Agent Communication
Venue: arXiv cs.SY 2017
Year: 2017

This paper proposes continuum deformation as a strategy for controlling the collective motion of a multiple quadcopter system (MQS) carrying a common payload. Continuum deformation allows expansion and contraction of inter-agent distances in a 2D motion plane to follow desired motions of three team leaders. The remaining quadcopter followers establish the desired continuum deformation only by knowing leaders positions at desired sample time waypoints without the need for inter-agent communication over the intermediate intervals. Each quadcopter applies a linear-quadratic-Gaussian (LQG) controller to track the desired trajectory given by the continuum deformation in the presence of disturbance and measurement noise. Results of simulated cooperative aerial payload transport in the presence of uncertainty illustrate the application of continuum deformation for coordinated transport through a narrow channel.
Spatio-Temporal Data Mining: A Survey of Problems and Methods
Venue: arXiv cs.LG 2017
Year: 2017

Large volumes of spatio-temporal data are increasingly collected and studied in diverse domains including, climate science, social sciences, neuroscience, epidemiology, transportation, mobile health, and Earth sciences. Spatio-temporal data differs from relational data for which computational approaches are developed in the data mining community for multiple decades, in that both spatial and temporal attributes are available in addition to the actual measurements/attributes. The presence of these attributes introduces additional challenges that needs to be dealt with. Approaches for mining spatio-temporal data have been studied for over a decade in the data mining community. In this article we present a broad survey of this relatively young field of spatio-temporal data mining. We discuss different types of spatio-temporal data and the relevant data mining questions that arise in the context of analyzing each of these datasets. Based on the nature of the data mining problem studied, we classify literature on spatio-temporal data mining into six major categories: clustering, predictive learning, change detection, frequent pattern mining, anomaly detection, and relationship mining. We discuss the various forms of spatio-temporal data mining problems in each of these categories.
Heavy Hitters and the Structure of Local Privacy
Venue: arXiv cs.DS 2017
Year: 2017

We present a new locally differentially private algorithm for the heavy hitters problem which achieves optimal worst-case error as a function of all standardly considered parameters. Prior work obtained error rates which depend optimally on the number of users, the size of the domain, and the privacy parameter, but depend sub-optimally on the failure probability. We strengthen existing lower bounds on the error to incorporate the failure probability, and show that our new upper bound is tight with respect to this parameter as well. Our lower bound is based on a new understanding of the structure of locally private protocols. We further develop these ideas to obtain the following general results beyond heavy hitters. $\bullet$ Advanced Grouposition: In the local model, group privacy for $k$ users degrades proportionally to $\approx \sqrt{k}$, instead of linearly in $k$ as in the central model. Stronger group privacy yields improved max-information guarantees, as well as stronger lower bounds (via "packing arguments"), over the central model. $\bullet$ Building on a transformation of Bassily and Smith (STOC 2015), we give a generic transformation from any non-interactive approximate-private local protocol into a pure-private local protocol. Again in contrast with the central model, this shows that we cannot obtain more accurate algorithms by moving from pure to approximate local privacy.
Neural Networks Architecture Evaluation in a Quantum Computer
Venue: arXiv cs.NE 2017
Year: 2017

In this work, we propose a quantum algorithm to evaluate neural networks architectures named Quantum Neural Network Architecture Evaluation (QNNAE). The proposed algorithm is based on a quantum associative memory and the learning algorithm for artificial neural networks. Unlike conventional algorithms for evaluating neural network architectures, QNNAE does not depend on initialization of weights. The proposed algorithm has a binary output and results in 0 with probability proportional to the performance of the network. And its computational cost is equal to the computational cost to train a neural network.
From Word Segmentation to POS Tagging for Vietnamese
Venue: arXiv cs.CL 2017
Year: 2017

This paper presents an empirical comparison of two strategies for Vietnamese Part-of-Speech (POS) tagging from unsegmented text: (i) a pipeline strategy where we consider the output of a word segmenter as the input of a POS tagger, and (ii) a joint strategy where we predict a combined segmentation and POS tag for each syllable. We also make a comparison between state-of-the-art (SOTA) feature-based and neural network-based models. On the benchmark Vietnamese treebank (Nguyen et al., 2009), experimental results show that the pipeline strategy produces better scores of POS tagging from unsegmented text than the joint strategy, and the highest accuracy is obtained by using a feature-based model.
Near-optimal sample complexity for convex tensor completion
Venue: arXiv cs.LG 2017
Year: 2017

We analyze low rank tensor completion (TC) using noisy measurements of a subset of the tensor. Assuming a rank-$r$, order-$d$, $N \times N \times \cdots \times N$ tensor where $r=O(1)$, the best sampling complexity that was achieved is $O(N^{\frac{d}{2}})$, which is obtained by solving a tensor nuclear-norm minimization problem. However, this bound is significantly larger than the number of free variables in a low rank tensor which is $O(dN)$. In this paper, we show that by using an atomic-norm whose atoms are rank-$1$ sign tensors, one can obtain a sample complexity of $O(dN)$. Moreover, we generalize the matrix max-norm definition to tensors, which results in a max-quasi-norm (max-qnorm) whose unit ball has small Rademacher complexity. We prove that solving a constrained least squares estimation using either the convex atomic-norm or the nonconvex max-qnorm results in optimal sample complexity for the problem of low-rank tensor completion. Furthermore, we show that these bounds are nearly minimax rate-optimal. We also provide promising numerical results for max-qnorm constrained tensor completion, showing improved recovery results compared to matricization and alternating least squares.
Robust Online Speed Scaling With Deadline Uncertainty
Venue: arXiv cs.DS 2017
Year: 2017

A speed scaling problem is considered, where time is divided into slots, and jobs with payoff $v$ arrive at the beginning of the slot with associated deadlines $d$. Each job takes one slot to be processed, and multiple jobs can be processed by the server in each slot with energy cost $g(k)$ for processing $k$ jobs in one slot. The payoff is accrued by the algorithm only if the job is processed by its deadline. We consider a robust version of this speed scaling problem, where a job on its arrival reveals its payoff $v$, however, the deadline is hidden to the online algorithm, which could potentially be chosen adversarially and known to the optimal offline algorithm. The objective is to derive a robust (to deadlines) and optimal online algorithm that achieves the best competitive ratio. We propose an algorithm (called min-LCR) and show that it is an optimal online algorithm for any convex energy cost function $g(.)$. We do so without actually evaluating the optimal competitive ratio, and give a general proof that works for any convex $g$, which is rather novel. For the popular choice of energy cost function $g(k) = k^α, α\ge 2$, we give concrete bounds on the competitive ratio of the algorithm, which ranges between $2.618$ and $3$ depending on the value of $α$. The best known online algorithm for the same problem, but where deadlines are revealed to the online algorithm has competitive ratio of $2$ and a lower bound of $\sqrt{2}$. Thus, importantly, lack of deadline knowledge does not make the problem degenerate, and the effect of deadline information on the optimal competitive ratio is limited.
The return to higher education: evidence from Romania
Venue: arXiv cs.CY 2017
Year: 2017

Education is one of the most important components of the human capital, and an important determinant of the personal income. Estimating the rate of return to education is a main topic of economic research. In this paper we analyzed the rate of return to higher education in Romania using the well-known Mincer equation. Besides the educational level and the number of years of experience on the labor market we also used a series of socio-demographic variables such as gender, civil status, the area of residence. We were interested mainly in calculating the rate of return to higher education, therefore we computed this rate for bachelor, master and doctoral degrees separately. We also investigated the rate of return to higher education on technical, science, economics, law, medicine, and arts fields. Our results showed that the rate of return to higher education has a greater value than most of the developed countries of EU and the field of higher education that brings the highest rate of return is medicine
A Comparative Case Study on the Impact of Test-Driven Development on Program Design and Test Coverage
Venue: arXiv cs.SE 2017
Year: 2017

Test-driven development (TDD) is a programming technique in which the tests are written prior to the source code. It is proposed that TDD is one of the most fundamental practices enabling the development of software in an agile and iterative manner. Both the literature and practice suggest that TDD practice yields several benefits. Essentially, it is claimed that TDD leads to an improved software design, which has a dramatic impact on the maintainability and further development of the system. The impact of TDD on program design has seldom come under the researchers' focus. This paper reports the results from a comparative case study of three software development projects where the effect of TDD on program design was measured using object-oriented metrics. The results show that the effect of TDD on program design was not as evident as expected, but the test coverage was significantly superior to iterative test-last development.
Web Robot Detection in Academic Publishing
Venue: arXiv cs.AI 2017
Year: 2017

Recent industry reports assure the rise of web robots which comprise more than half of the total web traffic. They not only threaten the security, privacy and efficiency of the web but they also distort analytics and metrics, doubting the veracity of the information being promoted. In the academic publishing domain, this can cause articles to be faulty presented as prominent and influential. In this paper, we present our approach on detecting web robots in academic publishing websites. We use different supervised learning algorithms with a variety of characteristics deriving from both the log files of the server and the content served by the website. Our approach relies on the assumption that human users will be interested in specific domains or articles, while web robots crawl a web library incoherently. We experiment with features adopted in previous studies with the addition of novel semantic characteristics which derive after performing a semantic analysis using the Latent Dirichlet Allocation (LDA) algorithm. Our real-world case study shows promising results, pinpointing the significance of semantic features in the web robot detection problem.
On Extending Neural Networks with Loss Ensembles for Text Classification
Venue: arXiv cs.CL 2017
Year: 2017

Ensemble techniques are powerful approaches that combine several weak learners to build a stronger one. As a meta learning framework, ensemble techniques can easily be applied to many machine learning techniques. In this paper we propose a neural network extended with an ensemble loss function for text classification. The weight of each weak loss function is tuned within the training phase through the gradient propagation optimization method of the neural network. The approach is evaluated on several text classification datasets. We also evaluate its performance in various environments with several degrees of label noise. Experimental results indicate an improvement of the results and strong resilience against label noise in comparison with other methods.
LAA LTE and WiFi based Smart Grid Metering Infrastructure in 3.5 GHz Band
Venue: arXiv cs.IT 2017
Year: 2017

Advanced metering infrastructure (AMI) of smart grid requires bidirectional communication for transferring data to billing center, for which WiFi is an attractive choice. However, WiFi operates in the unlicensed bands and LTE needs to offload data in the same unlicensed band. Recent release of 3.5 GHz (also termed as citizen broadband radio service (CBRS)) can be an attractive shared band where LTE and WiFi can coexist. In our study, we propose a fixed duty cycled LTE-U and WiFi based smart grid metering infrastructure where smart meter uses WiFi and data collector (termed as Access Point (AP)) of smart meters uses LTE for transferring data. We investigate the coexistence performance of LTE-WiFi in the 3.5 GHz band using a time division duplexing (TDD) LTE confederated by WiFi along with FTP traffic model for system level simulation. The simulation results demonstrate a good neighboring coexistence between LTE and WiFi resulting a candidate AMI architecture for smart grid in the 3.5 GHz band.