Query: Improve Chinese Word Embeddings by Exploiting Internal Structure

Relevant passages:
A Convolutional Neural Network for Modelling Sentences <=SEP=> The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.

Learning representations by back-propagating errors <=SEP=> We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal ‘hidden’ units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure1.

Neural Probabilistic Language Models <=SEP=> 

Multilingual Models for Compositional Distributed Semantics <=SEP=> We present a novel technique for learning semantic representations, which extends the distributional hypothesis to multilingual data and joint-space embeddings. Our models leverage parallel data and learn to strongly align the embeddings of semantically equivalent sentences, while maintaining sufficient distance between those of dissimilar sentences. The models do not rely on word alignments or any syntactic information and are successfully applied to a number of diverse languages. We extend our approach to learn semantic representations at the document level, too. We evaluate these models on two cross-lingual document classification tasks, outperforming the prior state of the art. Through qualitative analysis and the study of pivoting effects we demonstrate that our representations are semantically plausible and can capture semantic relationships across languages without parallel data.

Modeling Documents with Deep Boltzmann Machines <=SEP=> We introduce a type of Deep Boltzmann Machine (DBM) that is suitable for extracting distributed semantic representations from a large unstructured collection of documents. We overcome the apparent difficulty of training a DBM with judicious parameter tying. This enables an efficient pretraining algorithm and a state initialization scheme for fast inference. The model can be trained just as efficiently as a standard Restricted Boltzmann Machine. Our experiments show that the model assigns better log probability to unseen data than the Replicated Softmax model. Features extracted from our model outperform LDA, Replicated Softmax, and DocNADE models on document retrieval and document classification tasks.

Sentiment Classification Using Machine Learning Techniques with Syntax Features <=SEP=> Sentiment classification has adopted machine learning techniques to improve its precision and efficiency. However, the features are always produced by basic words-bag methods without much consideration for words' syntactic properties, which could play an important role in the judgment of sentiment meanings. To remedy this, we firstly generate syntax trees of the sentences, with the analysis of syntactic features of the sentences. Then we introduce multiple sentiment features into the basic words-bag features. Such features were trained on movie reviews as data, with machine learning methods (Naive Bayes and support vector machines). The features and factors introduced by syntax tree were examined to generate a more accurate solution for sentiment classification.

Energy-Brushes: Interactive Tools for Illustrating Stylized Elemental Dynamics <=SEP=> Dynamic effects such as waves, splashes, fire, smoke, and explosions are an integral part of stylized animations. However, such dynamics are challenging to produce, as manually sketching key-frames requires significant effort and artistic expertise while physical simulation tools lack sufficient expressiveness and user control. We present an interactive interface for designing these elemental dynamics for animated illustrations. Users draw with coarse-scale energy brushes which serve as control gestures to drive detailed flow particles which represent local velocity fields. These fields can convey both realistic and artistic effects based on user specification. This painting metaphor for creating elemental dynamics simplifies the process, providing artistic control, and preserves the fluidity of sketching. Our system is fast, stable, and intuitive. An initial user evaluation shows that even novice users with no prior animation experience can create intriguing dynamics using our system.

DeepPore: Fingerprint Pore Extraction Using Deep Convolutional Neural Networks <=SEP=> As technological developments have enabled high-quality fingerprint scanning, sweat pores, one of the Level 3 features of fingerprints, have been successfully used in automatic fingerprint recognition systems (AFRS). Since the pore extraction process is a critical step for AFRS, high accuracy is required. However, it is difficult to extract the pore correctly because the pore shape depends on the person, region, and pore type. To solve the problem, we have presented a pore extraction method using deep convolutional neural networks and pore intensity refinement. The deep networks are used to detect pores in detail using a large area of a fingerprint image. We then refine the pore information by finding local maxima to identify pores with different intensities in the fingerprint image. The experimental results show that our pore extraction method performs better than the state-of-the-art methods.

SEM-PPA: A semantical pattern and preference-aware service mining method for personalized point of interest recommendation <=SEP=> Point-of-Interest (POI) recommendation has received increasing attention in Location-based Social Networks (LBSNs). It involves user behavior analysis, movement pattern model and trajectory sequence prediction, in order to recommend personalized services to target user. Existing POI recommendation methods are confronted with three problems: (1) they only consider the location information of users' check-ins, which causes data sparsity; (2) they fail to consider the order of users' visited locations, which is valuable to reflect the interest or preference of users; (3) users cannot be recommended the suitable services when they move into the new place. To address the above issues, we propose a semantical pattern and preference-aware service mining method called SEM-PPA to make full use of the semantic information of locations for personalized POI recommendation. In SEM-PPA, we firstly propose a novel algorithm to classify the locations into different types for location identification; then we construct the user model for each user from four aspects, which are location trajectory, semantic trajectory, location popularity and user familiarity; in addition, a potential friends discovery algorithm based on movement pattern is proposed. Finally, we conduct extensive experiments to evaluate the recommendation accuracy and recommendation effectiveness on two real-life datasets from GeoLife and Beijing POI. Experimental results show that SEM-PPA can achieve better recommendation performance in particular for sparse data and recommendation accuracy in comparison with other methods.

Depixelizing pixel art <=SEP=> We describe a novel algorithm for extracting a resolution-independent vector representation from pixel art images, which enables magnifying the results by an arbitrary amount without image degradation. Our algorithm resolves pixel-scale features in the input and converts them into regions with smoothly varying shading that are crisply separated by piecewise-smooth contour curves. In the original image, pixels are represented on a square pixel lattice, where diagonal neighbors are only connected through a single point. This causes thin features to become visually disconnected under magnification by conventional means, and creates ambiguities in the connectedness and separation of diagonal neighbors. The key to our algorithm is in resolving these ambiguities. This enables us to reshape the pixel cells so that neighboring pixels belonging to the same feature are connected through edges, thereby preserving the feature connectivity under magnification. We reduce pixel aliasing artifacts and improve smoothness by fitting spline curves to contours in the image and optimizing their control points.

Morphological, molecular and statistical tools to identify Castanea species and their hybrids <=SEP=> The aim of this study was to investigate, in relation to reference samples, the genetic structure of several natural Castanea populations of unknown origin and of hybrids from the clone collection of the Lourizán Forest Research Center. A total of 115 individuals sampled from four Castanea sativa stands located in the northwest of Spain, 61 Castanea crenata individuals and 27 Castanea mollissima individuals were classified on the basis of four morphological traits, and genotyped with 11 microsatellite loci to define a set of reference samples. The data analyzed with the program STRUCTURE detected four clusters: the two Asiatic species and two clusters in C. sativa. From these reference samples, pure individuals and hybrids of known genealogy were simulated to determine the efficiency with which STRUCTURE and NEWHYBRIDS assigned them to pure or hybrid groups and to a specific genealogical class, respectively. As expected, the discrimination and assignment of the simulated individuals improved with increasing F st value. The two clusters identified within C. sativa may correspond to gene pools with different adaptive characteristics previously identified in provenance tests; pure and admixed populations of both C. sativa gene pools were identified.

Compact slow-wave millimeter-wave bandpass filter (BPF) using open-loop resonator <=SEP=> A compact slow-wave millimeter-wave bandpass filter for wireless communication systems is developed and proposed in this paper. The filter is based on an open-loop resonator that is composed of a microstrip line with both ends loaded with folded open stubs. The folded arms of the open stubs are utilized to increase the loading capacitance to ground. In addition, the folded arms of the open stubs are also used to introduce cross couplings. The cross couplings can create transmission zeros at the edges of the desired passband. As a result, the filter can exhibit a passband with sharp rejection skirts. The filter is designed to have a fractional bandwidth of about 3.3% at a center frequency of 30.0 GHz. The filter design is realized using Rogers RT6002 substrate with a dielectric constant of 2.94 and a thickness of 0.127 mm. The filter design is successfully fabricated using the printed circuit board (PCB) technology. The fabricated filter is measured and an excellent agreement between the simulated and measured results is achieved. The fabricated filter has the advantages of compact size and excellent performance.

Context-aware application scheduling in mobile systems: what will users do and not do next? <=SEP=> Usage patterns of mobile devices depend on a variety of factors such as time, location, and previous actions. Hence, context-awareness can be the key to make mobile systems to become personalized and situation dependent in managing their resources. We first reveal new findings from our own Android user experiment: (i) the launching probabilities of applications follow Zipf's law, and (ii) inter-running and running times of applications conform to log-normal distributions. We also find context-dependency in application usage patterns, for which we classify contexts in a personalized manner with unsupervised learning methods. Using the knowledge acquired, we develop a novel context-aware application scheduling framework, CAS that adaptively unloads and preloads background applications in a timely manner. Our trace-driven simulations with 96 user traces demonstrate the benefits of CAS over existing algorithms. We also verify the practicality of CAS by implementing it on the Android platform.

HUmanoid Robotic Leg via pneumatic muscle actuators : implementation and control <=SEP=> In this article, a HUmanoid Robotic Leg (HURL) via the utilization of pneumatic muscle actuators (PMAs) is presented. PMAs are a pneumatic form of actuation possessing crucial attributes for the implementation of a design that mimics the motion characteristics of a human ankle. HURL acts as a feasibility study in the conceptual goal of developing a 10 degree-of-freedom (DoF) lower-limb humanoid for compliance and postural control, while serving as a knowledge basis for its future alternative use in prosthetic robotics. HURL’s design properties are described in detail, while its 2-DoF motion capabilities (dorsiflexion–plantar flexion, eversion–inversion) are experimentally evaluated via an advanced nonlinear PID-based control algorithm.

Design Challenges/Solutions for Environments Supporting the Analysis of Social Media Data in Crisis Informatics Research <=SEP=> Crisis informatics investigates how society's pervasive access to technology is transforming how it responds to mass emergency events. To study this transformation, researchers require access to large sets of data that because of their volume and heterogeneous nature are difficult to collect and analyze. To address this concern, we have designed and implemented an environment - EPIC Analyze - that supports researchers with the collection and analysis of social media data. Our research has identified the types of components - such as NoSQL, MapReduce, caching, and search - needed to ensure that these services are reliable, scalable, extensible, and efficient. We describe the design challenges encountered - such as data modeling, time vs. Space tradeoffs, and the need for a useful and usable system - when building EPIC Analyze and discuss its scalability, performance, and functionality.

Evasion and Hardening of Tree Ensemble Classifiers <=SEP=> Classifier evasion consists in finding for a given instance x the “nearest” instance x′ such that the classifier predictions of x and x′ are different. We present two novel algorithms for systematically computing evasions for tree ensembles such as boosted trees and random forests. Our first algorithm uses a Mixed Integer Linear Program solver and finds the optimal evading instance under an expressive set of constraints. Our second algorithm trades off optimality for speed by using symbolic prediction, a novel algorithm for fast finite differences on tree ensembles. On a digit recognition task, we demonstrate that both gradient boosted trees and random forests are extremely susceptible to evasions. Finally, we harden a boosted tree model without loss of predictive accuracy by augmenting the training set of each boosting round with evading instances, a technique we call adversarial boosting.

Synaptic basis of cortical persistent activity: the importance of NMDA receptors to working memory. <=SEP=> Delay-period activity of prefrontal cortical cells, the neural hallmark of working memory, is generally assumed to be sustained by reverberating synaptic excitation in the prefrontal cortical circuit. Previous model studies of working memory emphasized the high efficacy of recurrent synapses, but did not investigate the role of temporal synaptic dynamics. In this theoretical work, I show that biophysical properties of cortical synaptic transmission are important to the generation and stabilization of a network persistent state. This is especially the case when negative feedback mechanisms (such as spike-frequency adaptation, feedback shunting inhibition, and short-term depression of recurrent excitatory synapses) are included so that the neural firing rates are controlled within a physiological range (10-50 Hz), in spite of the exuberant recurrent excitation. Moreover, it is found that, to achieve a stable persistent state, recurrent excitatory synapses must be dominated by a slow component. If neuronal firings are asynchronous, the synaptic decay time constant needs to be comparable to that of the negative feedback; whereas in the case of partially synchronous dynamics, it needs to be comparable to a typical interspike interval (or oscillation period). Slow synaptic current kinetics also leads to the saturation of synaptic drive at high firing frequencies that contributes to rate control in a persistent state. For these reasons the slow NMDA receptor-mediated synaptic transmission is likely required for sustaining persistent network activity at low firing rates. This result suggests a critical role of the NMDA receptor channels in normal working memory function of the prefrontal cortex.

A survey of multicore processors <=SEP=> General-purpose multicore processors are being accepted in all segments of the industry, including signal processing and embedded space, as the need for more performance and general-purpose programmability has grown. Parallel processing increases performance by adding more parallel resources while maintaining manageable power characteristics. The implementations of multicore processors are numerous and diverse. Designs range from conventional multiprocessor machines to designs that consist of a "sea" of programmable arithmetic logic units (ALUs). In this article, we cover some of the attributes common to all multicore processor implementations and illustrate these attributes with current and future commercial multicore designs. The characteristics we focus on are application domain, power/performance, processing elements, memory system, and accelerators/integrated peripherals.

Diabetes Disease Diagnosis Method based on Feature Extraction using K-SVM <=SEP=> Now-a-days, diabetes disease is considered one of the key reasons of death among the people in the world. The availability of extensive medical information leads to the search for proper tools to support physicians to diagnose diabetes disease accurately. This research aimed at improving the diagnostic accuracy and reducing diagnostic missclassification based on the extracted significant diabetes features. Feature selection is critical to the superiority of classifiers founded through knowledge discovery approaches, thereby solving the classification problems relating to diabetes patients. This study proposed an integration approach between the SVM technique and K-means clustering algorithms to diagnose diabetes disease. Experimental results achieved high accuracy for differentiating the hidden patterns of the Diabetic and Nondiabetic patients compared with the modern diagnosis methods in term of the performance measure. The T-test statistical method obtained significant improvement results based on KSVM technique when tested on the UCI Pima Indian standard dataset. Keywords—K-means Clustering; Diabetes Patients; SVM;

Detecting Code Smells in Python Programs <=SEP=> As a traditional dynamic language, Python is increasingly used in various software engineering tasks. However, due to its flexibility and dynamism, Python is a particularly challenging language to write code in and maintain. Consequently, Python programs contain code smells which indicate potential comprehension and maintenance problems. With the aim of supporting refactoring strategies to enhance maintainability, this paper describes how to detect code smells in Python programs. We introduce 11 Python smells and describe the detection strategy. We also implement a smell detection tool named Pysmell and use it to identify code smells in five real world Python systems. The results show that Pysmell can detect 285 code smell instances in total with the average precision of 97.7%. It reveals that Large Class and Large Method are most prevalent. Our experiment also implies Python programs may be suffering code smells further.

End-to-end text recognition with convolutional neural networks <=SEP=> Full end-to-end text recognition in natural images is a challenging problem that has received much attention recently. Traditional systems in this area have relied on elaborate models incorporating carefully hand-engineered features or large amounts of prior knowledge. In this paper, we take a different route and combine the representational power of large, multilayer neural networks together with recent developments in unsupervised feature learning, which allows us to use a common framework to train highly-accurate text detector and character recognizer modules. Then, using only simple off-the-shelf methods, we integrate these two modules into a full end-to-end, lexicon-driven, scene text recognition system that achieves state-of-the-art performance on standard benchmarks, namely Street View Text and ICDAR 2003.

A Comparative Study of Demographic Attribute Inference in Twitter <=SEP=> Social media platforms have become a major gateway to receive and analyze public opinions. Understanding users can provide invaluable context information of their social media posts and significantly improve traditional opinion analysis models. Demographic attributes, such as ethnicity, gender, age, among others, have been extensively applied to characterize social media users. While studies have shown that user groups formed by demographic attributes can have coherent opinions towards political issues, these attributes are often not explicitly coded by users through their profiles. Previous work has demonstrated the effectiveness of different user signals such as users’ posts and names in determining demographic attributes. Yet, these efforts mostly evaluate linguistic signals from users’ posts and train models from artificially balanced datasets. In this paper, we propose a comprehensive list of user signals: self-descriptions and posts aggregated from users’ friends and followers, users’ profile images, and users’ names. We provide a comparative study of these signals side-by-side in the tasks on inferring three major demographic attributes, namely ethnicity, gender, and age. We utilize a realistic unbalanced datasets that share similar demographic makeups in Twitter for training models and evaluation experiments. Our experiments indicate that self-descriptions provide the strongest signal for ethnicity and age inference and clearly improve the overall performance when combined with tweets. Profile images for gender inference have the highest precision score with overall score close to the best result in our setting. This suggests that signals in self-descriptions and profile images have potentials to facilitate demographic attribute inferences in Twitter, and are promising for future investigation.

Imaging cellular network dynamics in three dimensions using fast 3D laser scanning <=SEP=> Spatiotemporal activity patterns in three-dimensionally organized cellular networks are fundamental to the function of the nervous system. Despite advances in functional imaging of cell populations, a method to resolve local network activity in three dimensions has been lacking. Here we introduce a three-dimensional (3D) line-scan technology for two-photon microscopy that permits fast fluorescence measurements from several hundred cells distributed in 3D space. We combined sinusoidal vibration of the microscope objective at 10 Hz with 'smart' movements of galvanometric x-y scanners to repeatedly scan the laser focus along a closed 3D trajectory. More than 90% of cell somata were sampled by the scan line within volumes of 250 μm side length. Using bulk-loading of calcium indicator, we applied this method to reveal spatiotemporal activity patterns in neuronal and astrocytic networks in the rat neocortex in vivo. Two-photon population imaging using 3D scanning opens the field for comprehensive studies of local network dynamics in intact tissue.

Autoencoder node saliency: Selecting relevant latent representations <=SEP=> The autoencoder is an artificial neural network that learns hidden representations of unlabeled data. With a linear transfer function it is similar to the principal component analysis (PCA). While both methods use weight vectors for linear transformations, the autoencoder does not come with any indication similar to the eigenvalues in PCA that are paired with eigenvectors. We propose a novel supervised node saliency (SNS) method that ranks the hidden nodes, which contain weight vectors for transformations. SNS is able to indicate the nodes specialized in a learning task. The latent representations of a hidden node can be described using a one-dimensional histogram. We apply normalized entropy difference (NED) to measure the ”interestingness” of the histograms, and conclude a property for NED values to identify a good classifying node. By applying our methods to real datasets, we demonstrate their ability to find valuable nodes and explain the learned tasks in autoencoders.

Syntax Annotation for the GENIA Corpus <=SEP=> Linguistically annotated corpus based on texts in biomedical domain has been constructed to tune natural language processing (NLP) tools for biotextmining. As the focus of information extraction is shifting from "nominal" information such as named entity to "verbal" information such as function and interaction of substances, application of parsers has become one of the key technologies and thus the corpus annotated for syntactic structure of sentences is in demand. A subset of the GENIA corpus consisting of 500 MEDLINE abstracts has been annotated for syntactic structure in an XMLbased format based on Penn Treebank II (PTB) scheme. Inter-annotator agreement test indicated that the writing style rather than the contents of the research abstracts is the source of the difficulty in tree annotation, and that annotation can be stably done by linguists without much knowledge of biology with appropriate guidelines regarding to linguistic phenomena particular to scientific texts.

Algorithms for time series knowledge mining <=SEP=> Temporal patterns composed of symbolic intervals are commonly formulated with Allen's interval relations originating in temporal reasoning. This representation has severe disadvantages for knowledge discovery. The Time Series Knowledge Representation (TSKR) is a new hierarchical language for interval patterns expressing the temporal concepts of coincidence and partial order. We present effective and efficient mining algorithms for such patterns based on itemset techniques. A novel form of search space pruning effectively reduces the size of the mining result to ease interpretation and speed up the algorithms. On a real data set a concise set of TSKR patterns can explain the underlying temporal phenomena, whereas the patterns found with Allen's relations are far more numerous yet only explain fragments of the data.

Kodu Game Lab: a programming environment <=SEP=> Kodu Game Lab is a tile-based visual programming tool that enables users to learn programming concepts through making and playing computer games. Kodu is a relatively new programming language designed specifically for young children to learn through independent exploration. It is integrated in a real-time isometric 3D gaming environment that is designed to compete with modern console games in terms of intuitive user interface and graphical production values.

KairosVM: Deterministic introspection for real-time virtual machine hierarchical scheduling <=SEP=> Consolidation and isolation are key technologies that promoted the undisputed popularity of virtualization in most of the computer industry. This popularity has recently led to a growing interest in real-time virtualization, making this technology enter the real-time system market. However, it has several issues due to the strict timing guarantees contracted. Moreover supporting legacy software stacks adds another level of complexity when the software is a black box. We present KairosVM, a latency-bounded, real-time extension to Linux's KVM module. It aims to bridge the lack of communication of the real-time requirements between the guest scheduler and the host scheduler, exploiting virtual machine introspection. The hypervisor captures the real-time requirements of the guest by catching previously added undefined instructions, without the need to do any modification to the guests. Our evaluations show that KairosVM's overhead is negligible when compared to existing introspection solutions thus can be used in real-time.

Music Feature Maps with Convolutional Neural Networks for Music Genre Classification <=SEP=> Nowadays, deep learning is more and more used for Music Genre Classification: particularly Convolutional Neural Networks (CNN) taking as entry a spectrogram considered as an image on which are sought different types of structure.
 But, facing the criticism relating to the difficulty in understanding the underlying relationships that neural networks learn in presence of a spectrogram, we propose to use, as entries of a CNN, a small set of eight music features chosen along three main music dimensions: dynamics, timbre and tonality. With CNNs trained in such a way that filter dimensions are interpretable in time and frequency, results show that only eight music features are more efficient than 513 frequency bins of a spectrogram and that late score fusion between systems based on both feature types reaches 91% accuracy on the GTZAN database.

A study of the documentation essential to software maintenance <=SEP=> Software engineering has been striving for years to improve the practice of software development and maintenance. Documentation has long been prominent on the list of recommended practices to improve development and help maintenance. Recently however, agile methods started to shake this view, arguing that the goal of the game is to produce software and that documentation is only useful as long as it helps to reach this goal.On the other hand, in the re-engineering field, people wish they could re-document useful legacy software so that they may continue maintain them or migrate them to new platform.In these two case, a crucial question arises: "How much documentation is enough?" In this article, we present the results of a survey of software maintainers to try to establish what documentation artifacts are the most useful to them.