Query: Exponential Moving Average Model in Parallel Speech Recognition Training

Relevant passages:
Large Scale Distributed Deep Networks <=SEP=> Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider the problem of training a deep network with billions of parameters using tens of thousands of CPU cores. We have developed a software framework called DistBelief that can utilize computing clusters with thousands of machines to train large models. Within this framework, we have developed two algorithms for large-scale distributed training: (i) Downpour SGD, an asynchronous stochastic gradient descent procedure supporting a large number of model replicas, and (ii) Sandblaster, a framework that supports a variety of distributed batch optimization procedures, including a distributed implementation of L-BFGS. Downpour SGD and Sandblaster L-BFGS both increase the scale and speed of deep network training. We have successfully used our system to train a deep network 30x larger than previously reported in the literature, and achieves state-of-the-art performance on ImageNet, a visual object recognition task with 16 million images and 21k categories. We show that these same techniques dramatically accelerate the training of a more modestlysized deep network for a commercial speech recognition service. Although we focus on and report performance of these methods as applied to training large neural networks, the underlying algorithms are applicable to any gradient-based machine learning algorithm.

Speech recognition with deep recurrent neural networks <=SEP=> Recurrent neural networks (RNNs) are a powerful model for sequential data. End-to-end training methods such as Connectionist Temporal Classification make it possible to train RNNs for sequence labelling problems where the input-output alignment is unknown. The combination of these methods with the Long Short-term Memory RNN architecture has proved particularly fruitful, delivering state-of-the-art results in cursive handwriting recognition. However RNN performance in speech recognition has so far been disappointing, with better results returned by deep feedforward networks. This paper investigates deep recurrent neural networks, which combine the multiple levels of representation that have proved so effective in deep networks with the flexible use of long range context that empowers RNNs. When trained end-to-end with suitable regularisation, we find that deep Long Short-term Memory RNNs achieve a test set error of 17.7% on the TIMIT phoneme recognition benchmark, which to our knowledge is the best recorded score.

Revisiting Distributed Synchronous SGD <=SEP=> Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the cost of additional noise introduced from asynchrony. In contrast, the synchronous approach is often thought to be impractical due to idle time wasted on waiting for straggling workers. We revisit these conventional beliefs in this paper, and examine the weaknesses of both approaches. We demonstrate that a third approach, synchronous optimization with backup workers, can avoid asynchronous noise while mitigating for the worst stragglers. Our approach is empirically validated and shown to converge faster and to better test accuracies.

Deep Residual Learning for Image Recognition <=SEP=> Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

Highway long short-term memory RNNS for distant speech recognition <=SEP=> In this paper, we extend the deep long short-term memory (DL-STM) recurrent neural networks by introducing gated direct connections between memory cells in adjacent layers. These direct links, called highway connections, enable unimpeded information flow across different layers and thus alleviate the gradient vanishing problem when building deeper LSTMs. We further introduce the latency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole history while keeping the latency under control. Efficient algorithms are proposed to train these novel networks using both frame and sequence discriminative criteria. Experiments on the AMI distant speech recognition (DSR) task indicate that we can train deeper LSTMs and achieve better improvement from sequence training with highway LSTMs (HLSTMs). Our novel model obtains 43.9/47.7% WER on AMI (SDM) dev and eval sets, outperforming all previous works. It beats the strong DNN and DLSTM baselines with 15.7% and 5.3% relative improvement respectively.

A REVIEW OF POINT CLOUDS SEGMENTATION AND CLASSIFICATION ALGORITHMS <=SEP=> Today 3D models and point clouds are very popular being currently used in several fields, shared through the internet and even accessed on mobile phones. Despite their broad availability, there is still a relevant need of methods, preferably automatic, to provide 3D data with meaningful attributes that characterize and provide significance to the objects represented in 3D. Segmentation is the process of grouping point clouds into multiple homogeneous regions with similar properties whereas classification is the step that labels these regions. The main goal of this paper is to analyse the most popular methodologies and algorithms to segment and classify 3D point clouds. Strong and weak points of the different solutions presented in literature or implemented in commercial software will be listed and shortly explained. For some algorithms, the results of the segmentation and classification is shown using real examples at different scale in the Cultural Heritage field. Finally, open issues and research topics will be discussed.

Inferring user demographics and social strategies in mobile social networks <=SEP=> Demographics are widely used in marketing to characterize different types of customers. However, in practice, demographic information such as age, gender, and location is usually unavailable due to privacy and other reasons. In this paper, we aim to harness the power of big data to automatically infer users' demographics based on their daily mobile communication patterns. Our study is based on a real-world large mobile network of more than 7,000,000 users and over 1,000,000,000 communication records (CALL and SMS). We discover several interesting social strategies that mobile users frequently use to maintain their social connections. First, young people are very active in broadening their social circles, while seniors tend to keep close but more stable connections. Second, female users put more attention on cross-generation interactions than male users, though interactions between male and female users are frequent. Third, a persistent same-gender triadic pattern over one's lifetime is discovered for the first time, while more complex opposite-gender triadic patterns are only exhibited among young people.
 We further study to what extent users' demographics can be inferred from their mobile communications. As a special case, we formalize a problem of double dependent-variable prediction-inferring user gender and age simultaneously. We propose the WhoAmI method, a Double Dependent-Variable Factor Graph Model, to address this problem by considering not only the effects of features on gender/age, but also the interrelation between gender and age. Our experiments show that the proposed WhoAmI method significantly improves the prediction accuracy by up to 10% compared with several alternative methods.

An implementation of the earliest deadline first algorithm in Linux <=SEP=> Recently, many projects have been started to introduce some real-time mechanisms into general purpose operating systems (GPOS) in order to make them capable of providing the users with some temporal guarantees. Many of these projects focused especially on Linux for its capillary and widespread adoption throughout many different research and industrial environments.
 By tracking the kernel release cycle, we propose an efficient Earliest Deadline First implementation in the form of a patch-set against the 2.6.27 version, that is the latest released one, as of now. Our implementation provides the user with the possibility to choose SCHED_EDF as one of the possible scheduling policies for a task, with an enhanced version of the standard algorithm. In fact, we propose a new approach to shared resources' access which, differently from many other previous existing works, does not require the user to specify any parameters about the critical sections every task will enter during its execution.

Embedded 3D printing of strain sensors within highly stretchable elastomers. <=SEP=> A new method, embedded-3D printing (e-3DP), is reported for fabricating strain sensors within highly conformal and extensible elastomeric matrices. e-3DP allows soft sensors to be created in nearly arbitrary planar and 3D motifs in a highly programmable and seamless manner. Several embodiments are demonstrated and sensor performance is characterized.

Path-space manipulation of physically-based light transport <=SEP=> Industry-quality content creation relies on tools for lighting artists to quickly prototype, iterate, and refine final renders. As industry-leading studios quickly adopt physically-based rendering (PBR) across their art generation pipelines, many existing tools have become unsuitable as they address only simple effects without considering underlying PBR concepts and constraints. We present a novel light transport manipulation technique that operates directly on path-space solutions of the rendering equation. We expose intuitive direct and indirect manipulation approaches to edit complex effects such as (multi-refracted) caustics, diffuse and glossy indirect bounces, and direct/indirect shadows. With our sketch- and object-space selection, all built atop a parameterized regular expression engine, artists can search and isolate shading effects to inspect and edit. We classify and filter paths on the fly and visualize the selected transport phenomena. We survey artists who used our tool to manipulate complex phenomena on both static and animated scenes.

Commutation torque ripple reduction in brushless DC motor drives using a single DC current sensor <=SEP=> This paper presents a comprehensive study on reducing commutation torque ripples generated in brushless DC motor drives with only a single DC-link current sensor provided. In such drives, commutation torque ripple suppression techniques that are practically effective in low speed as well as high speed regions are scarcely found. The commutation compensation technique proposed here is based on a strategy that the current slopes of the incoming and the outgoing phases during the commutation interval can be equalized by a proper duty-ratio control. Being directly linked with deadbeat current control scheme, the proposed control method accomplishes suppression of the spikes and dips superimposed on the current and torque responses during the commutation intervals of the inverter. Effectiveness of the proposed control method is verified through simulations and experiments.

What's Your Next Move: User Activity Prediction in Location-based Social Networks <=SEP=> Location-based social networks have been gaining increasing popularity in recent years. To increase users’ engagement with location-based services, it is important to provide attractive features, one of which is geo-targeted ads and coupons. To make ads and coupon delivery more effective, it is essential to predict the location that is most likely to be visited by a user at the next step. However, an inherent challenge in location prediction is a huge prediction space, with millions of distinct check-in locations as prediction target. In this paper we exploit the check-in category information to model the underlying user movement pattern. We propose a framework which uses a mixed hidden Markov model to predict the category of user activity at the next step and then predict the most likely location given the estimated category distribution. The advantages of modeling the category level include a significantly reduced prediction space and a precise expression of the semantic meaning of user activities. Extensive experimental results show that, with the predicted category distribution, the number of location candidates for prediction is 5.45 times smaller, while the prediction accuracy is 13.21% higher.

Informal social communication. <=SEP=> 

Texture Segmentation Based Video Compression Using Convolutional Neural Networks <=SEP=> There has been a growing interest in using different approaches to improve the coding efficiency of modern video codec in recent years as demand for web-based video consumption increases. In this paper, we propose a model-based approach that uses texture analysis/synthesis to reconstruct blocks in texture regions of a video to achieve potential coding gains using the AV1 codec developed by the Alliance for Open Media (AOM). The proposed method uses convolutional neural networks to extract texture regions in a frame, which are then reconstructed using a global motion model. Our preliminary results show an increase in coding efficiency while maintaining satisfactory visual quality. Introduction With the increasing amount of videos being created and consumed, better video compression tools are needed to provide fast transmission and high visual quality. Modern video coding standards utilize spatial and temporal redundancy in the videos to achieve high coding efficiency and high visual quality with motion compensation techniques and 2-D orthogonal transforms. However, efficient exploitation of statistical dependencies measured by a mean squared error (MSE) does not always produce the best psychovisual result, and may require higher data rate to preserve detail information in the video. Recent advancement in GPU computing has enabled the analysis of large scale data using deep learning method. Deep learning techniques have shown promising performance in many applications such as object detection, natural language process, and synthetic images generation [1, 2, 3, 4]. Several methods have been developed for video applications to improve coding efficiency using deep learning. In [5], sample adaptive offset (SAO) is replaced by a CNN-based in-loop filter (IFCNN) to improve the coding efficiency in HEVC. By learning the predicted residue between the quantized reconstructed frames obtained after deblocking filter (DF) and the original frames, IFCNN is able to reconstruct video frames with higher quality without requiring any bit transmission during coding process. Similar to [5], [6] proposes a Variable-filter-size Residue-learning CNN (VRCNN) to improving coding efficiency by replacing DF and SAO in HEVC. VRCNN is based on the concept of ARCNN [7] which is originally designed for JPEG applications. Instead of only using spatial information to train a CNN to reduce the coding artifacts in HEVC, [8] proposed a spatial temporal residue network (STResNet) as an additional in-loop filter after SAO. A rate-distortion optimization strategy is used to control the on/off switch of the proposed in-loop filter. There are also some works that have been done in the decoder of HEVC to improve the coding efficiency. In [9], a deep CNN-based auto decoder (DCAD) is implemented in the decoder of HEVC to improve the video quality of decoded video. DCAD is trained to learn the predict residue between decoded video frames and original video frames. By adding the predicted residual generated from DCAD to the compressed video frames, this method enhances the compressed video frames to higher quality. In summary, the above methods improve the coding efficiency by enhancing the quality of reconstructed video frames. However, they require different trained models for video reconstruction at different quantization levels. We are interested in developing deep learning approaches to only encode visually relevant information and use a different coding method for “perceptually insignificant” regions in a frame, which can lead to substantial data rate reductions while maintaining visual quality. In particular, we have developed a model based approach that can be used to improve the coding efficiency by identifying texture areas in a video frame that contain detail irrelevant information, which the viewer does not perceive specific details and can be skipped or encoded at a much lower data rate. The task is then to divide a frame into “perceptually insignificant” texture region and then use a texture model for the pixels in that region. In 1959, Schreiber and colleagues proposed a coding method that divides an image into textures and edges and used it in image coding [10]. This work was later extended by using the human visual system and statistical model to determine the texture region [11, 12, 13]. More recently, several groups have focused on adapting perceptual based approaches to the video coding framework [14]. In our previous work [15], we introduced a texture analyzer before encoding the input sequences to identify detail irrelevant regions in the frame which are classified into different texture classes. At the encoder, no inter-frame prediction is performed for these regions. Instead, displacement of the entire texture region is modeled by just one set of motion parameters. Therefore, only the model parameters are transmitted to the decoder for reconstructing the texture regions using a texture synthesizer. Non-texture regions in the frame are coded conventionally. Since this method uses feature extraction based on texture segmentation technique, a proper set of parameters are required to achieve accurate texture segmentation for different videos. Deep learning methods usually do not require such parameter tuning for inference. As a result, deep learning techniques can be developed to perform texture segmentation and classification for the proposed model-based video coding. A Fisher vector convolution neural networks (FVCNN) that can produce segmentation labels for different texture classes was proposed in [16]. One of the advantage of FV-CNN is that the image input size is flexible and is not limited by the network architecture. Instead of doing pixel-wise classification on texture regions, a texture classification CNN network was described in [17]. To reduce computational expenses, [17] uses a small classification network to classify image patches with size of 227 × 227. A smaller network is needed to classify smaller image patches in our case. In this paper, we propose a block-based texture segmentation method to extract texture region in a video frame using convolutional neural networks. The block-based segmentation network classifies each 16 × 16 block in a frame as texture or non-texture. The identified texture region is then synthesized using the temporal correlations among the frames. Our method was implemented using the AOM/AV1 codec. Preliminary results show significant bitrate savings while maintaining a satisfactory visual quality.

Visual Descriptors for Dense Tensor Fields in Computational Turbulent Combustion: A Case Study <=SEP=> Simulation and modeling of turbulent flow, and of turbulent reacting flow in particular, involve solving for and analyzing time-dependent and spatially dense tensor quantities, such as turbulent stress tensors. The interactive visual exploration of these tensor quantities can effectively steer the computational modeling of combustion systems. In this article, the authors analyze the challenges in dense symmetric-tensor visualization as applied to turbulent combustion calculation; most notable among these challenges are the dataset size and density. They analyze, together with domain experts, the feasibility of using several established tensor visualization techniques in this application domain. They further examine and propose visual descriptors for volume rendering of the data. Of these novel descriptors, one is a density-gradient descriptor which results in Schlieren-style images, and another one is a classification descriptor inspired by machine-learning techniques. The result is a hybrid visual analysis tool to be utilized in the debugging, benchmarking and verification of models and solutions in turbulent combustion. The authors demonstrate this analysis tool on two example configurations, report feedback from combustion researchers, and summarize the design lessons learned. c © 2016 Society for Imaging Science and Technology. [DOI: 10.2352/J.ImagingSci.Technol.2016.60.1.010404] INTRODUCTION Computational simulation of turbulent combustion for gas turbine design has become increasingly important in the last two decades, due in part to environmental concerns and regulations on toxic emissions. Such modern gas turbine designs feature a variety of mixing fuel compositions and possible flow configurations,1,2 which make non-computational simulations difficult. The focus of the computational research effort in this direction is on the development of computational tools for the modeling and prediction of turbulent combustion flows. Received June 30, 2015; accepted for publication Nov. 4, 2015; published online Dec. 10, 2015. Associate Editor: Song Zhang. 1062-3701/2016/60(1)/010404/11/$25.00 Tensor quantities are common features in these turbulent combustion models. In particular, stress and strain tensors are often correlated to turbulent quantities—which appear unclosed in the mathematical formulation and thus need to be modeled as part of the computational simulation. Visual identification of the characteristics of such tensor quantities can bring significant insights into the computational modeling process. However, these computational tensor fields are very large and spatially dense—a good example of the Big Data revolution across sciences and engineering. Figure 1 shows an example turbulent combustion configuration, featuring a grid size of 106 and 6× 106 particles (shown as spheres); this dataset should be considered in contrast to traditional tensor datasets, which feature grid sizes in the 102 range. At such large scales, typical glyph encodings become cluttered and illegible. Furthermore, combustion experts seldom have an intuitive understanding of the tensor quantities. In this respect, froma tensor visualization perspective, workingwith these datasets poses an array of challenges. Are traditional tensor and flow representations useful in this context? Does increasing the level of complexity or expressiveness of such representations help or hinder? Is interaction speed more important than the benefits gained from complex descriptors? In this article, we address a specific application design problem. In the process of exploring the design space, we also investigate some of the larger visualization questions above, through the opportunity of a case study in the computational-combustion domain. In thiswork,motivated by an ongoing collaborationwith domain experts,3 we investigate the challenges associated with the exploratory visualization of tensor quantities in turbulent combustion simulations. We first provide a characterization of the problem domain, including a data analysis. Through a case study involving five senior combustion researchers, we then iteratively explore the space of tensor visual encodings. We implement and evaluate several J. Imaging Sci. Technol. 010404-1 Jan.-Feb. 2016 Marai et al.: Visual descriptors for dense tensor fields in computational turbulent combustion: a case study Figure 1. One timestep in an example turbulent combustion configuration. The grid size is 106. In this image, 6× 106 particles are shown as spheres. This dataset should be contrasted to traditional tensor datasets, which feature sparse grids in the 102 range. At this scale, typical glyph encodings become cluttered. approaches advocated by the visualization community in an interactive prototype, and we contrast these approaches with the best-of-breed visualization practices in the target domain. Based on domain expert feedback, we then focus our efforts on identifying effective visual descriptors for volume rendering of the combustion tensor data. Our contributions include a novel density-gradient descriptor and the adaptation of a machine-learning classification technique. Next, we evaluate the visual descriptors on two computational-combustion datasets of particular interest, and we show the importance of the proposed approach for debugging the numerical simulation of complex configurations. In an effort to better bridge the gap between the combustion and tensor visualization communities, we describe these tensor field datasets. Last but not least, we contribute a summary of design lessons learned from the study and from the application design process. To the best of our knowledge, this is the first formal, exploratory case study of tensor visualization techniques in the context of very large, high-density turbulent combustion flow. TENSORS IN TURBULENT COMBUSTION MODELING Turbulent Combustion Modeling. A sufficiently accurate, flexible and reliable model can be used for an in silico combustor rig test as a much cheaper alternative to the reallife rig tests employed in combustor design and optimization. In order to achieve such amodel, themethodology should be well tested and proven with lab-scale configurations. Multiple numerical approaches exist for the generation of such computational models of combustion, most notably Direct numerical simulation (DNS), Reynolds-averaged Navier–Stokes (RANS) and Large eddy simulation (LES). DNS, RANS and LES have complementary strengths. However, allmodels begin by describing the compressible reacting flow via a set of partial differential equations (PDEs) that represent the conservation of mass, momentum and energy. These PDEs are a fully coupled set of multi-dimensional non-linear equations and can be posed in a variety of forms depending on the flow conditions (compressibility, scale, flow regime, etc.).4 In this article, we exemplify the visualization of stress/strain tensors, and therefore restrict the presentation to the pertinent subset of these PDEs, namely the momentum transport equation. Stress, Strain and Turbulent Stress Tensors. A tensor is an extension of the concept of a scalar and a vector to higher orders. For example, while a stress vector is the force acting on a given unit surface, a stress tensor is defined as the components of stress vectors acting on each coordinate surface; thus, stress can be described by a symmetric second-order tensor (a matrix). The velocity stress and strain tensor fields aremanifested in the transport of fluid momentum, which is a vector quantity governed by the following conservation equation:

Towards the Semantic Web: Collaborative Tag Suggestions <=SEP=> Content organization over the Internet went through several interesting phases of evolution: from structured di rectories to unstructured Web search engines and more recently, to tagging as a way for aggregating information, a step toward s the semantic web vision. Tagging allows ranking and dat a organization to directly utilize inputs from end us er , enabling machine processing of Web content. Since tags are c reated by individual users in a free form, one important prob lem facing tagging is to identify most appropriate tags, while eliminating noise and spam. For this purpose, we define a set o f general criteria for a good tagging system. These criteria include high coverage of multiple facets to ensure good recall, least effort to reduce the cost involved in browsing, and high popu larity to ensure tag quality. We propose a collaborative tag su gestion algorithm using these criteria to spot high-quality ags. The proposed algorithm employs a goodness measure for t ags derived from collective user authorities to combat spam. Th e goodness measure is iteratively adjusted by a reward-penalty lgorithm, which also incorporates other sources of tags, e.g. , content-based auto-generated tags. Our experiments based on My We b 2.0 show that the algorithm is effective.

The Effect of Focal Loss in Semantic Segmentation of High Resolution Aerial Image <=SEP=> The semantic segmentation of High Resolution Remote Sensing (HRRS) images is the fundamental research area of the earth observation. Convolutional Neural Network (CNN), which has achieved superior performance in computer vision task, is also useful for semantic segmentation of HRRS images. In this work, focal loss is used instead of cross-entropy loss in training of CNN to handle the imbalance in training data. To evaluate the effect of focal loss, we train SegNet and FCN with focal loss and confirm improvement in accuracy in ISPRS 2D Semantic Labeling Contest dataset, especially when $\gamma$ is 0.5 in SegNet.

Co-Design for Low Warpage and High Reliability in Advanced Package with TSV-Free Interposer (TFI) <=SEP=> TSV-Free Interposer (TFI) technology eliminates TSV fabrication and reduces manufacturing and material cost. Co-design modelling methodology is established for TFI technology with considering wafer process, package assembly and package/board level reliability and thermal performance to optimize structure design, wafer process, assembly process and material selection. Experimental results are used for validating warpage modelling results. Through wafer level modelling, suitable carrier wafer and EMC materials are recommended to control wafer warpage less than 2mm. Effects of package substrate coefficient of thermal expansion (CTE) and stiffener on assembly induced package warpage are simulated to reduce package warpage. The recommended materials and geometry design based on reliability are aligned with that from wafer and package warpage simulation results. The final test vehicle (TV) design and material selection are determined based on co-design modelling results for achieving successful TFI wafer process and package assembly process and long term package/board level reliability.

Learning Fine-grained Features via a CNN Tree for Large-scale Classification <=SEP=> We propose a novel approach to enhance the discriminability of Convolutional Neural Networks (CNN). The key idea is to build a tree structure that could progressively learn fine-grained features to distinguish a subset of classes, by learning features only among these classes. Such features are expected to be more discriminative, compared to features learned for all the classes. We develop a new algorithm to effectively learn the tree structure among a large number of classes. Experiments on large-scale image classification tasks demonstrate that our method could boost the performance of a given basic CNN model. Our method is quite general, hence it can potentially be used in combination with many other deep learning models.

SIMPLIFIED SOLAR TRACKING PROTOTYPE <=SEP=> Solar energy is rapidly advancing as an important means of renewable energy resource. More energy is produced by tracking the solar panel to remain aligned to the sun at a right angle to the rays of light. This paper describes in detail the design and construction of a prototype for solar tracking system with two degrees of freedom, which detects the sunlight using photocells. The control circuit for the solar tracker is based on a PIC16F84A microcontroller (MCU). This is programmed to detect the sunlight through the photocells and then actuate the motor to position the solar panel where it can receive maximum sunlight. Keyword: PIC MCU, tracking, photocell, frame, motion system

Real-time automatic tag recommendation <=SEP=> Tags are user-generated labels for entities. Existing research on tag recommendation either focuses on improving its accuracy or on automating the process, while ignoring the efficiency issue. We propose a highly-automated novel framework for real-time tag recommendation. The tagged training documents are treated as triplets of (words, docs, tags), and represented in two bipartite graphs, which are partitioned into clusters by Spectral Recursive Embedding (SRE). Tags in each topical cluster are ranked by our novel ranking algorithm. A two-way Poisson Mixture Model (PMM) is proposed to model the document distribution into mixture components within each cluster and aggregate words into word clusters simultaneously. A new document is classified by the mixture model based on its posterior probabilities so that tags are recommended according to their ranks. Experiments on large-scale tagging datasets of scientific documents (CiteULike) and web pages del.icio.us) indicate that our framework is capable of making tag recommendation efficiently and effectively. The average tagging time for testing a document is around 1 second, with over 88% test documents correctly labeled with the top nine tags we suggested.

Superfamilies of evolved and designed networks. <=SEP=> Complex biological, technological, and sociological networks can be of very different sizes and connectivities, making it difficult to compare their structures. Here we present an approach to systematically study similarity in the local structure of networks, based on the significance profile (SP) of small subgraphs in the network compared to randomized networks. We find several superfamilies of previously unrelated networks with very similar SPs. One superfamily, including transcription networks of microorganisms, represents "rate-limited" information-processing networks strongly constrained by the response time of their components. A distinct superfamily includes protein signaling, developmental genetic networks, and neuronal wiring. Additional superfamilies include power grids, protein-structure networks and geometric networks, World Wide Web links and social networks, and word-adjacency networks from different languages.

Improving Alzheimer's disease diagnosis with machine learning techniques. <=SEP=> There is not a specific test to diagnose Alzheimer's disease (AD). Its diagnosis should be based upon clinical history, neuropsychological and laboratory tests, neuroimaging and electroencephalography (EEG). Therefore, new approaches are necessary to enable earlier and more accurate diagnosis and to follow treatment results. In this study we used a Machine Learning (ML) technique, named Support Vector Machine (SVM), to search patterns in EEG epochs to differentiate AD patients from controls. As a result, we developed a quantitative EEG (qEEG) processing method for automatic differentiation of patients with AD from normal individuals, as a complement to the diagnosis of probable dementia. We studied EEGs from 19 normal subjects (14 females/5 males, mean age 71.6 years) and 16 probable mild to moderate symptoms AD patients (14 females/2 males, mean age 73.4 years. The results obtained from analysis of EEG epochs were accuracy 79.9% and sensitivity 83.2%. The analysis considering the diagnosis of each individual patient reached 87.0% accuracy and 91.7% sensitivity.

Cloud Automation: Precomputing Roadmaps for Flexible Manipulation <=SEP=> The goal of this article is to highlight the benefits of cloud automation for industrial adopters and some of the research challenges that must be addressed in this process. The focus is on the use of cloud computing for efficiently planning the motion of new robot manipulators designed for flexible manufacturing floors. In particular, different ways that a robot can interact with a computing cloud are considered, where an architecture that splits computation between the remote cloud and the robot appears advantageous. Given this synergistic robot-cloud architecture, this article describes how solutions from the recent literature can be employed on the cloud during a periodically updated preprocessing phase to efficiently answer manipulation queries on the robot given changes in the workspace. In this setup, interesting tradeoffs arise between path quality and computational efficiency, which are evaluated through simulation. These tradeoffs motivate further research on how motion planning should be executed given access to a computing cloud.

Curriculum Learning Based on Reward Sparseness for Deep Reinforcement Learning of Task Completion Dialogue Management <=SEP=> Learning from sparse and delayed reward is a central issue in reinforcement learning. In this paper, to tackle reward sparseness problem of task oriented dialogue management, we propose a curriculum based approach on the number of slots of user goals. This curriculum makes it possible to learn dialogue management for sets of user goals with large number of slots. We also propose a dialogue policy based on progressive neural networks whose modules with parameters are appended with previous parameters fixed as the curriculum proceeds, and this policy improves performances over the one with single set of parameters.

Soft Robot Arm Inspired by the Octopus <=SEP=> The octopus is a marine animal whose body has no rigid structures. It has eight arms composed of a peculiar muscular structure, named a muscular hydrostat. The octopus arms provide it with both locomotion and grasping capabilities, thanks to the fact that their stiffness can change over a wide range and can be controlled through combined contractions of the muscles. The muscular hydrostat can better be seen as a modifiable skeleton. Furthermore, the morphology the arms and the mechanical characteristics of their tissues are such that the interaction with the environment (i.e., water) is exploited to simplify control. Thanks to this effective mechanism of embodied intelligence, the octopus can control a very high number of degrees of freedom, with relatively limited computing resources. From these considerations, the octopus emerges as a good model for embodied intelligence and for soft robotics. The prototype of a robot arm has been built based on an artificial muscular hydrostat inspired to the muscular hydrostat of the Octopus vulgaris. The prototype presents the morphology of the biological model and the broad arrangement of longitudinal and transverse muscles. Actuation is obtained with cables (longitudinally) and with shape memory alloy springs (transversally). The robot arm combines contractions and it can show the basic movements of the octopus arm, like elongation, shortening and bending, in water. © Koninklijke Brill NV, Leiden and The Robotics Society of Japan, 2012

Power Modeling and Optimization for OLED Displays <=SEP=> Emerging organic light-emitting diode (OLED)-based displays obviate external lighting, and consume drastically different power when displaying different colors, due to their emissive nature. This creates a pressing need for OLED display power models for system energy management, optimization as well as energy-efficient GUI design, given the display content or even the graphical-user interface (GUI) code. In this work, we study this opportunity using commercial QVGA OLED displays and user studies. We first present a comprehensive treatment of power modeling of OLED displays, providing models that estimate power consumption based on pixel, image, and code, respectively. These models feature various tradeoffs between computation efficiency and accuracy so that they can be employed in different layers of a mobile system. We validate the proposed models using a commercial QVGA OLED module and a mobile device with a QVGA OLED display. Then, based on the models, we propose techniques that adapt GUIs based on existing mechanisms as well as arbitrarily under usability constraints. Our measurement and user studies show that more than 75 percent display power reduction can be achieved with user acceptance.

The 12-Core POWER8™ Processor With 7.6 Tb/s IO Bandwidth, Integrated Voltage Regulation, and Resonant Clocking <=SEP=> POWER8™ is a 12-core processor fabricated in IBM's 22 nm SOI technology with core and cache improvements driven by big data applications, providing 2.5× socket performance over POWER7+™. Core throughput is supported by 7.6 Tb/s of off-chip I/O bandwidth which is provided by three primary interfaces, including two new variants of Elastic Interface as well as embedded PCI Gen-3. Power efficiency is improved with several techniques. An on-chip controller based on an embedded PowerPC™ 405 processor applies per-core DVFS by adjusting DPLLs and fully integrated voltage regulators. Each voltage regulator is a highly distributed system of digitally controlled microregulators, which achieves a peak power efficiency of 90.5%. A wide frequency range resonant clock design is used in 13 clock meshes and demonstrates a minimum power savings of 4%. Power and delay efficiency is achieved through the use of pulsed-clock latches, which require statistical validation to ensure robust yield.

Visceral pain: the ins and outs, the ups and downs. <=SEP=> PURPOSE OF REVIEW
Visceral pain represents a major clinical problem, yet far less is known about its mechanisms compared with somatic pains, for example, from cutaneous and muscular structures.


RECENT FINDINGS
In this review, we describe the neuroanatomical bases of visceral pain signalling in the peripheral and central nervous system, comparing to somatic pains and also the channels and receptors involved in these events. We include an overview of potential new targets in the context of mechanisms of visceral pain and hypersensitivity.


SUMMARY
This review should inform on the recognition of what occurs in patients with visceral pain, why comorbidities are common and how analgesic treatments work.

On the Use of Logic Trees for Ground-Motion Prediction Equations in Seismic-Hazard Analysis <=SEP=> Logic trees are widely used in probabilistic seismic hazard analysis as a tool to capture the epistemic uncertainty associated with the seismogenic sources and the ground-motion prediction models used in estimating the hazard. Combining two or more ground-motion relations within a logic tree will generally require several conversions to be made, because there are several definitions available for both the predicted ground-motion parameters and the explanatory parameters within the predictive ground-motion relations. Procedures for making conversions for each of these factors are presented, using a suite of predictive equations in current use for illustration. The sensitivity of the resulting ground-motion models to these conversions is shown to be pronounced for some of the parameters, especially the measure of source-to-site distance, highlighting the need to take into account any incompatibilities among the selected equations. Procedures are also presented for assigning weights to the branches in the ground-motion section of the logic tree in a transparent fashion, considering both intrinsic merits of the individual equations and their degree of applicability to the particular application.