Reviewerly/exHarmony huggingface dataset:
Topology-aware Sparse Allreduce for Large-scale Deep Learning

Data parallelism is the dominant method used to scale-up deep learning (DL) training across multiple compute nodes. Collective communication of the local gradients between nodes is a critical bottleneck due to the significant increase in complexity and size of DL models. Researchers cope with this problem by one of the following solutions: a) optimizing the collective communication algorithm to account for the underlying network topology (topology-aware), and b) reducing the amount of transferred data. In the latter approach, sparse communication techniques communicate only the essential data, which helps to significantly cut down the communication cost. However, the diversity of message sizes, and unknown overlap of indicessets among compute nodes (i.e. irregular size), restricts their communication into the many-to-many scheme, which is ineffective in large-scale implementations. In this paper, we present allreduce algorithms that can exploit both sparse-communication and topology-aware techniques by mixing heterogeneous data formats (dense and sparse) with a trivial cost of computation.
Exploring Segment Representations for Neural Semi-Markov Conditional Random Fields

Many problems in natural language processing (NLP) can be cast as the problem of segmenting a sequence. In this article, we combine the semi-Markov conditional random fields (semi-CRF) with neural networks to solve NLP segmentation problems. We focus on the segment representation in neural semi-CRF which is important to the performance. Based on our preliminary work in Liu et al.[1], we represent a segment by both encoding the subsequence and embedding the segment string. We conduct a systematic study of the utility of various components in subsequence encoding and propose a method of constructing and deriving segment string embeddings. Extensive experiments on three typical segmentation problems, namely, shallow syntax parsing, named entity recognition, and Chinese word segmentation are conducted. The results show that we can achieve equally-performed subsequence encoding with a three times faster concatenation network compared to previous work. The results also show that the segment string embeddings help our neural semi-CRF model to achieve a macro-averaged error reduction of 13.15% over a strong baseline using deep contextualized embeddings and bidirectional long-short-term memory CRF, which also show the usefulness of semi-CRF even with contextualized embeddings. These results are competitive with the state-of-the-art segmentation systems.
Preliminary Study of Multi Convolution Neural Network-Based Model To Identify Pills Image Using Classification Rules

Personal medicine is very important for those who have special health problems. Having several types of pills can make it hard for people to remember every pill especially aged citizen who easily forget his or her own medication. Another problem often encountered is the difficulty of recognizing the drug pills whose labels or the packaging are damaged and hard to read. This research, we developed a multi convolutional neural network (CNN) model to identify pills using classification rules. The idea of using multi CNN model is that almost all type of pills have three main identifiers, namely color, shape and imprint. Three CNNs model are developed to represent each identifier. The number of data collected is 24.000 images, which 95% of the data is used for training purpose and 5% is used for data test. The results of each CNN model is processed with some predefined rules to generate the classes of pills. From the results of different CNN architectures, number of epochs, optimizers and input size experiments, LeNet architecture with input size 64×64 pixels and Adadelta optimization shows the best accuracy up to 99.16%.
Sequential Association Rule Mining for Autonomously Extracting Hierarchical Task Structures in Reinforcement Learning

Reinforcement learning (RL) techniques, while often powerful, can suffer from slow learning speeds, particularly in high dimensional spaces or in environments with sparse rewards. The decomposition of tasks into a hierarchical structure holds the potential to significantly speed up learning, generalization, and transfer learning. However, the current task decomposition techniques often cannot extract hierarchical task structures without relying on high-level knowledge provided by an expert (e.g., using dynamic Bayesian networks (DBNs) in factored Markov decision processes), which is not necessarily available in autonomous systems. In this paper, we propose a novel method based on Sequential Association Rule Mining that can extract Hierarchical Structure of Tasks in Reinforcement Learning (SARM-HSTRL) in an autonomous manner for both Markov decision processes (MDPs) and factored MDPs. The proposed method leverages association rule mining to discover the causal and temporal relationships among states in different trajectories and extracts a task hierarchy that captures these relationships among sub-goals as termination conditions of different sub-tasks. We prove that the extracted hierarchical policy offers a hierarchically optimal policy in MDPs and factored MDPs. It should be noted that SARM-HSTRL extracts this hierarchical optimal policy without having dynamic Bayesian networks in scenarios with a single task trajectory and also with multiple tasks' trajectories. Furthermore, we show theoretically and empirically that the extracted hierarchical task structure is consistent with trajectories and provides the most efficient, reliable, and compact structure under appropriate assumptions. The numerical results compare the performance of the proposed SARM-HSTRL method with conventional HRL algorithms in terms of the accuracy in detecting the sub-goals, the validity of the extracted hierarchies, and the speed of learning in several testbeds. The key capabilities of SARM-HSTRL including handling multiple tasks and autonomous hierarchical task extraction can lead to the application of this HRL method in reusing, transferring, and generalization of knowledge in different domains.
A Multi-Agent Platform to Support Knowledge Based Modelling in Engineering Design

Nowadays engineering design process requires the involvement of multiple partners from multiple disciplines throughout the Product Life Cycle (PLC). Consequently, the construction of numerical models became a difficult task due to the distribution of experts. This article proposes an agent based approach to support a coherent know-how elicitation, to enrich design problem analysis, based on the re-use of experiences and their storage in a standardized knowledge base. A set of Tutor-Agents (TAs) aid experts in the knowledge modelling process focusing on Variables, Domains and Constraints as a key component of engineering knowledge. A shared and coherent knowledge base is the main purpose of the proposed Multi-Agent System (MAS). The interaction among agents enables to highlight potential incoherencies during the modelling process to avoid inconsistent information. The Multi-Agent approach is implemented in a software prototype and a knowledge base can then be constructed, providing standardized Product Life Cycle (PLC) constraints (based on the product related knowledge) for creating models to be analyzed by traditional inference engines such as Optimization solvers, Constraint Satisfaction programming, etc.
Adaptive Bilinear Pooling for Fine-grained Representation Learning

Fine-grained representation learning targets to generate discriminative description for fine-grained visual objects. Recently, the bilinear feature interaction has been proved effective in generating powerful high-order representation with spatially invariant information. However, the existing methods apply a fixed feature interaction strategy to all samples, which ignore the image and region heterogeneity in a dataset. To this end, we propose a generalized feature interaction method, named Adaptive Bilinear Pooling (ABP), which can adaptively infer a suitable pooling strategy for a given sample based on image content. Specifically, ABP consists of two learning strategies: p-order learning (P-net) and spatial attention learning (S-net). The p-order learning predicts an optimal exponential coefficient rather than a fixed order number to extract moderate visual information from an image. The spatial attention learning aims to infer a weighted score that measures the importance of each local region, which can compact the image representations. To make ABP compatible with kernelized bilinear feature interaction, a crossed two-branch structure is utilized to combine the P-net and S-net. This structure can facilitate complementary information exchange between two different visual branches. The experiments on three widely used benchmarks, including fine-grained object classification and action recognition, demonstrate the effectiveness of the proposed method.
Effective Horizon Detection on Complex Seas using Back Propagation Neural Network

Object detection is one of the main features of surface vehicles that do not require a driver (USV). Research on the detection of the horizon is one of the basic factors in conducting obstructions. Precisely detection helps to improve the efficiency and accuracy of object detection. Because it will eliminate most irrelevant areas. In this paper, there are four different algorithms that apply to three image sequences that have been taken in the open sea. This paper focuses on the accuracy rate and efficiency of horizon detection. By testing four different algorithms, including Hough transform, least squares, RANSAC and neural networks to separate the horizon from the image. Experimental results show that artificial neural network methods use more time but accuracy better than others.
Gesture Correctness Estimation with Deep Neural Networks and Rough Path Descriptors

One of the classical problems in computer vision is the automatic identification of different gestures or motions from a sequence of frames, captured via cameras or sensors, that usually contain angles or positions of many body joints. A large body of research addressing this problem exists in the literature. However, an interesting variant is not just to identify the gesture, but to decide whether a certain gesture is correctly or incorrectly executed. This can be in the context of physical exercises for rehabilitation in healthcare, robotics, sports, sign language and more. In this work we present our research on the automatic classification of gestures into correct or incorrect categories using the following three different but powerful techniques: convolutional neural networks (CNNs), long short-term memory (LSTM) neural networks and a technique based on rough path theory to extract unique sequences (i.e. path signatures) from gesture data and to use them as inputs to classical classifiers such as RandomForest (RF) and k-Nearest Neighbor (kNN). We collected the data we used as part of a collaborative project from Perkeso Tun Razak Rehabilitation Centre, Malacca, Malaysia. We recorded 17 patients performing nine types of gestures. For controls, we recorded 14 healthy individuals performing the same gestures. Our cross subject evaluation yields high accuracy for several subjects and gestures. CNNs and LSTMs achieve accuracy values up to 100% for several subjects in many gestures with the highest average accuracy for some gestures reaching 92%. The best overall classifier when using the rough paths representation was kNN as its average accuracy can reach 96.74% (100% accuracy for a subset of subjects in some gestures). For the three methods, the average accuracy across all gestures is 85.24%, 85.63% and 94.67% respectively. Overall, our research shows a great potential in identifying the right learning algorithm in motion correctness identification for real data.
An improved NLM filter with increased noise robustness and adaptive similarity function

The paper proposes an improved Non-Local Means (NLM) approach that employs a prefiltering step prior to the evaluation of the patch similarity function and uses a modified similarity function allowing a better and a finer control of the influence of the patches involved in the filtering process. Experimental results show that the improved method allows the obtention of better visual, and objective measures-quantified results than the original NLM formulation and recent developments in NLM filtering. The results are close to the ones obtained with state-of-the-art approaches like BM3D (Block Matching 3D).
Set-Membership Filtering for Nonlinear Dynamic Systems With Quadratic Inequality Constraints

The present study investigates the problem of set-membership filtering for nonlinear dynamic systems with general nonconvex inhomogeneous quadratic inequality constraints. The investigators propose an ellipsoidal state bounding estimation in the setting of unknown but bounded noise. In order to guarantee the on-line usage, the nonlinear function is linearized by Taylor expansion at each time step, where the bounding ellipsoid of the remainder is updated on-line based on the current state bounding ellipsoid. Furthermore, based on the remainder bounds and the constraints, both the state prediction and measurement update of the filtering can be transformed into a semidefinite programming problem that can be efficiently solved. In order to further reduce the computational complexity, a part-analytical formula of the shape matrix and the center of the bounding ellipsoid is derived using a decoupled technique, which is also helpful to clarify how these constraints affect the state estimation. Finally, typical numerical examples demonstrate the effectiveness of this filtering.
Weight-Mate

Weightlifting is easy to learn, but difficult to master. People who do weightlifting do it to improve their health, strengthen their muscles and build their physique. However, due to the complex and precise body positioning required, even experienced weightlifters require assistance in perfecting their technique. At the same time, the training requirements of the individual change over time, as they perfect and hone their craft. To help weightlifters achieve optimum personal performance, we designed Weight-Mate, a prototype wearable system for giving weightlifters of different skill levels personalized, precise and non-distracting immediate feedback on how to correct their current body positioning during deadlift training. By iterating Weight-Mate using cooperative usability testing (CUT) with weightlifters of different competencies with their coaches we designed a system that could adapt to individual physiology and training needs. The Weight-Mate sensor suit maps the lifter's body configuration against the ideal deadlift position throughout all stages of the life, as defined by their coach, and provides non-intrusive feedback to the lifter to correct their body position. Our formative evaluation with ten weightlifters shows that an adaptive approach to digital weight training offers great promise in assisting weight lifters of all levels to improve their technique, and hence improve the safety of the sport.
Extraction of Characteristic Subgraph Patterns with Support Threshold from Databases of Floor Plans

The rental price of apartments is determined by various factors such as building age, floor space, and so forth. Although room layouts, or connection structures among rooms, are considered to affect the price, it is not obvious that what kind of layout has a strong influence. Identification of characteristic layouts on the price has a large impact on property lenders to determine appropriate rental price as well as on property borrowers to judge the reasonableness. In this paper, a framework to extract characteristic patterns on room layouts is proposed based on advanced techniques in frequent subgraph mining. Floor plans are represented as graphs, and class labels are given based on the estimated residuals by certain basic attributes. For the set of labeled floor plans, we develop a new class of contrast pattern having support threshold, under new definitions of support value of a pattern in a floor plan. In addition, we extract decision lists and decision sets consisting of significant subgraph patterns with thresholds by using methods in interpretability research of machine learning. The effectiveness of the proposed framework is evaluated in preliminary experiments using a real world dataset.
Source Code Retrieval for Bug Localization using Bug Report

Bug localization helps software developers to track post-released faulty source files with the help of user's reported bug files. Information retrieval (IR) based bug localization have been widely used in recent years. It recommends relevant faulty source files to fix according to their highest similarity scores. We propose a combined approach of IR-based bug localization by operating previously fixed bug reports and source code structure. From the query, bug report structure is also considered to get more accurate faulty source files. In our approach, three parts in the source code file and two parts in bug report are combined as six combinations score. In some bug localization approach, features are usually linearly combined. Our approach uses linearly combine with the weight value. We perform experiments on three projects, i.e. SWT, AspectJ, and Eclipse. The result shows that the proposed approach achieves the relevance accuracy for bug localization process. According to the evaluation result, using the structure is more localized than no structured approach.
GrowingNet: An end-to-end growing network for semi-supervised learning

Semi-supervised learning (SSL) typically involves a small quantity of labeled data and a large quantity of unlabeled data. As such, the successful application of semi-supervised learning (SSL) depends on distinguishing easy and hard samples which contributes substantial recognition, as well as obtaining a more accurate pseudo target for a hard sample. However, existing SSL network models with deeper layers will suffer from overfitting or optimization difficulties. To address these problems, we propose a growing network (GrowingNet) where the convolution depth of the model can expand and contract. We also propose an incremental learning method by which the amount of pseudo labeled data can be increased uniformly. During training, the goal is to increase the convolutional layers of our model and the number of pseudo labeled data synchronously. We divide training epochs, the convolutional layers of GrowingNet, and pseudo labeled data into u equal parts. During each part of training epochs, we increase one part of convolutional layers, select one division of pseudo labeled data into the training process. The accuracy of the model will improve as training progresses, which distinguishes easy and hard samples and also provides more reliable pseudo labels during subsequent part of training epochs. This provides significant improvements over state-of-the-art networks in most of cases on SSL benchmark tasks (CIFAR-10, CIFAR-100, and SVHN). Specifically, without data augmentation, our model produces error rates of 20.86%, 18.22%, and 12.02% on CIFAR-10 with 1000, 2000, and 4000 labeled data, as well as error rates of 5.03% and 3.46% on SVHN with 500 and 1000 labeled data, respectively. With data augmentation, the error rate reaches 12.16% on CIFAR-10 with 2000 labeled data and 31.06% on CIFAR-100 with 10,000 labeled data.
Robotics at Work - News Headline Analysis 2016

User acceptance of new technologies is a key aspect to implement successful human-robot teams at work. The acceptance of new technologies, according to the Unified theory of Acceptance and Use Model, is determined by user's behavioral intention to use a technology and based on performance expectancy, effort expectancy, social influence and facilitating conditions. Furthermore, it is influenced by the experience a user has made with the new technology in the past. As not everyone can make their own experiences with robots in the workplace, many develop a first opinion by media consumption. To get an overview on media opinions on robots in working contexts, we extracted and coded the tone of headlines of three major English online newspapers. Subsequently, we compared the results of a human coding team with an open source approach for sentiment analysis. The comparison between human coders and the algorithm showed diverging results. Based on our analysis a team of human coders seems to be better in tone coding of short text passages like headlines. The overall results showed a negative overall tone in online newspaper headlines from 2016 to 2018. While 24% of the identified headlines had a positive tone, 43% contained a negative tone and 33% of the included headlines were coded as neutral. The negative tone in headlines on robots at work and their possible impact on workers expectations should be taken into account when introducing these new technologies in work contexts to ensure a human-centered implementation.
Enhancing Rock Painting Tour Experience with Outdoor Augmented Reality

The use of augmented reality technology is becoming a necessity of many cultural heritage sites to stay competitive and attractive to tourists. In this paper, we present an outdoor augmented reality guidance system for ancient rock painting aiming to enhance tourist experience and introduce the culture and historical background. We propose a two-stage approach that combines GPS and feature-based recognition methods. In the first stage, we build a navigation map in advance, which consists of the distribution information of major scenic spots, then visit these spots guided by GPS data on the standard smartphone. In the second stage, we adopt a feature-based recognition and tracking method to identify rock painting accurately and augment them with interactive contextual annotations like 3D textual labels, images, and videos. The reliability of this system is verified through on-site demonstrations - RockArt of the Helan Mountain, which is one of National Key Historical and Cultural Sites in China.
Visualization-Guided Attention Direction in Dynamic Control Tasks

In high-paced applications like air traffic control, disaster management, and medical triage, information is often routed through a central location such as a dispatch or control center. In this work, we introduce a set of color-coded and gaze-aware visualizations designed to simultaneously manage both information priority and user attention. These notifications are unique in that they not only respond to whether or not they have been observed by the user, but they use color to represent the priority of the object to assist processing of other incoming information in the environment. We built both heads-up display (HUD)-based and in-situ information management paradigms and combined these with gaze and color visualizations. In an interactive virtual reality (VR) control center simulation, HUD visualizations slightly reduced unnecessary head movement without sacrificing performance, but were distracting to participants.