Reviewerly/exHarmony huggingface dataset:
Deep Association

Multiple Object Tracking (MOT) has a wide range of applications in surveillance retrieval and autonomous driving. The majority of existing methods focus on extracting features by deep learning and hand-crafted optimizing bipartite graph or network flow. In this paper, we proposed an efficient end-to-end model, Deep Association Network (DAN), to learn the graph-based training data, which are constructed by spatial-temporal interaction of objects. DAN combines Convolutional Neural Network (CNN), Motion Encoder (ME) and Graph Neural Network (GNN). The CNNs and Motion Encoders extract appearance features from bounding box images and motion features from positions respectively, and then the GNN optimizes graph structure to associate the same object among frames together. In addition, we presented a novel end-to-end training strategy for Deep Association Network. Our experimental results demonstrate the effectiveness of DAN up to the state-of-the-art methods without extra-dataset on MOT16 and DukeMTMCT.
Automatic Classification of Microseismic Signals Based on MFCC and GMM-HMM in Underground Mines

In order to mitigate economic and safety risks during mine life, a microseismic monitoring system is installed in a number of underground mines. The basic step for successfully analyzing those microseismic data is the correct detection of various event types, especially the rock mass rupture events. The visual scanning process is a time-consuming task and requires experience. Therefore, here we present a new method for automatic classification of microseismic signals based on the Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) by using only Mel-frequency cepstral coefficient (MFCC) features extracted from the waveform. The detailed implementation of our proposed method is described. The performance of this method is tested by its application to microseismic events selected from the Dongguashan Copper Mine (China). A dataset that contains a representative set of different microseismic events including rock mass rupture, blasting vibration, mechanical drilling, and electromagnetic noise is collected for training and testing. The results show that our proposed method obtains an accuracy of 92.46%, which demonstrates the effectiveness of the method for automatic classification of microseismic data in underground mines.
2D Drawing Visualization Framework for Applying Projection-Based Augmented Reality in a Panelized Construction Manufacturing Facility: Proof of Concept

Product quality is recognized as a major benefit in industrialized construction because of the utilization of machines in manufacturing facilities. However, manual work continues to exist in complex assembly projects, which can potentially cause quality issues. To address such quality issues in industrialized manufacturing processes, projection-based augmented reality (AR) techniques have been applied in other industries (e.g., manufacturing), but on a smaller scale and at a short distance. To apply projection-based AR in the construction manufacturing facility, this paper proposes a framework that enables the user to perform a vision-based projection alignment using a projector and camera. The designated projection area determined by the user is marked out, the coordinates of which are then computed through segmentation and object-detection algorithms. After the acquisition of the markers’ coordinates is complete, a two-dimensional (2D) image is overlaid on the surface of the designated area using a projector by computing transformation matrix for the projection. To evaluate the potential performance in a field setting, the offset distances between the four corners of the projection boundary and the center of the markers are measured in various environments, such as at distances ranging from 5 to 8 m, and with different illumination conditions (i.e., low and high brightness) as well as in a manufacturing shop. The results indicate that average offset distances in all the experimental conditions are shorter than the factory tolerance level of 6.35 mm. Also, the statistical analysis reveals that both the distance and the illumination are significant factors affecting the projection alignment performance. This study provides a novel approach for visualizing vital information within a user’s field of view during the manufacturing processes at shops and offers considerations for implementing projection-based AR in practice.
Video processing with serverless computing

The growing demand for video processing and the advantages in scalability and cost reduction brought by the emerging serverless computing have attracted significant attention in serverless computing powered video processing. However, how to implement and configure serverless functions to optimize the performance and cost of video processing applications remains unclear. In this paper, we explore the configuration and implementation schemes of typical video processing functions deployed to the serverless platforms and quantify their influence on the execution duration and monetary cost from a developer's perspective. Our measurement reveals that memory configuration is non-trivial. Dynamic profiling of workloads is necessary to find the best memory configuration. Moreover, compared with calling external video processing APIs, implementing these services locally in serverless functions can be competitive. We also find that the performance of video processing applications could be affected by the underlying infrastructure. Our work provides guidelines for further function-level optimization and complements the existing measurement studies for both serverless computing and video processing.
Could I Have a Stack Trace to Examine the Dependency Conflict Issue?

Intensive use of libraries in Java projects brings potential risk of dependency conflicts, which occur when a project directly or indirectly depends on multiple versions of the same library or class. When this happens, JVM loads one version and shadows the others. Runtime exceptions can occur when methods in the shadowed versions are referenced. Although project management tools such as Maven are able to give warnings of potential dependency conflicts when a project is built, developers often ask for crashing stack traces before examining these warnings. It motivates us to develop Riddle, an automated approach that generates tests and collects crashing stack traces for projects subject to risk of dependency conflicts. Riddle, built on top of Asm and Evosuite, combines condition mutation, search strategies and condition restoration. We applied Riddle on 19 real-world Java projects with duplicate libraries or classes. We reported 20 identified dependency conflicts including their induced crashing stack traces and the details of generated tests. Among them, 15 conflicts were confirmed by developers as real issues, and 10 were readily fixed. The evaluation results demonstrate the effectiveness and usefulness of Riddle.
Testing Linear Regressions by StatsModel Library of Python for Oceanological Data Interpretation

The study area is focused on the Mariana Trench, west Pacific Ocean.The research aim is to investigate correlation between various factors, such as bathymetric depths, geomorphic shape, geographic location on four tectonic plates of the sampling points along the trench, and their influence on the geologic sediment thickness.Technically, the advantages of applying Python programming language for oceanographic data sets were tested.The methodological approaches include GIS data collecting, data analysis, statistical modelling, plotting and visualizing.Statistical methods include several algorithms that were tested: 1) weighted least square linear regression between geological variables, 2) autocorrelation; 3) design matrix, 4) ordinary least square regression, 5) quantile regression.The spatial and statistical analysis of the correlation of these factors aimed at the understanding, which geological and geodetic factors affect the distribution of the steepness and shape of the trench.Following factors were analysed: geology (sediment thickness), geographic location of the trench on four tectonics plates: Philippines, Pacific, Mariana and Caroline and bathymetry along the profiles: maximal and mean, minimal values, as well as the statistical calculations of the 1st and 3rd quantiles.The study revealed correlations between the sediment thickness and distinct variations of the trench geomorphology and sampling locations across various segments along the crescent of the trench.
Fine-grain watermarking for intellectual property protection

The current online digital world, consisting of thousands of newspapers, blogs, social media, and cloud file sharing services, is providing easy and unlimited access to a large treasure of text contents. Making copies of these text contents is simple and virtually costless. As a result, producers and owners of text content are interested in the protection of their intellectual property (IP) rights. Digital watermarking has become crucially important in the protection of digital contents. Out of all, text watermarking poses many challenges, since text is characterized by a low capacity to embed a watermark and allows only a restricted number of alternative syntactic and semantic permutations. This becomes even harder when authors want to protect not just a whole book or article, but each single sentence or paragraph, a problem well known to copyright law. In this paper, we present a fine-grain text watermarking method that protects even small portions of the digital content. The core method is based on homoglyph characters substitution for latin symbols and whitespaces. It allows to produce a watermarked version of the original text, preserving the anonymity of the users according to the right to privacy. In particular, the embedding and extraction algorithms allow to continuously protect the watermark through the whole document in a fine-grain fashion. It ensures visual indistinguishability and length preservation, meaning that it does not cause overhead to the original document, and it is robust to the copy and past of small excerpts of the text. We use a real dataset of 1.8 million New York articles to evaluate our method. We evaluate and compare the robustness against common attacks, and we propose a new measure for partial copy and paste robustness. The results show the effectiveness of our approach providing an average length of 101 characters needed to embed the watermark and allowing to protect paragraph-long excerpt or smaller the 94.5% of the times.
OpenSolar: Promoting the openness and accessibility of diverse public solar datasets

Observational solar data is the foundation of data-driven research in solar power grid integration and power system operations. Compared to other fields in data science, the openness and accessibility of solar data is lacking, which prevents solar data science from catching up with the emerging trends of data science (e.g., deep learning). In this paper, OpenSolar, a package with both R and Python versions, is developed to enhance the openness and accessibility of publicly available solar datasets. The OpenSolar package provides access to multiple types of solar data, primarily from four datasets: (1) the National Renewable Energy Laboratory (NREL) Solar Power Data for Integration Studies dataset, (2) the NREL Solar Radiation Research Laboratory dataset, (3) the Sheffield Solar-Microgen database, and (4) the Dataport database. Unlike other open solar datasets that only contain meteorological data, the four datasets in the OpenSolar package also contain behind-the-meter power data, sky images, and solar power data with satisfactory temporal and spatial resolution and coverage. The overview, quality-control methods, and potential usage of the datasets, in conjunction with sample code implementing the OpenSolar functions, are described in this paper. The package is expected to assist in bridging the gaps between the research fields of solar energy, power systems, and data science.
The Bitcoin mining breakdown: Is mining still profitable?

We provide an updated estimation of the energy consumption of the Bitcoin network, and a calculation of the evolution of the production cost of Bitcoin over time. Using these data, we conclude that since June 2018 Bitcoin mining is no longer profitable for commodity miners without access to electricity prices below 0.14 $/kWh. This phenomenon explains why many Western miners have dropped out of the circuit, further increasing the centralization of mining activity in China. In addition, we estimate that the marginal cost of the production of bitcoin is around 1,952 US dollars. Below this price the cost of mining would not be profitable, even with the most efficient equipment and the lowest possible price for the energy required. This could lead to a massive exit of the biggest mining players, with unpredictable consequences for the future of this cryptocurrency.
Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay

Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding space. We learn an abstract generative distribution in the embedding that allows generation of data points to represent past experience. We sample from this distribution and utilize experience replay to avoid forgetting and simultaneously accumulate new knowledge to the abstract distribution in order to couple the current task with past experience. We demonstrate theoretically and empirically that our framework learns a distribution in the embedding, which is shared across all tasks, and as a result tackles catastrophic forgetting.
A Mask Based Deep Ranking Neural Network for Person Retrieval

Person retrieval faces many challenges including cluttered background, appearance variations (e.g., illumination, pose, occlusion) among different camera views and the similarity among different person's images. To address these issues, we put forward a novel mask based deep ranking neural network with a skipped fusing layer. Firstly, to alleviate the problem of cluttered background, masked images with only the foreground regions are incorporated as input in the proposed neural network. Secondly, to reduce the impact of the appearance variations, the multi-layer fusion scheme is developed to obtain more discriminative fine-grained information. Lastly, considering person retrieval is a special image retrieval task, we propose a novel ranking loss to optimize the whole network. The proposed ranking loss can further mitigate the interference problem of similar negative samples when producing ranking results. The extensive experiments validate the superiority of the proposed method compared with the state-of-the-art methods on many benchmark datasets.
Attribute-Aware Convolutional Neural Networks for Facial Beauty Prediction

Facial beauty prediction (FBP) aims to develop a machine that automatically makes facial attractiveness assessment. To a large extent, the perception of facial beauty for a human is involved with the attributes of facial appearance, which provides some significant visual cues for FBP. Deep convolution neural networks (CNNs) have shown its power for FBP, but convolution filters with fixed parameters cannot take full advantage of the facial attributes for FBP. To address this problem, we propose an Attribute-aware Convolutional Neural Network (AaNet) that modulates the filters of the main network, adaptively, using parameter generators that take beauty-related attributes as extra inputs. The parameter generators update the filters in the main network in two different manners: filter tuning or filter rebirth. However, AaNet takes attributes information as prior knowledge, that is ill-suited to those datasets merely with task-oriented labels. Therefore, imitating the design of AaNet, we further propose a Pseudo Attribute-aware Convolutional Neural Network (P-AaNet) that modulates filters conditioned on global context embeddings (pseudo attributes) of input faces learnt by a lightweight pseudo attribute distiller. Extensive ablation studies show that the AaNet and P-AaNet improve the performance of FBP when compared to conventional convolution and attention scheme, which validates the effectiveness of our method.
DeltaDou: Expert-level Doudizhu AI through Self-play

Artificial Intelligence has seen several breakthroughs in two-player perfect information game. Nevertheless, Doudizhu, a three-player imperfect information game, is still quite challenging. In this paper, we present a Doudizhu AI by applying deep reinforcement learning from games of self-play. The algorithm combines an asymmetric MCTS on nodes of information set of each player, a policy-value network that approximates the policy and value on each decision node, and inference on unobserved hands of other players by given policy. Our results show that self-play can significantly improve the performance of our agent in this multi-agent imperfect information game. Even starting with a weak AI, our agent can achieve human expert level after days of self-play and training.
Why Can’t You Do That HAL? Explaining Unsolvability of Planning Tasks

Explainable planning is widely accepted as a prerequisite for autonomous agents to successfully work with humans. While there has been a lot of research on generating explanations of solutions to planning problems, explaining the absence of solutions remains an open and under-studied problem, even though such situations can be the hardest to understand or debug. In this paper, we show that hierarchical abstractions can be used to efficiently generate reasons for unsolvability of planning problems. In contrast to related work on computing certificates of unsolvability, we show that these methods can generate compact, human-understandable reasons for unsolvability. Empirical analysis and user studies show the validity of our methods as well as their computational efficacy on a number of benchmark planning domains.
EmoCo: Visual Analysis of Emotion Coherence in Presentation Videos

Emotions play a key role in human communication and public presentations. Human emotions are usually expressed through multiple modalities. Therefore, exploring multimodal emotions and their coherence is of great value for understanding emotional expressions in presentations and improving presentation skills. However, manually watching and studying presentation videos is often tedious and time-consuming. There is a lack of tool support to help conduct an efficient and in-depth multi-level analysis. Thus, in this paper, we introduce EmoCo, an interactive visual analytics system to facilitate efficient analysis of emotion coherence across facial, text, and audio modalities in presentation videos. Our visualization system features a channel coherence view and a sentence clustering view that together enable users to obtain a quick overview of emotion coherence and its temporal evolution. In addition, a detail view and word view enable detailed exploration and comparison from the sentence level and word level, respectively. We thoroughly evaluate the proposed system and visualization techniques through two usage scenarios based on TED Talk videos and interviews with two domain experts. The results demonstrate the effectiveness of our system in gaining insights into emotion coherence in presentations.
Equally-Guided Discriminative Hashing for Cross-modal Retrieval

Cross-modal hashing intends to project data from two modalities into a common hamming space to perform cross-modal retrieval efficiently. Despite satisfactory performance achieved on real applications, existing methods are incapable of effectively preserving semantic structure to maintain inter-class relationship and improving discriminability to make intra-class samples aggregated simultaneously, which thus limits the higher retrieval performance. To handle this problem, we propose Equally-Guided Discriminative Hashing (EGDH), which jointly takes into consideration semantic structure and discriminability. Specifically, we discover the connection between semantic structure preserving and discriminative methods. Based on it, we directly encode multi-label annotations that act as high-level semantic features to build a common semantic structure preserving classifier. With the common classifier to guide the learning of different modal hash functions equally, hash codes of samples are intra-class aggregated and inter-class relationship preserving. Experimental results on two benchmark datasets demonstrate the superiority of EGDH compared with the state-of-the-arts.
Multi-View Multiple Clustering

Multiple clustering aims at exploring alternative clusterings to organize the data into meaningful groups from different perspectives. Existing multiple clustering algorithms are designed for single-view data. We assume that the individuality and commonality of multi-view data can be leveraged to generate high-quality and diverse clusterings. To this end, we propose a novel multi-view multiple clustering (MVMC) algorithm. MVMC first adapts multi-view self-representation learning to explore the individuality encoding matrices and the shared commonality matrix of multi-view data. It additionally reduces the redundancy (i.e., enhancing the individuality) among the matrices using the Hilbert-Schmidt Independence Criterion (HSIC), and collects shared information by forcing the shared matrix to be smooth across all views. It then uses matrix factorization on the individual matrices, along with the shared matrix, to generate diverse clusterings of high-quality. We further extend multiple co-clustering on multi-view data and propose a solution called multi-view multiple co-clustering (MVMCC). Our empirical study shows that MVMC (MVMCC) can exploit multi-view data to generate multiple high-quality and diverse clusterings (co-clusterings), with superior performance to the state-of-the-art methods.
Risk Contours Map for Risk Bounded Motion Planning under Perception Uncertainties

In this paper, we introduce "risk contours map" that contains the risk information of different regions in uncertain environments.Risk is defined as the probability of collision of robots with obstacles in presence of probabilistic uncertainties in location, size, and geometry of obstacles.We use risk contours to obtain safe paths for robots with guaranteed bounded risk.We formulate the problem of obtaining risk contours as a chance constrained optimization.We leverage the theory of moments and nonnegative polynomials to provide a convex optimization in the form of sum of squares optimization.Provided approach deals with nonconvex obstacles and probabilistic bounded and unbounded uncertainties.We demonstrate the performance of the provided approach by solving risk bounded motion planning problems.