Reviewerly/exHarmony huggingface dataset:
Deep Ensemble Capsule Network for Intelligent Compound Fault Diagnosis Using Multisensory Data

With the manufacturing industry stepping into the emerging new era of big data and intelligence, the amount of data collected from perception and monitoring systems with multiple smart sensors has increased tremendously. Such huge amount of multisensory data may not only power many aspects of fault diagnosis, but also bring great opportunities and challenges in modern manufacturing industry. In addition, with respect to intelligent fault diagnosis for machinery, few researches have been focused on the compound fault diagnosis under big-data circumstance. Therefore, a novel, intelligent, compound, fault decoupling method based on deep capsule network (CN) and ensemble learning is developed for compound fault decoupling and diagnosis using multisensory data. First, a decoupling CN (DCN) is constructed as the basic model. Second, taking the full advantage of multisensory data, the DCN model can be pretrained with multiple sensor data, which can obtain various pretrained DCN models. Finally, combining with ensemble learning skill, the pretrained DCN models are integrated by a combination strategy to obtain the deep ensemble CN (DECN) model for intelligent compound fault decoupling and diagnosis. The performance of the DECN model is validated on an automobile transmission (AT) data set with two compound faults, and the experimental results illustrate that the DECN model obtains higher diagnosis accuracy and decouples the compound fault correctly.
Comparative Study of Deep Learning-Based Sentiment Classification

The purpose of sentiment classification is to determine whether a particular document has a positive or negative nuance.Sentiment classification is extensively used in many business domains to improve products or services by understanding the opinions of customers regarding these products.Deep learning achieves state-of-the-art results in various challenging domains.With the success of deep learning, many studies have proposed deep-learning-based sentiment classification models and achieved better performances compared with conventional machine learning models.However, one practical issue occurring in deep-learning-based sentiment classification is that the best model structure depends on the characteristics of the dataset on which the deep learning model is trained; moreover, it is manually determined based on the domain knowledge of an expert or selected from a grid search of possible candidates.Herein, we present a comparative study of different deep-learning-based sentiment classification model structures to derive meaningful implications for building sentiment classification models.Specifically, eight deeplearning models, three based on convolutional neural networks and five based on recurrent neural networks, with two types of input structures, i.e., word level and character level, are compared for 13 review datasets, and the classification performances are discussed under different perspectives.
Semi-supervised robust deep neural networks for multi-label image classification

This paper introduces a robust method for semi-supervised training of deep neural networks for multi-label image classification. To this end, a ramp loss is utilized since it is more robust against noisy and incomplete image labels compared to the classic hinge loss. The proposed method allows for learning from both labeled and unlabeled data in a semi-supervised setting. This is achieved by propagating labels from the labeled images to their unlabeled neighbors in the feature space. Using a robust loss function becomes crucial here, as the initial label propagations may include many errors, which degrades the performance of non-robust loss functions. In contrast, the proposed robust ramp loss restricts extreme penalties from the samples with incorrect labels, and the label assignment improves in each iteration and contributes to the learning process. The proposed method achieves state-of-the-art results in semi-supervised learning experiments on the CIFAR-10 and STL-10 datasets, and comparable results to the state-of the-art in supervised learning experiments on the NUS-WIDE and MS-COCO datasets. Experimental results also verify that our proposed method is more robust against noisy image labels as expected.
From Oracles to Trustworthy Data On-Chaining Systems

Many blockchain transactions require blockchain-external data sources to provide data. Oracle systems have been proposed as a link between blockchains and blockchain-external resources. However, these Oracle systems vary greatly in assumptions and applicability and each system addresses the challenge of data on-chaining partly. We argue that Data On-chaining must be done in a trustworthy manner and, as a first contribution, define a set of key requirements for Trustworthy Data On-chaining. Further, we provide an in-depth assessment and comparison of state-of-the-art Oracle systems with regards to these requirements. This differentiation pinpoints the need for a uniform understanding of and directions for future research on Trustworthy Data On-chaining.
Interpretable Click-Through Rate Prediction through Hierarchical Attention

Click-through rate (CTR) prediction is a critical task in online advertising and marketing. For this problem, existing approaches, with shallow or deep architectures, have three major drawbacks. First, they typically lack persuasive rationales to explain the outcomes of the models. Unexplainable predictions and recommendations may be difficult to validate and thus unreliable and untrustworthy. In many applications, inappropriate suggestions may even bring severe consequences. Second, existing approaches have poor efficiency in analyzing high-order feature interactions. Third, the polysemy of feature interactions in different semantic subspaces is largely ignored. In this paper, we propose InterHAt that employs a Transformer with multi-head self-attention for feature learning. On top of that, hierarchical attention layers are utilized for predicting CTR while simultaneously providing interpretable insights of the prediction results. InterHAt captures high-order feature interactions by an efficient attentional aggregation strategy with low computational complexity. Extensive experiments on four public real datasets and one synthetic dataset demonstrate the effectiveness and efficiency of InterHAt.
Autonomous aerial cinematography in unstructured environments with learned artistic decision‐making

Abstract Aerial cinematography is revolutionizing industries that require live and dynamic camera viewpoints such as entertainment, sports, and security. However, safely piloting a drone while filming a moving target in the presence of obstacles is immensely taxing, often requiring multiple expert human operators. Hence, there is a demand for an autonomous cinematographer that can reason about both geometry and scene context in real‐time. Existing approaches do not address all aspects of this problem; they either require high‐precision motion‐capture systems or global positioning system tags to localize targets, rely on prior maps of the environment, plan for short time horizons, or only follow fixed artistic guidelines specified before the flight. In this study, we address the problem in its entirety and propose a complete system for real‐time aerial cinematography that for the first time combines: (a) vision‐based target estimation; (b) 3D signed‐distance mapping for occlusion estimation; (c) efficient trajectory optimization for long time‐horizon camera motion; and (d) learning‐based artistic shot selection. We extensively evaluate our system both in simulation and in field experiments by filming dynamic targets moving through unstructured environments. Our results indicate that our system can operate reliably in the real world without restrictive assumptions. We also provide in‐depth analysis and discussions for each module, with the hope that our design tradeoffs can generalize to other related applications. Videos of the complete system can be found at https://youtu.be/ookhHnqmlaU .
Saliency detection for strip steel surface defects using multiple constraints and improved texture features

Surface defect detection of strip steel is still a challenging task for its complex variances, e.g., intra-class defects exist large differences in appearance while inter-class defects contain similar parts. To address these issues, we regard the defect object as the salient part of the image and propose a novel, effective saliency propagation algorithm based on multiple constraints and improved texture features (MCITF). Firstly, we deliberately design 83-dim texture features that are used to generate label matrix (among which the label information viewed as the important basis of diffusion process) in the framework of multiple-instance learning. Then we resort to Laplacian regularization viewed as smoothness constraint for enlarging the gap between defect objects and background, and high-level prior (background, object, and mid-level feature) constraints for constraining the label information propagation process locally in order to uniformly highlight the complete defect objects while effectively suppress the non-salient background. Finally, we observe that the superpixel segmentation algorithm based on spectral clustering can adequately capture the edge information of defects, thus promoting to yield high-quality pixel-level saliency maps. Experimental results implemented on the real challenging strip steel benchmark database demonstrate that our MCITF model outperforms state-of-the-art methods with large margins and strong robustness in terms of eight evaluation metrics.
Explainable AI under contract and tort law: legal incentives and technical challenges

Abstract This paper shows that the law, in subtle ways, may set hitherto unrecognized incentives for the adoption of explainable machine learning applications. In doing so, we make two novel contributions. First, on the legal side, we show that to avoid liability, professional actors, such as doctors and managers, may soon be legally compelled to use explainable ML models. We argue that the importance of explainability reaches far beyond data protection law, and crucially influences questions of contractual and tort liability for the use of ML models. To this effect, we conduct two legal case studies, in medical and corporate merger applications of ML. As a second contribution, we discuss the (legally required) trade-off between accuracy and explainability and demonstrate the effect in a technical case study in the context of spam classification.
Tensor Oriented No-Reference Light Field Image Quality Assessment

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.
Transfer learning from adult to children for speech recognition: Evaluation, analysis and recommendations

Children speech recognition is challenging mainly due to the inherent high variability in children’s physical and articulatory characteristics and expressions. This variability manifests in both acoustic constructs and linguistic usage due to the rapidly changing developmental stage in children’s life. Part of the challenge is due to the lack of large amounts of available children speech data for efficient modeling. This work attempts to address the key challenges using transfer learning from adult’s models to children’s models in a Deep Neural Network (DNN) framework for children’s Automatic Speech Recognition (ASR) task evaluating on multiple children’s speech corpora with a large vocabulary. The paper presents a systematic and an extensive analysis of the proposed transfer learning technique considering the key factors affecting children’s speech recognition from prior literature. Evaluations are presented on (i) comparisons of earlier GMM-HMM and the newer DNN Models, (ii) effectiveness of standard adaptation techniques versus transfer learning, (iii) various adaptation configurations in tackling the variabilities present in children speech, in terms of (a) acoustic spectral variability, and (b) pronunciation variability and linguistic constraints. Our Analysis spans over (i) number of DNN model parameters (for adaptation), (ii) amount of adaptation data, (iii) ages of children, (iv) age dependent-independent adaptation. Finally, we provide Recommendations on (i) the favorable strategies over various aforementioned - analyzed parameters, and (ii) potential future research directions and relevant challenges/problems persisting in DNN based ASR for children’s speech.
From Senones to Chenones: Tied Context-Dependent Graphemes for Hybrid Speech Recognition

There is an implicit assumption that traditional hybrid approaches for automatic speech recognition (ASR) cannot directly model graphemes and need to rely on phonetic lexicons to get competitive performance, especially on English which has poor grapheme-phoneme correspondence. In this work, we show for the first time that, on English, hybrid ASR systems can in fact model graphemes effectively by leveraging tied context-dependent graphemes, i.e., chenones. Our chenone-based systems significantly outperform equivalent senone baselines by 4.5% to 11.1% relative on three different English datasets. Our results on Librispeech are state-of-the-art compared to other hybrid approaches and competitive with previously published end-to-end numbers. Further analysis shows that chenones can better utilize powerful acoustic models and large training data, and require context- and position-dependent modeling to work well. Chenone-based systems also outperform senone baselines on proper noun and rare word recognition, an area where the latter is traditionally thought to have an advantage. Our work provides an alternative for end-to-end ASR and establishes that hybrid systems can be improved by dropping the reliance on phonetic knowledge.
The Overlooked Elephant of Object Detection: Open Set

Even though object detection is a popular area of research that has found considerable applications in the real world, it has some fundamental aspects that have never been formally discussed and experimented. One of the core aspects of evaluating object detectors has been the ability to avoid false detections. While major datasets like PASCAL VOC or MSCOCO extensively test the detectors on their ability to avoid false positives, they do not differentiate between their closed-set and open-set performance. Despite systems being trained to reject everything other than the classes of interest, unknown objects from the open world end up being incorrectly detected as known objects, often with very high confidence. This paper is the first to formalize the problem of open-set object detection and propose the first open-set object detection protocol. Moreover, the paper provides a new evaluation metric to analyze the performance of some state-of-the-art detectors and discusses their performance differences.
Multi-Target Multi-Camera Tracking by Tracklet-to-Target Assignment

This paper focuses on the Multi-Target Multi-Camera Tracking task (MTMCT), which aims at tracking multiple targets within a multi-camera network. As the trajectory of each target is inherently split into multiple sub-trajectories (namely local tracklets) in a multi-camera network, a major challenge of MTMCT is how to accurately match the local tracklets generated within each camera across different cameras and generate a complete global trajectory for each target, i.e., the cross-camera tracklet matching problem. We solve the cross-camera tracklet matching problem by TRACklet-to-Target Assignment (TRACTA), and propose the Restricted Non-negative Matrix Factorization (RNMF) algorithm to compute the optimal assignment solution that meets a set of constraints, which should be in force in practice. TRACTA can correct the tracking errors caused by occlusions and missed detections in local tracklets, and produce a complete global trajectory for each target across all the cameras. Moreover, we also develop an analytical way of estimating the total number of targets in the camera network, which plays an important role to compute the tracklet-to-target assignment. Experimental evaluations and ablation studies on four MTMCT benchmark datasets show the superiority of the proposed TRACTA method.
Multi-block SSD based on small object detection for UAV railway scene surveillance

A method of multi-block Single Shot MultiBox Detector (SSD) based on small object detection is proposed to the railway scene of unmanned aerial vehicle surveillance. To address the limitation of small object detection, a multi-block SSD mechanism, which consists of three steps, is designed. First, the original input images are segmented into several overlapped patches. Second, each patch is separately fed into an SSD to detect the objects. Third, the patches are merged together through two stages. In the first stage, the truncated object of the sub-layer detection result is spliced. In the second stage, a sub-layer suppression and filtering algorithm applying the concept of non-maximum suppression is utilized to remove the overlapped boxes of sub-layers. The boxes that are not detected in the main-layer are retained. In addition, no sufficient labeled training samples of railway circumstance are available, thereby hindering the deployment of SSD. A two-stage training strategy leveraging to transfer learning is adopted to solve this issue. The deep learning model is preliminarily trained using labeled data of numerous auxiliaries, and then it is refined using only a few samples of railway scene. A railway spot in China, which is easily damaged by landslides, is investigated as a case study. Experimental results show that the proposed multi-block SSD method produces an overall accuracy of 96.6% and obtains an improvement of up to 9.2% compared with the traditional SSD.
Study of statistical methods for texture analysis and their modern evolutions

Abstract Texture analysis is widely performed in the current time as it is considered as an intimate property of the surface. It is widely used in the field of image processing, remote sensing applications, biomedical analysis, document processing, and so on. In this investigation, we present a detailed study of four different methodologies that have been developed for texture classification. These methodologies include gray level cooccurrence matrix (GLCM), local binary pattern (LBP), autocorrelation function (ACF), and histogram pattern. The detailed investigation on these methods suggests that GLCM is best for analyzing the surface texture, land‐use/landcover classification, and satellite data processing. LBP is widely used to analyze the facial features of an individual. The autocorrelation is used to identify the regularity of the textured surface. Finally, through histograms, one can visually identify the changes developed while analyzing the texture of the image data. Furthermore, we present a brief summary for newly developed texture classification techniques such as binary Gabor pattern, local spiking pattern, SRITCSD method, scale inversion, and deep perception models for texture analysis. Some benchmark texture datasets used in image processing are also discussed in this work.
Practical limitations of earthquake early warning

Earthquake early warning (EEW) entails detection of initial earthquake shaking and rapid estimation and notification to users prior to imminent, stronger shaking. EEW (ShakeAlert Phase 1, version 2.0) went operational in California in October 2019 and is coming to the rest of the U.S. West Coast. But what are the technical and social challenges to delivering actionable information on earthquake shaking before it arrives? Although there will be tangible benefits, there are also limitations. Basic seismological principles, alert communication challenges, and potential response actions, as well as substantial lessons learned from the use of EEW in Japan, point to more limited opportunities to warn and protect than perhaps many expect. This is in part because potential warning times vary by region and are influenced by tectonic environment, hypocentral depth, and the fault’s proximity to the alert user. For the U.S. West Coast, particularly for crustal earthquakes, warning times are shorter—and possible mitigation actions are likely to be less effective—than often maintained. Nevertheless, EEW is an additional arrow in the quiver of earthquake information tools available in the service of earthquake risk reduction. What is called for, then, is transparency and balance in the EEW discussion: along with its potential, the acknowledgment of EEW’s inherent and practical limitations is needed. Recognizing these limitations could, in fact, make EEW implementation more successful as part of a holistic earthquake mitigation strategy, where its role among other earthquake information tools is quite natural.
ROIMIX: Proposal-Fusion Among Multiple Images for Underwater Object Detection

Generic object detection algorithms have proven their excellent performance in recent years. However, object detection on underwater datasets is still less explored. In contrast to generic datasets, underwater images usually have color shift and low contrast; sediment would cause blurring in underwater images. In addition, underwater creatures often appear closely to each other on images due to their living habits. To address these issues, our work investigates augmentation policies to simulate overlapping, occluded and blurred objects, and we construct a model capable of achieving better generalization. We propose an augmentation method called RoIMix, which characterizes interactions among images. Proposals extracted from different images are mixed together. Previous data augmentation methods operate on a single image while we apply RoIMix to multiple images to create enhanced samples as training data. Experiments show that our proposed method improves the performance of region-based object detectors on both Pascal VOC and URPC datasets.