Reviewerly/exHarmony huggingface dataset:
Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey

Future buildings will offer new convenience, comfort, and efficiency possibilities to their residents. Changes will occur to the way people live as technology involves people's lives and information processing is fully integrated into their daily living activities and objects. The future expectation of smart buildings includes making the residents' experience as easy and comfortable as possible. The massive streaming data generated and captured by smart building appliances and devices contain valuable information that needs to be mined to facilitate timely actions and better decision making. Machine learning and big data analytics will undoubtedly play a critical role to enable the delivery of such smart services. In this paper, we survey the area of smart building with a special focus on the role of techniques from machine learning and big data analytics. This survey also reviews the current trends and challenges faced in the development of smart building services.
A Blockchain-Based Trusted Data Management Scheme in Edge Computing

With rapid development of computing technologies, large amount of data are gathered from edge terminals or Internet of Things (IoT) devices, however data trust and security in edge computing environment are very important issues to be considered, especially when the gathered data are fraud or dishonest, or the data are misused or spread without any authorization, which may lead to serious problems. In this article, a blockchain-based trusted data management scheme (called BlockTDM) in edge computing is proposed to solve the above problems, in which we proposed a flexible and configurable blockchain architecture that includes mutual authentication protocol, flexible consensus, smart contract, block and transaction data management, blockchain nodes management, and deployment. The BlockTDM scheme can support matrix-based multichannel data segment and isolation for sensitive or privacy data protection, and moreover, we have designed user-defined sensitive data encryption before the transaction payload stores in blockchain system, and have implemented conditional access and decryption query of the protected blockchain data and transactions through smart contract. Finally, we have evaluated the proposed BlockTDM scheme security, availability, and efficiency with large amount of experiments. Analysis and evaluations manifest that the proposed BlockTDM scheme provides a general, flexible, and configurable blockchain-based paradigm for trusted data management with tamper-resistance, which is suitable for edge computing with high-level security and creditability.
Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding

The application of most existing multi-view spectral clustering methods is generally limited by the following three deficiencies. First, the requirement to post-processing, such as K-means or spectral rotation. Second, the susceptibility to parameter selection. Third, the high computation cost. To this end, in this paper we develop a novel method that integrates nonnegative embedding and spectral embedding into a unified framework. Two promising advantages of proposed method include 1) the learned nonnegative embedding directly reveals the consistent clustering result, such that the uncertainty brought by post-processing can be avoided; 2) the involved model is parameter-free, which makes our method more applicable than existing algorithms that introduce many additional parameters. Furthermore, we develop an efficient inexact Majorization-Minimization method to solve the involved model which is non-convex and non-smooth. Experiments on multiple benchmark datasets demonstrate that our method achieves state-of-the-art performance.
Optimized Skeleton-based Action Recognition via Sparsified Graph Regression

With the prevalence of accessible depth sensors, dynamic human body skeletons have attracted much attention as a robust modality for action recognition. Previous methods model skeletons based on RNN or CNN, which has limited expressive power for irregular skeleton joints. While graph convolutional networks (GCN) have been proposed to address irregular graph-structured data, the fundamental graph construction remains challenging. In this paper, we represent skeletons naturally on graphs, and propose a graph regression based GCN (GR-GCN) for skeleton-based action recognition, aiming to capture the spatio-temporal variation in the data. As the graph representation is crucial to graph convolution, we first propose graph regression to statistically learn the underlying graph from multiple observations. In particular, we provide spatio-temporal modeling of skeletons and pose an optimization problem on the graph structure over consecutive frames, which enforces the sparsity of the underlying graph for efficient representation. The optimized graph not only connects each joint to its neighboring joints in the same frame strongly or weakly, but also links with relevant joints in the previous and subsequent frames. We then feed the optimized graph into the GCN along with the coordinates of the skeleton sequence for feature learning, where we deploy high-order and fast Chebyshev approximation of spectral graph convolution. Further, we provide analysis of the variation characterization by the Chebyshev approximation. Experimental results validate the effectiveness of the proposed graph regression and show that the proposed GR-GCN achieves the state-of-the-art performance on the widely used NTU RGB+D, UT-Kinect and SYSU 3D datasets.
Industrial blockchain based framework for product lifecycle management in industry 4.0

Product lifecycle management (PLM) aims to seamlessly manage all products and information and knowledge generated throughout the product lifecycle for achieving business competitiveness. Conventionally, PLM is implemented based on standalone and centralized systems provided by software vendors. The information of PLM is hardly to be integrated and shared among the cooperating parties. It is difficult to meet the requirements of the openness, interoperability and decentralization of the Industry 4.0 era. To address these challenges, this paper proposed an industrial blockchain-based PLM framework to facilitate the data exchange and service sharing in the product lifecycle. Firstly, we proposed the concept of industrial blockchain as the use of blockchain technology in the industry with the integration of IoT, M2M, and efficient consensus algorithms. It provided an open but secured information storage and exchange platform for the multiple stakeholders to achieve the openness, interoperability and decentralization in era of industry 4.0. Secondly, we proposed and developed customized blockchain information service to fulfill the connection between a single node with the blockchain network. As a middleware, it can not only process the multi-source and heterogeneous data from varied stages in the product lifecycle, but also broadcast the processed data to the blockchain network. Moreover, smart contract is used to automate the alert services in the product lifecycles. Finally, we illustrated the blockchain-based application between the cooperating partners in four emerging product lifecycle stages, including co-design and co-creation, quick and accurate tracking and tracing, proactive maintenance, and regulated recycling. A simulation experiment demonstrated the effectiveness and efficiency of the proposed framework. The results showed that the proposed framework is scalable and efficient, and hence it is feasible to be adopted in industry. With the successful development of the proposed platform, it is promising to provide an effective PLM for improving interoperability and cooperation between stakeholders in the entire product lifecycle.
P-MVSNet: Learning Patch-Wise Matching Confidence Aggregation for Multi-View Stereo

Learning-based methods are demonstrating their strong competitiveness in estimating depth for multi-view stereo reconstruction in recent years. Among them the approaches that generate cost volumes based on the plane-sweeping algorithm and then use them for feature matching have shown to be very prominent recently. The plane-sweep volumes are essentially anisotropic in depth and spatial directions, but they are often approximated by isotropic cost volumes in those methods, which could be detrimental. In this paper, we propose a new end-to-end deep learning network of P-MVSNet for multi-view stereo based on isotropic and anisotropic 3D convolutions. Our P-MVSNet consists of two core modules: a patch-wise aggregation module learns to aggregate the pixel-wise correspondence information of extracted features to generate a matching confidence volume, from which a hybrid 3D U-Net then infers a depth probability distribution and predicts the depth maps. We perform extensive experiments on the DTU and Tanks & Temples benchmark datasets, and the results show that the proposed P-MVSNet achieves the state-of-the-art performance over many existing methods on multi-view stereo.
Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation

Most progress in semantic segmentation reports on daytime images taken under favorable illumination conditions. We instead address the problem of semantic segmentation of nighttime images and improve the state-of-the-art, by adapting daytime models to nighttime without using nighttime annotations. Moreover, we design a new evaluation framework to address the substantial uncertainty of semantics in nighttime images. Our central contributions are: 1) a curriculum framework to gradually adapt semantic segmentation models from day to night via labeled synthetic images and unlabeled real images, both for progressively darker times of day, which exploits cross-time-of-day correspondences for the real images to guide the inference of their labels; 2) a novel uncertainty-aware annotation and evaluation framework and metric for semantic segmentation, designed for adverse conditions and including image regions beyond human recognition capability in the evaluation in a principled fashion; 3) the Dark Zurich dataset, which comprises 2416 unlabeled nighttime and 2920 unlabeled twilight images with correspondences to their daytime counterparts plus a set of 151 nighttime images with fine pixel-level annotations created with our protocol, which serves as a first benchmark to perform our novel evaluation. Experiments show that our guided curriculum adaptation significantly outperforms state-of-the-art methods on real nighttime sets both for standard metrics and our uncertainty-aware metric. Furthermore, our uncertainty-aware evaluation reveals that selective invalidation of predictions can lead to better results on data with ambiguous content such as our nighttime benchmark and profit safety-oriented applications which involve invalid inputs.
Determine the Number of Unknown Targets in Open World Based on Elbow Method

Generalized evidence theory as an extension of Dempster-Shafer evidence theory can deal with uncertain information fusion in the open world. However, one of the open issues is to detect the number of unknown targets. In this article, a new method based on the elbow method is proposed to solve this problem. After the identification of the open world, K-means clustering is used to cluster categories. Then, the elbow method is used to find a correct number of unknown targets. The frame of discernment is reupdated. To test effectiveness of the proposed method, several experiments are conducted. The results illustrate the advantage of the proposed method in the open world.
Symmetry-Constrained Rectification Network for Scene Text Recognition

Reading text in the wild is a very challenging task due to the diversity of text instances and the complexity of natural scenes. Recently, the community has paid increasing attention to the problem of recognizing text instances with irregular shapes. One intuitive and effective way to handle this problem is to rectify irregular text to a canonical form before recognition. However, these methods might struggle when dealing with highly curved or distorted text instances. To tackle this issue, we propose in this paper a Symmetry-constrained Rectification Network (ScRN) based on local attributes of text instances, such as center line, scale and orientation. Such constraints with an accurate description of text shape enable ScRN to generate better rectification results than existing methods and thus lead to higher recognition accuracy. Our method achieves state-of-the-art performance on text with both regular and irregular shapes. Specifically, the system outperforms existing algorithms by a large margin on datasets that contain quite a proportion of irregular text instances, e.g., ICDAR 2015, SVT-Perspective and CUTE80.
A Comparison between ARIMA, LSTM, and GRU for Time Series Forecasting

A critical area of machine learning is Time Series forecasting, as various forecasting problems contain a time component. A series of observations taken chronologically in time is known as a Time Series. In this research, however, we aim to compare three different machine learning models in making a time series forecast. We are going to use the Bitcoin's price dataset as our time series data set and make predictions accordingly. The results show that the ARIMA model gave better results than the deep learning-based regression models. ARIMA gives the best results at 2.76% and 302.53 for MAPE and RMSE respectively. The Gated Recurrent Unit (GRU) however performed better than the Long Short-term Memory (LSTM), with 3.97% and 381.34 of MAPE and RMSE respectively.
Study of Subjective and Objective Quality Assessment of Audio-Visual Signals

The topics of visual and audio quality assessment (QA) have been widely researched for decades, yet nearly all of this prior work has focused only on single-mode visual or audio signals. However, visual signals rarely are presented without accompanying audio, including heavy-bandwidth video streaming applications. Moreover, the distortions that may separately (or conjointly) afflict the visual and audio signals collectively shape user-perceived quality of experience (QoE). This motivated us to conduct a subjective study of audio and video (A/V) quality, which we then used to compare and develop A/V quality measurement models and algorithms. The new LIVE-SJTU Audio and Video Quality Assessment (A/V-QA) Database includes 336 A/V sequences that were generated from 14 original source contents by applying 24 different A/V distortion combinations on them. We then conducted a subjective A/V quality perception study on the database towards attaining a better understanding of how humans perceive the overall combined quality of A/V signals. We also designed four different families of objective A/V quality prediction models, using a multimodal fusion strategy. The different types of A/V quality models differ in both the unimodal audio and video quality prediction models comprising the direct signal measurements and in the way that the two perceptual signal modes are combined. The objective models are built using both existing state-of-the-art audio and video quality prediction models and some new prediction models, as well as quality-predictive features delivered by a deep neural network. The methods of fusing audio and video quality predictions that are considered include simple product combinations as well as learned mappings. Using the new subjective A/V database as a tool, we validated and tested all of the objective A/V quality prediction models. We will make the database publicly available to facilitate further research.
Single Image Deraining: From Model-Based to Data-Driven and Beyond

The goal of single-image deraining is to restore the rain-free background scenes of an image degraded by rain streaks and rain accumulation. The early single-image deraining methods employ a cost function, where various priors are developed to represent the properties of rain and background layers. Since 2017, single-image deraining methods step into a deep-learning era, and exploit various types of networks, i.e., convolutional neural networks, recurrent neural networks, generative adversarial networks, etc., demonstrating impressive performance. Given the current rapid development, in this paper, we provide a comprehensive survey of deraining methods over the last decade. We summarize the rain appearance models, and discuss two categories of deraining approaches: model-based and data-driven approaches. For the former, we organize the literature based on their basic models and priors. For the latter, we discuss the developed ideas related to architectures, constraints, loss functions, and training datasets. We present milestones of single-image deraining methods, review a broad selection of previous works in different categories, and provide insights on the historical development route from the model-based to data-driven methods. We also summarize performance comparisons quantitatively and qualitatively. Beyond discussing the technicality of deraining methods, we also discuss the future possible directions.
A hybrid sampling algorithm combining M-SMOTE and ENN based on Random forest for medical imbalanced data

The problem of imbalanced data classification often exists in medical diagnosis. Traditional classification algorithms usually assume that the number of samples in each class is similar and their misclassification cost during training is equal. However, the misclassification cost of patient samples is higher than that of healthy person samples. Therefore, how to increase the identification of patients without affecting the classification of healthy individuals is an urgent problem. In order to solve the problem of imbalanced data classification in medical diagnosis, we propose a hybrid sampling algorithm called RFMSE, which combines the Misclassification-oriented Synthetic minority over-sampling technique (M-SMOTE) and Edited nearset neighbor (ENN) based on Random forest (RF). The algorithm is mainly composed of three parts. First, M-SMOTE is used to increase the number of samples in the minority class, while the over-sampling rate of M-SMOTE is the misclassification rate of RF. Then, ENN is used to remove the noise ones from the majority samples. Finally, RF is used to perform classification prediction for the samples after hybrid sampling, and the stopping criterion for iterations is determined according to the changes of the classification index (i.e. Matthews Correlation Coefficient (MCC)). When the value of MCC continuously drops, the process of iterations will be stopped. Extensive experiments conducted on ten UCI datasets demonstrate that RFMSE can effectively solve the problem of imbalanced data classification. Compared with traditional algorithms, our method can improve F-value and MCC more effectively.
Artificial intelligence within the interplay between natural and artificial computation: Advances in data science, trends and applications

Artificial intelligence and all its supporting tools, e.g. machine and deep learning in computational intelligence-based systems, are rebuilding our society (economy, education, life-style, etc.) and promising a new era for the social welfare state. In this paper we summarize recent advances in data science and artificial intelligence within the interplay between natural and artificial computation. A review of recent works published in the latter field and the state the art are summarized in a comprehensive and self-contained way to provide a baseline framework for the international community in artificial intelligence. Moreover, this paper aims to provide a complete analysis and some relevant discussions of the current trends and insights within several theoretical and application fields covered in the essay, from theoretical models in artificial intelligence and machine learning to the most prospective applications in robotics, neuroscience, brain computer interfaces, medicine and society, in general.
Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark. The dataset and additional resources are available at https://arielai.com/mesh_hands.
Self-Trained Deep Ordinal Regression for End-to-End Video Anomaly Detection

Video anomaly detection is of critical practical importance to a variety of real applications because it allows human attention to be focused on events that are likely to be of interest, in spite of an otherwise overwhelming volume of video. We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods, namely, 1) being highly dependent on manually labeled normal training data; and 2) sub-optimal feature learning. By formulating a surrogate two-class ordinal regression task we devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data. Experiments on eight real-world video scenes show that our proposed method outperforms state-of-the-art methods that require no labeled training data by a substantial margin, and enables easy and accurate localization of the identified anomalies. Furthermore, we demonstrate that our method offers effective human-in-the-loop anomaly detection which can be critical in applications where anomalies are rare and the false-negative cost is high.
An improved Dragonfly Algorithm for feature selection

Dragonfly Algorithm (DA) is a recent swarm-based optimization method that imitates the hunting and migration mechanisms of idealized dragonflies. Recently, a binary DA (BDA) has been proposed. During the algorithm iterative process, the BDA updates its five main coefficients using random values. This updating mechanism can be improved to utilize the survival-of-the-fittest principle by adopting different functions such as linear, quadratic, and sinusoidal. In this paper, a novel BDA is proposed. The algorithm uses different strategies to update the values of its five main coefficients to tackle Feature Selection (FS) problems. Three versions of BDA have been proposed and compared against the original DA. The proposed algorithms are Linear-BDA, Quadratic-BDA, and Sinusoidal-BDA. The algorithms are evaluated using 18 well-known datasets. Thereafter, they are compared in terms of classification accuracy, the number of selected features, and fitness value. The results show that Sinusoidal-BDA outperforms other proposed methods in almost all datasets. Furthermore, Sinusoidal-BDA exceeds three swarm-based methods in all the datasets in terms of classification accuracy and it excels in most datasets when compared in terms of the fitness function value. In a nutshell, the proposed Sinusoidal-BDA outperforms the comparable feature selection algorithms and the proposed updating mechanism has a high impact on the algorithm performance when tackling FS problems.