Reviewerly/exHarmony huggingface dataset:
Extreme gradient boosting and deep neural network based ensemble learning approach to forecast hourly solar irradiance

Prediction of solar irradiance is an essential requirement for reliable planning and efficient designing of solar energy systems. Thus, in present work, a new ensemble model, which consists of two advance base models, namely extreme gradient boosting forest and deep neural networks (XGBF-DNN), is proposed for hourly global horizontal irradiance forecast. These base models are integrated using ridge regression to avoid the over-fitting problem. Moreover, the proposed framework is designed carefully to ensure the diversity in base models, as diversity among base models is widely recognized as a key to the success of an ensemble model. Further, to enhance the model’s accuracy, a subset of input features is selected, which includes temperature, clear-sky index, relative humidity, and hour of the day as the most relevant features. To provide a comprehensive and reliable assessment, proposed model is validated with data from three different climatic Indian locations. Subsequently, a seasonal analysis is also carried out to provide a deeper insight into the model’s performance. The performance of the proposed model is evaluated by comparing the prediction results with different models, including benchmark smart persistence and traditional machine learning techniques, such as random forest, support vector regression, extreme gradient boosting forest and deep neural networks. The proposed ensemble model exhibits the best combination of stability and prediction accuracy irrespective of seasonal variations in weather conditions and shows a forecast skill score in a range of about 33%–40% in prediction error. The predictive performance and stability make XGBF-DNN an ideal and reliable model for hourly global horizontal irradiance prediction. Therefore, the developed model can be advised to serve as a potential prediction model in several domains, such as forecasting of solar power, wind power, electricity consumption, etc.
Controllable Multi-Interest Framework for Recommendation

Recently, neural networks have been widely used in e-commerce recommender systems, owing to the rapid development of deep learning. We formalize the recommender system as a sequential recommendation problem, intending to predict the next items that the user might be interacted with. Recent works usually give an overall embedding from a user's behavior sequence. However, a unified user embedding cannot reflect the user's multiple interests during a period. In this paper, we propose a novel controllable multi-interest framework for the sequential recommendation, called ComiRec. Our multi-interest module captures multiple interests from user behavior sequences, which can be exploited for retrieving candidate items from the large-scale item pool. These items are then fed into an aggregation module to obtain the overall recommendation. The aggregation module leverages a controllable factor to balance the recommendation accuracy and diversity. We conduct experiments for the sequential recommendation on two real-world datasets, Amazon and Taobao. Experimental results demonstrate that our framework achieves significant improvements over state-of-the-art models. Our framework has also been successfully deployed on the offline Alibaba distributed cloud platform.
Privacy for Free: Wireless Federated Learning via Uncoded Transmission With Adaptive Power Control

Federated Learning (FL) refers to distributed protocols that avoid direct raw data exchange among the participating devices while training for a common learning task. This way, FL can potentially reduce the information on the local data sets that is leaked via communications. In order to provide formal privacy guarantees, however, it is generally necessary to put in place additional masking mechanisms. When FL is implemented in wireless systems via uncoded transmission, the channel noise can directly act as a privacy-inducing mechanism. This paper demonstrates that, as long as the privacy constraint level, measured via differential privacy (DP), is below a threshold that decreases with the signal-to-noise ratio (SNR), uncoded transmission achieves privacy “for free”, i.e., without affecting the learning performance. More generally, this work studies adaptive power allocation (PA) for distributed gradient descent in wireless FL with the aim of minimizing the learning optimality gap under privacy and power constraints. Both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) transmission with “over-the-air-computing” are studied, and solutions are obtained in closed form for an offline optimization setting. Furthermore, heuristic online methods are proposed that leverage iterative one-step-ahead optimization. The importance of dynamic PA and the potential benefits of NOMA versus OMA are demonstrated through extensive simulations.
Design and implementation of construction cost prediction model based on SVM and LSSVM in industries 4.0

Purpose In order to improve the accuracy of project cost prediction, considering the limitations of existing models, the construction cost prediction model based on SVM (Standard Support Vector Machine) and LSSVM (Least Squares Support Vector Machine) is put forward. Design/methodology/approach In the competitive growth and industries 4.0, the prediction in the cost plays a key role. Findings At the same time, the original data is dimensionality reduced. The processed data are imported into the SVM and LSSVM models for training and prediction respectively, and the prediction results are compared and analyzed and a more reasonable prediction model is selected. Originality/value The prediction result is further optimized by parameter optimization. The relative error of the prediction model is within 7%, and the prediction accuracy is high and the result is stable.
Comparison of feature importance measures as explanations for classification models

Abstract Explainable artificial intelligence is an emerging research direction helping the user or developer of machine learning models understand why models behave the way they do. The most popular explanation technique is feature importance. However, there are several different approaches how feature importances are being measured, most notably global and local. In this study we compare different feature importance measures using both linear (logistic regression with L1 penalization) and non-linear (random forest) methods and local interpretable model-agnostic explanations on top of them. These methods are applied to two datasets from the medical domain, the openly available breast cancer data from the UCI Archive and a recently collected running injury data. Our results show that the most important features differ depending on the technique. We argue that a combination of several explanation techniques could provide more reliable and trustworthy results. In particular, local explanations should be used in the most critical cases such as false negatives.
First return, then explore

The promise of reinforcement learning is to solve complex sequential decision problems autonomously by specifying a high-level reward function only. However, reinforcement learning algorithms struggle when, as is often the case, simple and intuitive rewards provide sparse and deceptive feedback. Avoiding these pitfalls requires thoroughly exploring the environment, but creating algorithms that can do so remains one of the central challenges of the field. We hypothesise that the main impediment to effective exploration originates from algorithms forgetting how to reach previously visited states ("detachment") and from failing to first return to a state before exploring from it ("derailment"). We introduce Go-Explore, a family of algorithms that addresses these two challenges directly through the simple principles of explicitly remembering promising states and first returning to such states before intentionally exploring. Go-Explore solves all heretofore unsolved Atari games and surpasses the state of the art on all hard-exploration games, with orders of magnitude improvements on the grand challenges Montezuma's Revenge and Pitfall. We also demonstrate the practical potential of Go-Explore on a sparse-reward pick-and-place robotics task. Additionally, we show that adding a goal-conditioned policy can further improve Go-Explore's exploration efficiency and enable it to handle stochasticity throughout training. The substantial performance gains from Go-Explore suggest that the simple principles of remembering states, returning to them, and exploring from them are a powerful and general approach to exploration, an insight that may prove critical to the creation of truly intelligent learning agents.
Learning to Estimate Hidden Motions with Global Motion Aggregation

Occlusions pose a significant challenge to optical flow algorithms that rely on local evidences. We consider an occluded point to be one that is imaged in the reference frame but not in the next, a slight overloading of the standard definition since it also includes points that move out-of-frame. Estimating the motion of these points is extremely difficult, particularly in the two-frame setting. Previous work relies on CNNs to learn occlusions, without much success, or requires multiple frames to reason about occlusions using temporal smoothness. In this paper, we argue that the occlusion problem can be better solved in the two-frame case by modelling image self-similarities. We introduce a global motion aggregation module, a transformer-based approach to find long-range dependencies between pixels in the first image, and perform global aggregation on the corresponding motion features. We demonstrate that the optical flow estimates in the occluded regions can be significantly improved without damaging the performance in non-occluded regions. This approach obtains new state-of-the-art results on the challenging Sintel dataset, improving the average end-point error by 13.6% on Sintel Final and 13.7% on Sintel Clean. At the time of submission, our method ranks first on these benchmarks among all published and unpublished approaches. Code is available at https://github.com/zacjiang/GMA.
PENet: Towards Precise and Efficient Image Guided Depth Completion

Image guided depth completion is the task of generating a dense depth map from a sparse depth map and a high quality image. In this task, how to fuse the color and depth modalities plays an important role in achieving good performance. This paper proposes a two-branch backbone that consists of a color-dominant branch and a depth-dominant branch to exploit and fuse two modalities thoroughly. More specifically, one branch inputs a color image and a sparse depth map to predict a dense depth map. The other branch takes as inputs the sparse depth map and the previously predicted depth map, and outputs a dense depth map as well. The depth maps predicted from two branches are complimentary to each other and therefore they are adaptively fused. In addition, we also propose a simple geometric convolutional layer to encode 3D geometric cues. The geometric encoded backbone conducts the fusion of different modalities at multiple stages, leading to good depth completion results. We further implement a dilated and accelerated CSPN++ to refine the fused depth map efficiently. The proposed full model ranks 1st in the KITTI depth completion online leaderboard at the time of submission. It also infers much faster than most of the top ranked methods. The code of this work is available at https://github.com/JUGGHM/PENet_ICRA2021.
Solving Multiobjective Fuzzy Job-Shop Scheduling Problem by a Hybrid Adaptive Differential Evolution Algorithm

The job-shop scheduling problem (JSP) is NP hard, which has very important practical significance. Because of many uncontrollable factors, such as machine delay or human factors, it is difficult to use a single real-number to express the processing and completion time of the jobs. JSP with fuzzy processing time and completion time (FJSP) can model the scheduling more comprehensively, which benefits from the developments of fuzzy sets. Fuzzy relative entropy leads to a method that can evaluate the quality of a feasible solution following the comparison between the actual value and the ideal value (the due date). Therefore, the multiobjective FJSP can be transformed into a single-objective optimization problem and solved by a hybrid adaptive differential evolution (HADE) algorithm. The maximum completion time, the total delay time, and the total energy consumption of jobs will be considered. HADE adopts a mutation strategy based on DE-current-to-best. Its parameters (CR and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i> ) are all made adaptive and normally distributed. The new individuals are selected according to the fitness value (FRE) obtained from a population consisting of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> parents and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> children in HADE. The algorithm is analyzed from different viewpoints. As the experimental results demonstrate, the performance of the HADE algorithm is better than those of some other state-of-the-art algorithms (namely, ant colony optimization, artificial bee colony, and particle swarm optimization).
Direction-Aware Spatial Context Features for Shadow Detection and Removal

Shadow detection and shadow removal are fundamental and challenging tasks, requiring an understanding of the global image semantics. This paper presents a novel deep neural network design for shadow detection and removal by analyzing the spatial image context in a direction-aware manner. To achieve this, we first formulate the direction-aware attention mechanism in a spatial recurrent neural network (RNN) by introducing attention weights when aggregating spatial context features in the RNN. By learning these weights through training, we can recover direction-aware spatial context (DSC) for detecting and removing shadows. This design is developed into the DSC module and embedded in a convolutional neural network (CNN) to learn the DSC features at different levels. Moreover, we design a weighted cross entropy loss to make effective the training for shadow detection and further adopt the network for shadow removal by using a Euclidean loss function and formulating a color transfer function to address the color and luminosity inconsistencies in the training pairs. We employed two shadow detection benchmark datasets and two shadow removal benchmark datasets, and performed various experiments to evaluate our method. Experimental results show that our method performs favorably against the state-of-the-art methods for both shadow detection and shadow removal.
Identifying Mild Cognitive Impairment and mild Alzheimer’s disease based on spontaneous speech using ASR and linguistic features

Alzheimer’s disease (AD) is a neurodegenerative disorder that develops for years before clinical manifestation, while mild cognitive impairment is clinically considered as a prodromal stage of AD. For both types of neurodegenerative disorders, early diagnosis is crucial for the timely treatment and to decelerate progression. Unfortunately, the current diagnostic solutions are time-consuming. Here, we seek to exploit the observation that these illnesses frequently disturb the mental and linguistic functions, which might be detected from the spontaneous speech produced by the patient. First, we present an automatic speech recognition based procedure for the extraction of a special set of acoustic features. Second, we present a linguistic feature set that is extracted from the transcripts of the same speech signals. The usefulness of the two feature sets is evaluated via machine learning experiments, where our goal is not only to differentiate between the patients and the healthy control group, but also to tell apart Alzheimer’s patients from those with mild cognitive impairment. Our results show that based on only the acoustic features, we are able to separate the various groups with accuracy scores between 74–82%. We attained similar accuracy scores when using only the linguistic features. With the combination of the two types of features, the accuracy scores rise to between 80–86%, and the corresponding F1 values also fall between 78–86%. We hope that with the full automation of the processing chain, our method can serve as the basis of an automatic screening test in the future.
A Conditional Adversarial Network for Change Detection in Heterogeneous Images

Due to the distinct statistical properties in cross-sensor images, change detection in heterogeneous images is much more challenging than in homogeneous images. In this letter, we adopt a conditional generative adversarial network (cGAN) to transform the heterogeneous synthetic aperture radar (SAR) and optical images into some space where their information has a more consistent representation, making the direct comparison feasible. Our proposed framework contains a cGAN-based translation network that aims to translate the optical image with the SAR image as a target, and an approximation network that approximates the SAR image to the translated one by reducing their pixelwise difference. The two networks are updated alternately and when they are both trained well, the two translated and approximated images can be considered as homogeneous, from which the final change map can be acquired by direct comparison. Theoretical analysis and experimental results demonstrate the effectiveness and robustness of the proposed framework.
Three-way enhanced convolutional neural networks for sentence-level sentiment classification

Deep neural network models have achieved remarkable results in sentiment classification. Traditional feature-based methods perform slightly worse than deep learning methods in terms of classification accuracy, but they have their own advantages in interpretability and time complexity. To the best of our knowledge, few works study the ensemble of deep learning methods and traditional feature-based methods. Inspired by the methodology of three-way decisions, we proposed a three-way enhanced convolutional neural network model named 3W-CNN. 3W-CNN can be seen as an ensemble method which uses the enhance model to optimize convolutional neural networks (CNN). The enhance model is selected according to the classification accuracy and the difference in classification results compared to CNN. Support vector machine with naive bayes features (NB-SVM) is selected as the enhance model after comparing with several baseline models. However, the performance of NB-SVM is worse than CNN on most of benchmark datasets. To address this issue, we construct a component named confidence divider and design a confidence function to distinguish the classification quality of CNN. NB-SVM is further utilized to reclassify the predictions with weak confidence. The experimental results validated the effectiveness of 3W-CNN and showed three-way decisions could further improve the accuracy of sentiment classification.
A stacking model using URL and HTML features for phishing webpage detection

In this paper, we present a stacking model to detect phishing webpages using URL and HTML features. In terms of features, we design lightweight URL and HTML features and introduce HTML string embedding without using the third-party services, making it possible to develop real-time detection applications. Furthermore, we devise a stacking model by combining GBDT, XGBoost and LightGBM in multiple layers, which enables different models to be complementary, thus improving the performance on phishing webpage detection. In particular, we collect two real-world datasets for evaluations, named as 50K-PD and 50K-IPD, respectively. 50K-PD contains 49,947 webpages with URLs and HTML codes. 50K-IPD contains 53,103 webpages with screenshots in addition to URLs and HTML codes. The proposed approach outperforms quite a few machine learning models on multiple metrics, achieving 97.30% on accuracy, 4.46% on missing alarm rate, and 1.61% on false alarm rate on 50K-PD dataset. On 50K-IPD dataset, the proposed approach achieves 98.60% on accuracy, 1.28% on missing alarm rate, and 1.54% on false alarm rate.
Text document clustering using Spectral Clustering algorithm with Particle Swarm Optimization

Document clustering is a gathering of textual content documents into groups or clusters. The main aim is to cluster the documents, which are internally logical but considerably different from each other. It is a crucial process used in information retrieval, information extraction and document organization. In recent years, the spectral clustering is widely applied in the field of machine learning as an innovative clustering technique. This research work proposes a novel Spectral Clustering algorithm with Particle Swarm Optimization (SCPSO) to improve the text document clustering. By considering global and local optimization function, the randomization is carried out with the initial population. This research work aims at combining the spectral clustering with swarm optimization to deal with the huge volume of text documents. The proposed algorithm SCPSO is examined with the benchmark database against the other existing approaches. The proposed algorithm SCPSO is compared with the Spherical K-means, Expectation Maximization Method (EM) and standard PSO Algorithm. The concluding results show that the proposed SCPSO algorithm yields better clustering accuracy than other clustering techniques.
Neural Multi-task Recommendation from Multi-behavior Data

Most existing recommender systems leverage user behavior data of one type, such as the purchase behavior data in E-commerce. We argue that other types of user behavior data also provide valuable signal, such as views, clicks, and so on. In this work, we contribute a new solution named NMTR (short for Neural Multi-Task Recommendation) for learning recommender systems from user multi-behavior data. In particular, our model accounts for the cascading relationship among different types of behaviors (e.g., a user must click on a product before purchasing it). We perform a joint optimization based on the multi-task learning framework, where the optimization on a behavior is treated as a task. Extensive experiments on the real-world dataset demonstrate that NMTR significantly outperforms state-of-the-art recommender systems that are designed to learn from both single-behavior data and multi-behavior data.
Scalable muscle-actuated human simulation and control

Many anatomical factors, such as bone geometry and muscle condition, interact to affect human movements. This work aims to build a comprehensive musculoskeletal model and its control system that reproduces realistic human movements driven by muscle contraction dynamics. The variations in the anatomic model generate a spectrum of human movements ranging from typical to highly stylistic movements. To do so, we discuss scalable and reliable simulation of anatomical features, robust control of under-actuated dynamical systems based on deep reinforcement learning, and modeling of pose-dependent joint limits. The key technical contribution is a scalable, two-level imitation learning algorithm that can deal with a comprehensive full-body musculoskeletal model with 346 muscles. We demonstrate the predictive simulation of dynamic motor skills under anatomical conditions including bone deformity, muscle weakness, contracture, and the use of a prosthesis. We also simulate various pathological gaits and predictively visualize how orthopedic surgeries improve post-operative gaits.