Reviewerly/exHarmony huggingface dataset:
Inducing structure in reward learning by learning features

Reward learning enables robots to learn adaptable behaviors from human input. Traditional methods model the reward as a linear function of hand-crafted features, but that requires specifying all the relevant features a priori, which is impossible for real-world tasks. To get around this issue, recent deep Inverse Reinforcement Learning (IRL) methods learn rewards directly from the raw state but this is challenging because the robot has to implicitly learn the features that are important and how to combine them, simultaneously. Instead, we propose a divide-and-conquer approach: focus human input specifically on learning the features separately, and only then learn how to combine them into a reward. We introduce a novel type of human input for teaching features and an algorithm that utilizes it to learn complex features from the raw state space. The robot can then learn how to combine them into a reward using demonstrations, corrections, or other reward learning frameworks. We demonstrate our method in settings where all features have to be learned from scratch, as well as where some of the features are known. By first focusing human input specifically on the feature(s), our method decreases sample complexity and improves generalization of the learned reward over a deep IRL baseline. We show this in experiments with a physical 7-DoF robot manipulator, and in a user study conducted in a simulated environment.
Blockchain-Enabled Conditional Decentralized Vehicular Crowdsensing System

The rapid growth of connected and autonomous vehicles (CAVs) shows an urgent demand for driving and transportation-related data, which gives rise to vehicular crowdsensing systems (VCSs). Nevertheless, the existing centralized VCS framework mainly faces the system reliability problem while the decentralized one cannot satisfy the management flexibility. In addition, when the privacy preservation scheme that prevents information leakage encounters the user selection scheme that desires detailed information of participants, how to balance this seemingly irreconcilable contradiction is inevitable for VCS. To remedy that, we take the first research attempt and explore the balance point between the system management, privacy preservation, and quality of experience (QoE) of participants. By fully exploiting the characters of participating entities, a blockchain-enabled conditional decentralized VCS is proposed in this paper. Firstly, we propose a privacy-preserving scheme where the zk-SNARK proof combines with the mixed-task smart contract to guarantee the interaction process will not reveal any private information of participants. Secondly, we propose an efficient reputation management mechanism that renders certain the participants can get a satisfactory QoE even under the condition that the private information of users is secured. And also, the malicious operations in the system will be effectively supervised. Theoretical analysis and extensive simulations demonstrate the security and efficiency properties of privacy preservation and indicate the effectiveness of reputation management.
Supervised learning: From theory to applications

Supervised learning is one of the most important components of machine learning which deals with the theory and applications of algorithms that can discover patterns in data when provided with existing independent and dependent factors to predict the future values of dependent factors. Supervised learning is a broadly used machine learning methodology with its applications in diverse areas like Natural Language Processing, image and video classification, medical analysis, and many other prevailing fields. This chapter explains various supervised machine learning techniques and a single chapter may not be adequate for a complete review of the same, nevertheless, we hope to cover the key issues and state-of-the-art for suggesting researchers regarding their research directions and stating possible setbacks that have yet to be explored. In the preliminary section of this chapter, we will present the basic concepts behind supervised learning with some examples to comprehend the idea followed by different types of Supervised Learning problems. Researchers have proposed a variety of techniques to solve such problems like instance-based techniques logic-based techniques, stochastic techniques, and so on. Consequently, the middle section of this chapter will discuss some commonly used algorithms of supervised learning and their efficacy in terms of complexity, accuracy, overfitting risk measures, and speed of learning. To investigate the robustness of the model, it is important to evaluate the results and that’s where we will discuss evaluation metrics like accuracy, precision, recall, etc. along with main problems encountered while making our model learn. At last, we will look into the wide areas like computer vision and text media that exploit techniques from supervised learning.
Federated Learning with Blockchain Approach for Trust Management in IoV

The internet of vehicles (IoV), a variant of the traditional VANET, allows real-time data exchange between vehicles, roadside units, parking, and city infrastructure. Nevertheless, the IoV poses many security concerns due to its open nature. Traditional security solutions may not address all the IoV security risks and provide complete protection. Therefore, it is critical to establish trust and to identify dishonest nodes. As a result, trust management-based techniques are also required to improve IoV security. This paper proposes federated learning with a blockchain approach for trust management (FBTM) in IoV. Thus, a vehicular trust evaluation is designed to improve the data acquired for the federated learning model learning process. Moreover, a novel blockchain-based reputation system is developed to guarantee the storage and the share of global federated learning models. In the meanwhile, proof of reputation consensus is proposed to evaluate the roadside units operating as aggregators in the IoV network. Simulation results demonstrate that the proposed scheme is effective for IoV security.
Unintentional Compromising Electromagnetic Emanations from IT Equipment: A Concept Map of Domain Knowledge

The purpose of this study was to capture through a systematic literature review the domain specific knowledge related to unintentional compromising electromagnetic emanations from Information Technology equipment. It applies a qualitative thematic analysis to the identified papers to show the main themes and then uses concept mapping to organise and represent the acquired knowledge. The resulting concept map will be used in future work to develop a list of factors that Radio Frequency Engineers use to assess the likelihood of unintentional emanations leading to the compromise of sensitive information.
Medium Transmission Map Matters for Learning to Restore Real-World Underwater Images

Low illumination, light reflections, scattering, absorption, and suspended particles inevitably lead to critically degraded underwater image quality, which poses great challenges for recognizing objects from underwater images. The existing underwater enhancement methods that aim to promote underwater visibility heavily suffer from poor image restoration performance and generalization ability. To reduce the difficulty of underwater image enhancement, we introduce the media transmission map as guidance for image enhancement. Different from the existing frameworks, which also introduce the medium transmission map for better distribution modeling, we formulate the interaction between the underwater visual images and the transmission map explicitly to obtain better enhancement results. At the same time, our network only requires supervised learning of the media transmission map during training, and the corresponding prediction map can be generated in subsequent tests, which reduces the operation difficulty of subsequent tasks. Thanks to our formulation, the proposed method with a very lightweight network configuration can produce very promising results of 22.6 dB on the challenging Test-R90 with an impressive 30.3 FPS, which is faster than most current algorithms. Comprehensive experimental results have demonstrated the superiority on underwater perception.
A unified uncertainty network for tumor segmentation using uncertainty cross entropy loss and prototype similarity

Uncertainty estimation and out-of-distribution (OOD) detection are topics of practical significance in deep convolutional neural network-based tumor segmentation tasks. We propose a unified uncertainty segmentation network to handle these two tasks with a single network. The uncertainty cross entropy loss is proposed to guide the network to directly output the prediction uncertainty of each pixel instead of executing several times in the prediction phase. We extend the uncertainty to the case level to address OOD sample detection problems. Case-level uncertainty is estimated based on prototype similarity. We perform pixel-level uncertainty experiments and OOD detection experiments on four datasets. The experimental results show that our proposed method is more suitable for uncertainty estimation in tumor segmentation than existing methods. Our proposed method only needs to modify the network output and loss function and does not need to execute the network multiple times when estimating uncertainty. Moreover, our method shows improved performance in tumor segmentation.
DONE: Distributed Approximate Newton-type Method for Federated Edge Learning

There is growing interest in applying distributed machine learning to edge computing, forming <i>federated edge learning</i> . Federated edge learning faces non-i.i.d. and heterogeneous data, and the communication between edge workers, possibly through distant locations and with unstable wireless networks, is more costly than their local computational overhead. In this work, we propose <inline-formula><tex-math notation="LaTeX">${{\sf DONE}}$</tex-math></inline-formula> , a distributed approximate Newton-type algorithm with fast convergence rate for communication-efficient federated edge learning. First, with strongly convex and smooth loss functions, <inline-formula><tex-math notation="LaTeX">${{\sf DONE}}$</tex-math></inline-formula> approximates the Newton direction in a distributed manner using the classical Richardson iteration on each edge worker. Second, we prove that <inline-formula><tex-math notation="LaTeX">${{\sf DONE}}$</tex-math></inline-formula> has linear-quadratic convergence and analyze its communication complexities. Finally, the experimental results with non-i.i.d. and heterogeneous data show that <inline-formula><tex-math notation="LaTeX">${{\sf DONE}}$</tex-math></inline-formula> attains a comparable performance to Newton's method. Notably, <inline-formula><tex-math notation="LaTeX">${{\sf DONE}}$</tex-math></inline-formula> requires fewer communication iterations compared to distributed gradient descent and outperforms DANE, FEDL, and GIANT, state-of-the-art approaches, in the case of non-quadratic loss functions.
Privacy-Preserving Anomaly Detection in Cloud Manufacturing Via Federated Transformer

With the rapid development of cloud manufacturing, industrial production with edge computing as the core architecture has been greatly developed. However, edge devices often suffer from abnormalities and failures in industrial production. Therefore, detecting these abnormal situations timely and accurately is crucial for cloud manufacturing. As such, a straightforward solution is that the edge device uploads the data to the cloud for anomaly detection. However, Industry 4.0 puts forward higher requirements for data privacy and security so that it is unrealistic to upload data from edge devices directly to the cloud. Considering the abovementioned severe challenges, this article customizes a weakly supervised edge computing anomaly detection framework, i.e., federated learning-based transformer framework ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FedAnomaly</i> ), to deal with the anomaly detection problem in cloud manufacturing. Specifically, we introduce federated learning (FL) framework that allows edge devices to train an anomaly detection model in collaboration with the cloud without compromising privacy. To boost the privacy performance of the framework, we add differential privacy noise to the uploaded features. To further improve the ability of edge devices to extract abnormal features, we use the transformer to extract the feature representation of abnormal data. In this context, we design a novel collaborative learning protocol to promote efficient collaboration between FL and transformer. Furthermore, extensive case studies on four benchmark datasets verify the effectiveness of the proposed framework. To the best of our knowledge, this is the first time integrating FL and transformer to deal with anomaly detection problems in cloud manufacturing.
APRNet: A 3D Anisotropic Pyramidal Reversible Network With Multi-Modal Cross-Dimension Attention for Brain Tissue Segmentation in MR Images

Brain tissue segmentation in multi-modal magnetic resonance (MR) images is significant for the clinical diagnosis of brain diseases. Due to blurred boundaries, low contrast, and intricate anatomical relationships between brain tissue regions, automatic brain tissue segmentation without prior knowledge is still challenging. This paper presents a novel 3D fully convolutional network (FCN) for brain tissue segmentation, called APRNet. In this network, we first propose a 3D anisotropic pyramidal convolutional reversible residual sequence (3DAPC-RRS) module to integrate the intra-slice information with the inter-slice information without significant memory consumption; secondly, we design a multi-modal cross-dimension attention (MCDA) module to automatically capture the effective information in each dimension of multi-modal images; then, we apply 3DAPC-RRS modules and MCDA modules to a 3D FCN with multiple encoded streams and one decoded stream for constituting the overall architecture of APRNet. We evaluated APRNet on two benchmark challenges, namely MRBrainS13 and iSeg-2017. The experimental results show that APRNet yields state-of-the-art segmentation results on both benchmark challenge datasets and achieves the best segmentation performance on the cerebrospinal fluid region. Compared with other methods, our proposed approach exploits the complementary information of different modalities to segment brain tissue regions in both adult and infant MR images, and it achieves the average Dice coefficient of 87.22% and 93.03% on the MRBrainS13 and iSeg-2017 testing data, respectively. The proposed method is beneficial for quantitative brain analysis in the clinical study, and our code is made publicly available.
Sentiment Analysis of Roman Urdu on E-Commerce Reviews Using Machine Learning

Sentiment analysis task has widely been studied for various languages such as English and French. However, Roman Urdu sentiment analysis yet requires more attention from peer-researchers due to the lack of Off-the-Shelf Natural Language Processing (NLP) solutions. The primary objective of this study is to investigate the diverse machine learning methods for the sentiment analysis of Roman Urdu data which is very informal in nature and needs to be lexically normalized. To mitigate this challenge, we propose a fine-tuned Support Vector Machine (SVM) powered by Roman Urdu Stemmer. In our proposed scheme, the corpus data is initially cleaned to remove the anomalies from the text. After initial pre-processing, each user review is being stemmed. The input text is transformed into a feature vector using the bag-of-word model. Subsequently, the SVM is used to classify and detect user sentiment. Our proposed scheme is based on a dictionary based Roman Urdu stemmer. The creation of the Roman Urdu stemmer is aimed at standardizing the text so as to minimize the level of complexity. The efficacy of our proposed model is also empirically evaluated with diverse experimental configurations, so as to fine-tune the hyper-parameters and achieve superior performance. Moreover, a series of experiments are conducted on diverse machine learning and deep learning models to compare the performance with our proposed model. We also introduced the largest dataset on Roman Urdu, i.e., Roman Urdu e-commerce dataset (RUECD), which contains 26K+ user reviews annotated by the group of experts. The RUECD is challenging and the largest dataset available of Roman Urdu. The experiments show that the newly generated dataset is quite challenging and requires more attention from the peer researchers for Roman Urdu sentiment analysis.
A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study

Risk scores are widely used for clinical decision making and commonly generated from logistic regression models. Machine-learning-based methods may work well for identifying important predictors to create parsimonious scores, but such 'black box' variable selection limits interpretability, and variable importance evaluated from a single model can be biased. We propose a robust and interpretable variable selection approach using the recently developed Shapley variable importance cloud (ShapleyVIC) that accounts for variability in variable importance across models. Our approach evaluates and visualizes overall variable contributions for in-depth inference and transparent variable selection, and filters out non-significant contributors to simplify model building steps. We derive an ensemble variable ranking from variable contributions across models, which is easily integrated with an automated and modularized risk score generator, AutoScore, for convenient implementation. In a study of early death or unplanned readmission after hospital discharge, ShapleyVIC selected 6 variables from 41 candidates to create a well-performing risk score, which had similar performance to a 16-variable model from machine-learning-based ranking. Our work contributes to the recent emphasis on interpretability of prediction models for high-stakes decision making, providing a disciplined solution to detailed assessment of variable importance and transparent development of parsimonious clinical risk scores.
Contextual Importance and Utility: A Theoretical Foundation

This paper provides new theory to support to the eXplainable AI (XAI) method Contextual Importance and Utility (CIU). CIU arithmetic is based on the concepts of Multi-Attribute Utility Theory, which gives CIU a solid theoretical foundation. The novel concept of contextual influence is also defined, which makes it possible to compare CIU directly with so-called additive feature attribution (AFA) methods for model-agnostic outcome explanation. One key takeaway is that the ‘influence’ concept used by AFA methods is inadequate for outcome explanation purposes even for simple models to explain. Experiments with simple models show that explanations using contextual importance (CI) and contextual utility (CU) produce explanations where influence-based methods fail. It is also shown that CI and CU guarantees explanation faithfulness towards the explained model.
Optional purchase verification in e‐commerce platforms: More representative product ratings and higher quality reviews

E‐commerce platforms struggle to create and maintain high‐quality reputation systems. One promising option is “purchase verification,” which confirms that the user reviewing a product purchased the product from the platform. Previous works comparing platforms that require purchase verification with platforms that do not offer purchase verification found that review manipulation is easier in the latter. But what happens in platforms where purchase verification is optional? In such platforms, there is no monetary cost for posting fake reviews. Yet, optional purchase verification (OPV) might introduce indirect costs for fake reviewers through expectation disconfirmation, hence positively affecting the reputation ecosystem of an e‐commerce platform. To investigate, we use a quasi‐experimental setup to analyze 336,043 book reviews. We find empirical evidence that introducing OPV reduces fake reviews, most of which are positive. This reduction of fake reviews results in lower, more representative product ratings and longer and more helpful reviews posted by more experienced reviewers. These new findings extend our understanding of how OPV can improve a platform's reputation ecosystem and suggest managerial interventions for platforms that have yet to develop a verification mechanism.
Reconfigurable Activation Functions in Integrated Optical Neural Networks

The implementation of nonlinear activation functions is one of the key challenges that optical neural networks face.To the date, different approaches have been proposed, including switching to digital implementations, electro-optical or all optical.In this article, we compare the response of different electro-optic architectures where part of the input optical signal is converted into the electrical domain and used to self-phase modulate the intensity of the remaining optical signal.These architectures are made up of Mach Zehnder Interferometers (MZI) and microring resonators (MRR).We have compared the corresponding transfer functions with commonly used activation functions in state-of-theart machine learning models and carried out an in-depth analysis of the capabilities of those architectures to generate the proposed activation functions.We demonstrate that a ring assisted MZI and a two-ring assisted MZI present the highest expressivity among the proposed structures.To the best of our knowledge, this is the first time that a quantified analysis of the capabilities of optical devices to mimic state-of-the-art activation functions is presented.The obtained activation functions are benchmarked on two machine learning examples: classification task using the Iris dataset, and image recognition using the MNIST dataset.We use complex-valued feed-forward neural networks and get test accuracies of 97% and 95% respectively.
The Past, Present, and Prospective Future of XAI: A Comprehensive Review

With the increasing growth and availability of data, Artificial Intelligence (AI) based black-box models have shown significant effectiveness to solve real-world and mission-critical problems in a wide range of domains such as finance, healthcare, security, natural language processing, transportation, military, and retail. However, the importance of interpretable and explainable decision-making processes within AI-based systems is becoming crucial to provide transparency and confidence among end-users from diverse backgrounds. Acceptance of these black-box models considerably depends on to which extent the users or technical personnel understand and trust the underlying mechanism. However, ML methods and models are getting more sophisticated and transparent progressively. Though the domain experts understand the mathematical principle, they still face difficulties expressing the mechanism for a wide audience. To address this issue, a recent research domain—Explainable AI (XAI), has emerged intending to generate the contextual explanatory model for practical deployment. In this chapter, we systematically review and examine the existing literature and contribution of XAI in different domains. The presented taxonomy contains our investigation on the necessity of explaining and contextualizing methods and models in the AI ecosystem. Moreover, our study includes prospects in this field and the possible consequences of more explainable and interpretable ML models concerning different domains. This critical taxonomy provides the opportunities and challenges in the area of XAI that serves as a reference to future AI researchers.