Reviewerly/exHarmony huggingface dataset:
Implementation experiments on convolutional neural network training using synthetic images for 3D pose estimation of an excavator on real images

Remote and descriptive visualization of spatio-temporal information of excavator activities may increase awareness about jobsite hazards and operational performance in earthwork operations. One of the emerging approaches to collect this information is to extract the 3D pose of an excavator from the video frames using a convolutional neural network (CNN). However, this method requires labeling the training datasets, which are difficult to prepare because of conditions unsuitable for installing the motion capture sensors. This study investigates the performance of a CNN for estimating the 3D pose when trained on a synthetic dataset. In particular, a kinematic constraint is proposed to update the model parameters efficiently during training. The results show that the proposed method estimated the 3D poses of a real excavator with an average pose error of 9.63°. Hence, the proposed data augmentation method could help address the training data issues and improves the learning of real data complexity.
Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification

The speech representations learned from large-scale unlabeled data have shown better generalizability than those from supervised learning and thus attract a lot of interest to be applied for various downstream tasks. In this paper, we explore the limits of speech representations learned by different self-supervised objectives and datasets for automatic speaker verification (ASV), especially with a well-recognized SOTA ASV model, ECAPA-TDNN [1], as a downstream model. The representations from all hidden layers of the pre-trained model are firstly averaged with learnable weights and then fed into the ECAPA-TDNN as input features. The experimental results on Voxceleb dataset show that the weighted average representation is significantly superior to FBank, a conventional handcrafted feature for ASV. Our best single system achieves 0.537%, 0.569%, and 1.180% equal error rate (EER) on the three official trials of VoxCeleb1, separately. Accordingly, the ensemble system with three pre-trained models can further improve the EER to 0.479%, 0.536% and 1.023%. Among the three evaluation trials, our best system outperforms the winner system [2] of the VoxCeleb Speaker Recognition Challenge 2021 (VoxSRC2021) on the VoxCeleb1-E trial.
Visualization and interpretation of geochemical exploration data using GIS and machine learning methods

Geochemical exploration has provided significant clues for mineral exploration and has helped discover many mineral deposits. Although various methods, including classic statistics, multivariate statistics, geostatistics, fractal/multifractal models, and machine learning algorithms, have been successfully employed to process geochemical exploration data, efficient interpretation and visualization of geochemical exploration data in support of the discovery of mineral deposits remain challenging. In this study, a workflow for intelligent interpretation and visualization of geochemical exploration data, defined as processing geochemical survey data with support of a geographical information system (GIS) and machine learning algorithms, was proposed. The effectiveness of the intelligent interpretation and visualization of geochemical exploration data supported by GIS and machine learning algorithms was demonstrated using a case study of processing a regional-scale geochemical survey dataset collected from Sichuan Province, China. Future research should add more advanced mathematical and statistical models, such as deep learning algorithms, into GIS to support the intelligent interpretation and visualization of geochemical exploration data.
The fabrics of machine moderation: Studying the technical, normative, and organizational structure of Perspective API

Over recent years, the stakes and complexity of online content moderation have been steadily raised, swelling from concerns about personal conflict in smaller communities to worries about effects on public life and democracy. Because of the massive growth in online expressions, automated tools based on machine learning are increasingly used to moderate speech. While ‘design-based governance’ through complex algorithmic techniques has come under intense scrutiny, critical research covering algorithmic content moderation is still rare. To add to our understanding of concrete instances of machine moderation, this article examines Perspective API, a system for the automated detection of ‘toxicity’ developed and run by the Google unit Jigsaw that can be used by websites to help moderate their forums and comment sections. The article proceeds in four steps. First, we present our methodological strategy and the empirical materials we were able to draw on, including interviews, documentation, and GitHub repositories. We then summarize our findings along five axes to identify the various threads Perspective API brings together to deliver a working product. The third section discusses two conflicting organizational logics within the project, paying attention to both critique and what can be learned from the specific case at hand. We conclude by arguing that the opposition between ‘human’ and ‘machine’ in speech moderation obscures the many ways these two come together in concrete systems, and suggest that the way forward requires proactive engagement with the design of technologies as well as the institutions they are embedded in.
Deep learning based fall detection using smartwatches for healthcare applications

We implement a smart watch-based system to predict fall detection. We differentiate fall detection from four common daily activities: sitting, squatting, running, and walking. Moreover, we separate falling into falling from a chair and falling from a standing position. We develop a mobile application that collects the acceleration and gyroscope sensor data and transfers them to the cloud. In the cloud, we implement a deep learning algorithm to classify the activity according to the given classes. To increase the number of data samples available for training, we use the Bica cubic Hermite interpolation, which allows us to improve the accuracy of the neural network. The 38 statistical data features were calculated using the rolling update approach and used as input to the classifier. For activity classification, we have adopted the bi-directional long short-term memory (BiLSTM) neural network. The results demonstrate that our system can detect falling with an accuracy of 99.59% (using leave-one-activity-out cross-validation) and 97.35% (using leave-one-subject-out cross-validation) considering all activities. When considering only binary classification (falling vs. all other activities), perfect accuracy is achieved.
Privacy-Preserving Dynamic Personalized Pricing with Demand Learning

The prevalence of e-commerce has made customers’ detailed personal information readily accessible to retailers, and this information has been widely used in pricing decisions. When using personalized information, the question of how to protect the privacy of such information becomes a critical issue in practice. In this paper, we consider a dynamic pricing problem over T time periods with an unknown demand function of posted price and personalized information. At each time t, the retailer observes an arriving customer’s personal information and offers a price. The customer then makes the purchase decision, which will be utilized by the retailer to learn the underlying demand function. There is potentially a serious privacy concern during this process: a third-party agent might infer the personalized information and purchase decisions from price changes in the pricing system. Using the fundamental framework of differential privacy from computer science, we develop a privacy-preserving dynamic pricing policy, which tries to maximize the retailer revenue while avoiding information leakage of individual customer’s information and purchasing decisions. To this end, we first introduce a notion of anticipating [Formula: see text]-differential privacy that is tailored to the dynamic pricing problem. Our policy achieves both the privacy guarantee and the performance guarantee in terms of regret. Roughly speaking, for d-dimensional personalized information, our algorithm achieves the expected regret at the order of [Formula: see text] when the customers’ information is adversarially chosen. For stochastic personalized information, the regret bound can be further improved to [Formula: see text]. This paper was accepted by J. George Shanthikumar, big data analytics.
Cost-Effective Federated Learning in Mobile Edge Networks

Federated learning (FL) is a distributed learning paradigm that enables a large number of mobile devices to collaboratively learn a model under the coordination of a central server without sharing their raw data. Despite its practical efficiency and effectiveness, the iterative on-device learning process (e.g., local computations and global communications with the server) incurs a considerable cost in terms of learning time and energy consumption, which depends crucially on the number of selected clients and the number of local iterations in each training round. In this paper, we analyze how to design adaptive FL in mobile edge networks that optimally chooses these essential control variables to minimize the total cost while ensuring convergence. We establish the analytical relationship between the total cost and the control variables with the convergence upper bound. To efficiently solve the cost minimization problem, we develop a low-cost sampling-based algorithm to learn the convergence related unknown parameters. We derive important solution properties that effectively identify the design principles for different optimization metrics. Practically, we evaluate our theoretical results both in a simulated environment and on a hardware prototype. Experimental evidence verifies our derived properties and demonstrates that our proposed solution achieves near-optimal performance for different optimization metrics for various datasets and heterogeneous system and statistical settings.
EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference

Transformer-based language models such as BERT provide significant accuracy improvement to a multitude of natural language processing (NLP) tasks. However, their hefty computational and memory demands make them challenging to deploy to resource-constrained edge platforms with strict latency requirements.
Anomaly Detection: A Survey

Anomaly detection (AD) is considered one of the important research areas that have a diverse range of application domains. Some of the anomaly detection techniques presented in the literature were specifically implemented for certain domains, whereas others were more generic. In this paper, we aim at providing a structured yet extensive overview of current research directions on anomaly detection, including the definition of anomalies and their types, current anomaly detection modes, the output of anomaly detection techniques, and an overview of those techniques presented in the literature, in order to provide a guide when selecting the appropriate approach for certain application domains.
Graph-based multi-label disease prediction model learning from medical data and domain knowledge

In recent years, the means of disease diagnosis and treatment have been improved remarkably, along with the continuous development of technology and science. Researchers have spent tremendous time and effort to build models that aim to assist medical practitioners in decision-making support. However, one of the greatest challenges remains how to identify the connection between different diseases. This study aims to discover the relationship between diseases and symptoms to predict potential diseases for patients. Considering it a multi-label classification problem, the study proposed a new multi-disease prediction model learning from NHANES, an extensive health related dataset, and MEDLINE, a corpus with medical domain knowledge. A heterogeneous information graph is firstly constructed and then populated using medical domain knowledge discovered from MEDLINE. The knowledge graph is analysed for clarification of the relevancy within nodes in positive or negative space, helping to access to the correlation amongst multiple diseases and their symptoms. A multi-label disease prediction model is then developed adopting the medical domain knowledge graph. Empirical experiments are conducted to evaluate the proposed model. The experimental results show that the performance of the proposed model surpassed state-of-the-art related works representing the mainstreams of multi-label classification. This study contributes to the medical community with a novel model for multi-disease prediction and represents a new endeavour on multi-label classification using knowledge graphs.
An end-to-end tabular information-oriented causality event evolutionary knowledge graph for manufacturing documents

Industrial tabular information extraction and its semantic fusion with text (ITIESF) is of great significance in converting and fusing industrial unstructured data into structured knowledge to guide cognitive intelligence analysis in the manufacturing industry. A novel end-to-end ITIESF approach is proposed to integrate tabular information and construct a tabular information-oriented causality event evolutionary knowledge graph (TCEEKG). Specifically, an end-to-end joint learning strategy is presented to mine the semantic information in tables. The definition and modeling method of the intrinsic relationships between tables with their rows and columns in engineering documents are provided to model the tabular information. Due to this, an end-to-end joint entity relationship extraction method for textual and tabular information from engineering documents is proposed to construct text-based knowledge graphs (KG) and tabular information-based causality event evolutionary graphs (CEEG). Then, a novel NSGCN (neighborhoods sample graph convolution network)-based entity alignment is proposed to fuse the cross-knowledge graphs into a unified knowledge base. Furthermore, a translation-based graph structure-driven Q&A (question and answer) approach is designed to respond to cause analysis and problem tracing. Our models can be easily integrated into a prototype system to provide a joint information processing and cognitive analysis. Finally, the approach is evaluated by employing the aerospace machining documents to illustrate that the TCEEKG can considerably help workers strengthen their skills in the cause-and-effect analysis of machining quality issues from a global perspective.
Examining Attention Mechanisms in Deep Learning Models for Sentiment Analysis

Attention-based methods for deep neural networks constitute a technique that has attracted increased interest in recent years. Attention mechanisms can focus on important parts of a sequence and, as a result, enhance the performance of neural networks in a variety of tasks, including sentiment analysis, emotion recognition, machine translation and speech recognition. In this work, we study attention-based models built on recurrent neural networks (RNNs) and examine their performance in various contexts of sentiment analysis. Self-attention, global-attention and hierarchical-attention methods are examined under various deep neural models, training methods and hyperparameters. Even though attention mechanisms are a powerful recent concept in the field of deep learning, their exact effectiveness in sentiment analysis is yet to be thoroughly assessed. A comparative analysis is performed in a text sentiment classification task where baseline models are compared with and without the use of attention for every experiment. The experimental study additionally examines the proposed models’ ability in recognizing opinions and emotions in movie reviews. The results indicate that attention-based models lead to great improvements in the performance of deep neural models showcasing up to a 3.5% improvement in their accuracy.
Hidden Backdoors in Human-Centric Language Models

Natural language processing (NLP) systems have been proven to be vulnerable to backdoor attacks, whereby hidden features (backdoors) are trained into a language model and may only be activated by specific inputs (called triggers), to trick the model into producing unexpected behaviors. In this paper, we create covert and natural triggers for textual backdoor attacks, hidden backdoors, where triggers can fool both modern language models and human inspection. We deploy our hidden backdoors through two state-of-the-art trigger embedding methods. The first approach via homograph replacement, embeds the trigger into deep neural networks through the visual spoofing of lookalike characters replacement. The second approach uses subtle differences between text generated by language models and real natural text to produce trigger sentences with correct grammar and high fluency. We demonstrate that the proposed hidden backdoors can be effective across three downstream security-critical NLP tasks, representative of modern human-centric NLP systems, including toxic comment detection, neural machine translation (NMT), and question answering (QA). Our two hidden backdoor attacks can achieve an Attack Success Rate (ASR) of at least 97% with an injection rate of only 3% in toxic comment detection, 95.1% ASR in NMT with less than 0.5% injected data, and finally 91.12% ASR against QA updated with only 27 poisoning data samples on a model previously trained with 92,024 samples (0.029%). We are able to demonstrate the adversary's high success rate of attacks, while maintaining functionality for regular users, with triggers inconspicuous by the human administrators.
FedGraph: Federated Graph Learning With Intelligent Sampling

Federated learning has attracted much research attention due to its privacy protection in distributed machine learning. However, existing work of federated learning mainly focuses on Convolutional Neural Network (CNN), which cannot efficiently handle graph data that are popular in many applications. Graph Convolutional Network (GCN) has been proposed as one of the most promising techniques for graph learning, but its federated setting has been seldom explored. In this article, we propose FedGraph for federated graph learning among multiple computing clients, each of which holds a subgraph. FedGraph provides strong graph learning capability across clients by addressing two unique challenges. First, traditional GCN training needs feature data sharing among clients, leading to risk of privacy leakage. FedGraph solves this issue using a novel cross-client convolution operation. The second challenge is high GCN training overhead incurred by large graph size. We propose an intelligent graph sampling algorithm based on deep reinforcement learning, which can automatically converge to the optimal sampling policies that balance training speed and accuracy. We implement FedGraph based on PyTorch and deploy it on a testbed for performance evaluation. The experimental results of four popular datasets demonstrate that FedGraph significantly outperforms existing work by enabling faster convergence to higher accuracy.
Multi-scale semi-supervised clustering of brain images: Deriving disease subtypes

Disease heterogeneity is a significant obstacle to understanding pathological processes and delivering precision diagnostics and treatment. Clustering methods have gained popularity for stratifying patients into subpopulations (i.e., subtypes) of brain diseases using imaging data. However, unsupervised clustering approaches are often confounded by anatomical and functional variations not related to a disease or pathology of interest. Semi-supervised clustering techniques have been proposed to overcome this and, therefore, capture disease-specific patterns more effectively. An additional limitation of both unsupervised and semi-supervised conventional machine learning methods is that they typically model, learn and infer from data using a basis of feature sets pre-defined at a fixed anatomical or functional scale (e.g., atlas-based regions of interest). Herein we propose a novel method, "Multi-scAle heteroGeneity analysIs and Clustering" (MAGIC), to depict the multi-scale presentation of disease heterogeneity, which builds on a previously proposed semi-supervised clustering method, HYDRA. It derives multi-scale and clinically interpretable feature representations and exploits a double-cyclic optimization procedure to effectively drive identification of inter-scale-consistent disease subtypes. More importantly, to understand the conditions under which the clustering model can estimate true heterogeneity related to diseases, we conducted extensive and systematic semi-simulated experiments to evaluate the proposed method on a sizeable healthy control sample from the UK Biobank (N = 4403). We then applied MAGIC to imaging data from Alzheimer's disease (ADNI, N = 1728) and schizophrenia (PHENOM, N = 1166) patients to demonstrate its potential and challenges in dissecting the neuroanatomical heterogeneity of common brain diseases. Taken together, we aim to provide guidance regarding when such analyses can succeed or should be taken with caution. The code of the proposed method is publicly available at https://github.com/anbai106/MAGIC.
Explainable AI in Deep Reinforcement Learning Models for Power System Emergency Control

Artificial intelligence (AI) technology has become an important trend to support the analysis and control of complex and time-varying power systems. Although deep reinforcement learning (DRL) has been utilized in the power system field, most of these DRL models are regarded as black boxes, which are difficult to explain and cannot be used on occasions when human operators need to participate. Using the explainable AI (XAI) technology to explain why power system models make certain decisions is as important as the accuracy of the decisions themselves because it ensures trust and transparency in the model decision-making process. The interpretability issue in DRL models in power system emergency control is discussed in this article. The proposed interpretable method is a backpropagation deep explainer based on Shapley additive explanations (SHAPs), which is named the Deep-SHAP method. The Deep-SHAP method is adopted to provide a reasonable interpretable model for a DRL-based emergency control application. For the DRL model, the importance of input features has been quantified to obtain contributions for the outcome of the model. Further, feature classification of the inputs and probabilistic analysis of the outputs in the XAI model is added to interpretability results for better clarity.
Sizeless

Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain.