Reviewerly/exHarmony huggingface dataset:
Tree Decomposed Graph Neural Network

Graph Neural Networks (GNNs) have achieved significant success in learning better representations by performing feature propagation and transformation iteratively to leverage neighborhood information. Nevertheless, iterative propagation restricts the information of higher-layer neighborhoods to be transported through and fused with the lower-layer neighborhoods', which unavoidably results in feature smoothing between neighborhoods in different layers and can thus compromise the performance, especially on heterophily networks. Furthermore, most deep GNNs only recognize the importance of higher-layer neighborhoods while yet to fully explore the importance of multi-hop dependency within the context of different layer neighborhoods in learning better representations. In this work, we first theoretically analyze the feature smoothing between neighborhoods in different layers and empirically demonstrate the variance of the homophily level across neighborhoods at different layers. Motivated by these analyses, we further propose a tree decomposition method to disentangle neighborhoods in different layers to alleviate feature smoothing among these layers. Moreover, we characterize the multi-hop dependency via graph diffusion within our tree decomposition formulation to construct Tree Decomposed Graph Neural Network (TDGNN), which can flexibly incorporate information from large receptive fields and aggregate this information utilizing the multi-hop dependency. Comprehensive experiments demonstrate the superior performance of TDGNN on both homophily and heterophily networks under a variety of node classification settings. Extensive parameter analysis highlights the ability of TDGNN to prevent over-smoothing and incorporate features from shallow layers with deeper multi-hop dependencies, which provides new insights towards deeper graph neural networks.
Intelligent deep learning based bidirectional long short term memory model for automated reply of e-mail client prototype

E-mail is considered the commonly used and efficient way of communication over the globe. In the corporate sectors, the number of E-mails received every day is considerably high and the timely response to every E-mail is essential. Several researchers believe that natural language processing (NLP) techniques by the use of deep learning (DL) architectures have played a considerable part to reduce manual work for repeated E-mail responses and intended to develop E-mail systems with intelligent response function. In this view, this paper designs an intelligent DL enabled optimal bidirectional long short term memory (Bi-LSTM) technique for an automated E-mail reply (OBiLSTM-AER) of E-mail Client Prototype. The goal of the proposed model is to provide an automated E-mail reply solution for persons as well as corporates which receive massive identical E-mails daily. The presented model employs Glove and OBiLSTM model for feature extraction of receiving and response E-mails respectively. Finally, softmax classifier is applied to allocate the class labels. For improving the performance of the BiLSTM model, the hyperparameter tuning process takes place using an oppositional glowworm swarm optimization (OGSO) algorithm. An extensive set of simulations were performed to highlight the betterment of the proposed method and the results are examined interms of distinct measures.
Improving Query Representations for Dense Retrieval with Pseudo Relevance Feedback

Dense retrieval systems conduct first-stage retrieval using embedded representations and simple similarity metrics to match a query to documents. Its effectiveness depends on encoded embeddings to capture the semantics of queries and documents, a challenging task due to the shortness and ambiguity of search queries. This paper proposes ANCE-PRF, a new query encoder that uses pseudo relevance feedback (PRF) to improve query representations for dense retrieval. ANCE-PRF uses a BERT encoder that consumes the query and the top retrieved documents from a dense retrieval model, ANCE, and it learns to produce better query embeddings directly from relevance labels. It also keeps the document index unchanged to reduce overhead. ANCE-PRF significantly outperforms ANCE and other recent dense retrieval systems on several datasets. Analysis shows that the PRF encoder effectively captures the relevant and complementary information from PRF documents, while ignoring the noise with its learned attention mechanism.
Enabling automation and edge intelligence over resource constraint IoT devices for smart home

Smart home applications are pervasive and have gained popularity due to the overwhelming use of Internet of Things (IoT). The revolution in IoT technologies made homes more convenient, efficient and perhaps more secure. The need to advance smart home technology is necessary at this stage as IoT is abundantly used in automation industry. However, most of the proposed solutions are lacking in certain key areas of the system i.e., high interoperability, data independence, privacy, and optimization in general. The use of machine learning algorithms requires high-end hardware and are usually deployed on servers, where computation is convenient, but at the cost of bandwidth. However, more recently edge AI enabled systems are being proposed to shift the computation burden from the server side to the client side enabling smart devices. In this paper, we take advantage of the edge AI enabled technology to propose a fully featured cohesive system for smart home based on IoT and edge computing. The proposed system makes use of industry standards adopted for fog computing as well as providing robust responses from connected IoT sensors in a typical smart home. The proposed system employs edge devices as a computational platform in terms of reducing energy costs and provides security, while remotely controlling all appliances behind a secure gateway. A case study of human fall detection is evaluated by a custom lightweight deep neural network architecture implemented over the edge device of the proposed framework. The case study was validated using the Le2i dataset. During the training, the early stopping threshold was achieved with 98% accuracy for training set and 94% for validation set. The model size of the network was 6.4 MB which is significantly lower than other networks with similar performance.
Graph Convolutional Networks for 3D Object Detection on Radar Data

Despite its advantages as an inexpensive, weather-robust and long-range sensor which additionally provides velocity information, radar sensors still lead a shadowy existence compared to lidar and camera when it comes to fulfilling the requirements of fully autonomous driving. In this work, we focus on fully leveraging raw radar tensor data instead of building up on human-biased point clouds which are the typical result of traditional radar signal processing techniques. Utilizing a graph neural network on the raw radar tensor we gain a significant improvement of +10% in average precision over a grid-based convolutional baseline network. The performance of both networks is evaluated on a real world dataset with dense city traffic scenarios, diverse object orientations and distances as well as occlusions up to visually fully occluded objects. Our proposed network increases the maximum range for state-of-the-art full-3D object detection on radar data from previously 20m to 100m.
Industry 4.0-Oriented Deep Learning Models for Human Activity Recognition

According to the Industry 4.0 vision, humans in a smart factory, should be equipped with formidable and seamless communication capabilities and integrated into a cyber-physical system (CPS) that can be utilized to monitor and recognize human activity via artificial intelligence (e.g., deep learning). Recent advances in the accuracy of deep learning have contributed significantly to solving the human activity recognition issues but it remains necessary to develop high performance deep learning models that provide greater accuracy. In this paper, three models: long short-term memory (LSTM), convolutional neural network (CNN), and combined CNN-LSTM are proposed for classification of human activities. These models are applied to a dataset collected from 36 persons engaged in 6 classes of activities – downstairs, jogging, sitting, standing, upstairs, and walking. The proposed models are trained using TensorFlow framework with a hyper-parameter tuning method to achieve high accuracy. Experimentally, confusion matrices and receiver operating characteristic (ROC) curves are used to assess the performance of the proposed models. The results illustrate that the hybrid model CNN-LSTM provides a better performance than either LSTM or CNN in the classification of human activities. The CNN-LSTM model provides the best performance, with a testing accuracy of 97.76%, followed by the LSTM with a testing accuracy of 96.61%, while the CNN shows the least testing accuracy of 94.51%. The testing loss rates for the LSTM, CNN, and CNN-LSTM are 0.236, 0.232, and 0.167, respectively, while the precision, recall, F1-Measure, and the area under the ROC curves (AUCS) for the CNN-LSTM are 97.75%, 97.77%, 97.76%, and 100%, respectively.
Event-Based Motion Segmentation With Spatio-Temporal Graph Cuts

Identifying independently moving objects is an essential task for dynamic scene understanding. However, traditional cameras used in dynamic scenes may suffer from motion blur or exposure artifacts due to their sampling principle. By contrast, event-based cameras are novel bio-inspired sensors that offer advantages to overcome such limitations. They report pixel-wise intensity changes asynchronously, which enables them to acquire visual information at exactly the same rate as the scene dynamics. We develop a method to identify independently moving objects acquired with an event-based camera, that is, to solve the event-based motion segmentation problem. We cast the problem as an energy minimization one involving the fitting of multiple motion models. We jointly solve two sub-problems, namely event-cluster assignment (labeling) and motion model fitting, in an iterative manner by exploiting the structure of the input event data in the form of a spatio-temporal graph. Experiments on available datasets demonstrate the versatility of the method in scenes with different motion patterns and number of moving objects. The evaluation shows state-of-the-art results without having to predetermine the number of expected moving objects. We release the software and dataset under an open source license to foster research in the emerging topic of event-based motion segmentation.
Cost sensitive <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" display="inline" id="d1e1591" altimg="si115.svg"><mml:mi>ν</mml:mi></mml:math>-support vector machine with LINEX loss

Support vector machine (SVM) is a fundamental machine learning algorithm, while the traditional SVMs have limitations for massive and ubiquitous class-imbalanced data in practice. To address this issue, we propose a novel cost sensitive learning model called ν-CSSVM for imbalanced classification, which simultaneously inherits the advantages of ν-SVM and the asymmetric LINEX loss function. On one hand, the introduction of the LINEX loss function allocates different costs to each instance, demonstrating the high efficiency of realizing cost sensitive learning at the instance level. On the other hand, ν in the objective function is used to constrain the model sparsity. Compared with other studies lacking theoretical analysis, we provide detailed proof of ν-CSSVM generalization error bound through Rademacher complexity. Taking into full consideration of the intrinsic and potential properties including the smoothness, convexity and differentiability of ν-CSSVM, the alternating direction method of multipliers (ADMM) and gradient descent (GD) approach are designed. Considerable experiments validate that ν-CSSVM is more competitive than the benchmarks in class imbalance learning. The statistical test further confirms this conclusion.
Federated Learning and Privacy

Centralized data collection can expose individuals to privacy risks and organizations to legal risks if data is not properly managed. Federated learning is a machine learning setting where multiple entities collaborate in solving a machine learning problem, under the coordination of a central server or service provider. Each client's raw data is stored locally and not exchanged or transferred; instead, focused updates intended for immediate aggregation are used to achieve the learning objective. This article provides a brief introduction to key concepts in federated learning and analytics with an emphasis on how privacy technologies may be combined in real-world systems and how their use charts a path toward societal benefit from aggregate statistics in new domains and with minimized risk to individuals and to the organizations who are custodians of the data.
Representation Learning via Quantum Neural Tangent Kernels

Variational quantum circuits are used in quantum machine learning and variational quantum simulation tasks. Designing good variational circuits or predicting how well they perform for given learning or optimization tasks is still unclear. Here we discuss these problems, analyzing variational quantum circuits using the theory of neural tangent kernels. We define quantum neural tangent kernels, and derive dynamical equations for their associated loss function in optimization and learning tasks. We analytically solve the dynamics in the frozen limit, or lazy training regime, where variational angles change slowly and a linear perturbation is good enough. We extend the analysis to a dynamical setting, including quadratic corrections in the variational angles. We then consider hybrid quantum-classical architecture and define a large-width limit for hybrid kernels, showing that a hybrid quantum-classical neural network can be approximately Gaussian. The results presented here show limits for which analytical understandings of the training dynamics for variational quantum circuits, used for quantum machine learning and optimization problems, are possible. These analytical results are supported by numerical simulations of quantum machine learning experiments.
An ACO for energy-efficient and traffic-aware virtual machine placement in cloud computing

This paper formulates a virtual machine placement (VMP) problem, where the total power consumption of physical machines (PMs) and switches and the total network bandwidth resource consumption among VMs are jointly minimized. To address the problem, we present an energy- and traffic-aware ant colony optimization (ETA-ACO) algorithm. Three novel schemes are introduced to enhance the performance of ETA-ACO, including an energy- and bandwidth-aware PM selection scheme, a traffic-based VM ordering scheme, and a direct information exchange scheme. The first scheme consists of two steps when selecting a PM to host a given VM. In the first step, PMs with lower power consumption are preserved. In the second step, the one with the lowest bandwidth resource consumption is chosen to host the VM. In the second scheme, ETA-ACO places VMs in descending order by their traffic demands. The third scheme constructs new solutions by spreading the components of the best solution over a group of constructed solutions. Simulation results demonstrate that the three novel schemes are effective in adapting ETA-ACO for the VMP problem. Besides, ETA-ACO outperforms a number of state-of-the-art heuristics and metaheuristics in terms of solution quality.
Cracks in the Success Narrative: Rethinking Failure in Design Research through a Retrospective Trioethnography

What can design researchers learn from our own and each other's failures? We explore “failure” expansively—turning away from tidy success narratives toward messy unfoldings and reflexive discomfort—through retrospective trioethnography. Our findings reflect on failures we identified in six past design research projects: issues of relational labor of deployment, mismatched designer/participant imaginaries, burden of participation, and invisibility of researcher labor. Our discussion contributes to broader reflections on shifting design research practice: (a) methodological considerations inviting others to engage failures through retrospective trioethnography, (b) letting go as a mode of research care, (c) possibilities for more candid research reporting, and (d) how centering failure may contribute to design justice by providing a technique for attending to harm and healing in design research practices. Throughout, we call for challenging success narratives in design research, and underscore the need for systemic changes in design research practice.
A clinical specific BERT developed using a huge Japanese clinical text corpus

Generalized language models that are pre-trained with a large corpus have achieved great performance on natural language tasks. While many pre-trained transformers for English are published, few models are available for Japanese text, especially in clinical medicine. In this work, we demonstrate the development of a clinical specific BERT model with a huge amount of Japanese clinical text and evaluate it on the NTCIR-13 MedWeb that has fake Twitter messages regarding medical concerns with eight labels. Approximately 120 million clinical texts stored at the University of Tokyo Hospital were used as our dataset. The BERT-base was pre-trained using the entire dataset and a vocabulary including 25,000 tokens. The pre-training was almost saturated at about 4 epochs, and the accuracies of Masked-LM and Next Sentence Prediction were 0.773 and 0.975, respectively. The developed BERT did not show significantly higher performance on the MedWeb task than the other BERT models that were pre-trained with Japanese Wikipedia text. The advantage of pre-training on clinical text may become apparent in more complex tasks on actual clinical text, and such an evaluation set needs to be developed.
Compact and Malicious Private Set Intersection for Small Sets

We describe a protocol for two-party private set intersection (PSI) based on Diffie-Hellman key agreement. The protocol is proven secure against malicious parties, in the ideal permutation + random oracle model.
Small perturbations are enough: Adversarial attacks on time series prediction

Time-series data are widespread in real-world industrial scenarios. To recover and infer missing information in real-world applications, the problem of time-series prediction has been widely studied as a classical research topic in data mining. Deep learning architectures have been viewed as next-generation time-series prediction models. However, recent studies have shown that deep learning models are vulnerable to adversarial attacks. In this study, we prospectively examine the problem of time-series prediction adversarial attacks and propose an attack strategy for generating an adversarial time series by adding malicious perturbations to the original time series to deteriorate the performance of time-series prediction models. Specifically, a perturbation-based adversarial example generation algorithm is proposed using the gradient information of the prediction model. In practice, unlike the imperceptibility to humans in the field of image processing, time-series data are more sensitive to abnormal perturbations and there are more stringent requirements regarding the amount of perturbations. To address this challenge, we craft an adversarial time series based on the importance measurement to slightly perturb the original data. Based on comprehensive experiments conducted on real-world time-series datasets, we verify that the proposed adversarial attack methods not only effectively fool the target time-series prediction model LSTNet, they also attack state-of-the-art CNN-, RNN-, and MHANET-based models. Meanwhile, the results show that the proposed methods achieve a good transferability. That is, the adversarial examples generated for a specific prediction model can significantly affect the performance of the other methods. Moreover, through a comparison with existing adversarial attack approaches, we can see that much smaller perturbations are sufficient for the proposed importance-measurement based adversarial attack method. The methods described in this paper are significant in understanding the impact of adversarial attacks on a time-series prediction and promoting the robustness of such prediction technologies.
EXPLAINABILITY OF NEURAL NETWORK CLUSTERING IN INTERPRETING THE COVID-19 EMERGENCY DATA

Among other hospitalization causes and cases, the clinical emergency is a critical case and the data of the reporting patients are biased as well as poorly managed due to the chaotic situation. The world has faced chaos over the past year due to the frequent waves of COVID-19 and the resulting emergencies. The data banks, linked with the clinical emergencies require serious quantitative and qualitative analysis to drive interpretable conclusions for necessary future emergency measures and to develop explainable artificial intelligence tools. This important procedure involves the clear understanding of the data patterns and topologies, which is a great challenge for the multidimensional data sets. Mathematically, the topological mapping can resolve this problem by mapping higher-dimensional data to two-dimensional representation, based on the overall association. Proper data mining and pattern recognition can help in improving the rapid patients admission, in providing the medical resources timely and in proper patient administration. In this paper, the importance of self-organizing maps, to interpret the hospital data, particularly for the COVID-19 epidemic is discussed in detail. Important variables are identified with the aid of networks and mappings.
Dynamical analysis of the EIP-1559 Ethereum fee market

Participation in permissionless blockchains results in competition over system resources, which needs to be controlled with fees. Until recently, Ethereum's fee mechanism was implemented via a first-price auction that resulted in unpredictable fees as well as other inefficiencies. Launched on August 5, 2021, EIP-1559 is an improved proposal that introduces a number of innovative features such as a dynamically adaptive basefee that is burnt, instead of being paid to the miners. Despite intense interest in understanding its properties, several basic questions such as whether and under what conditions does this protocol self-stabilize have remained elusive thus far.