Reviewerly/exHarmony huggingface dataset:
Graph Neural Networks Exponentially Lose Expressive Power for Node Classification

Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of graph NNs via their asymptotic behaviors as the layer size tends to infinity. Our strategy is to generalize the forward propagation of a Graph Convolutional Network (GCN), which is a popular graph NN variant, as a specific dynamical system. In the case of a GCN, we show that when its weights satisfy the conditions determined by the spectra of the (augmented) normalized Laplacian, its output exponentially approaches the set of signals that carry information of the connected components and node degrees only for distinguishing nodes. Our theory enables us to relate the expressive power of GCNs with the topological information of the underlying graphs inherent in the graph spectra. To demonstrate this, we characterize the asymptotic behavior of GCNs on the Erd\H{o}s -- R\'{e}nyi graph. We show that when the Erd\H{o}s -- R\'{e}nyi graph is sufficiently dense and large, a broad range of GCNs on it suffers from the ``information loss in the limit of infinite layers with high probability. Based on the theory, we provide principled guidelines for weight normalization of graph NNs. We experimentally confirm that the proposed weight scaling enhances the predictive performance of GCNs in real data.
An Incentive Mechanism for Federated Learning in Wireless Cellular Networks: An Auction Approach

Federated Learning (FL) is a distributed learning framework that can deal with the distributed issue in machine learning and still guarantee high learning performance. However, it is impractical that all users will sacrifice their resources to join the FL algorithm. This motivates us to study the incentive mechanism design for FL. In this paper, we consider a FL system that involves one base station (BS) and multiple mobile users. The mobile users use their own data to train the local machine learning model, and then send the trained models to the BS, which generates the initial model, collects local models and constructs the global model. Then, we formulate the incentive mechanism between the BS and mobile users as an auction game where the BS is an auctioneer and the mobile users are the sellers. In the proposed game, each mobile user submits its bids according to the minimal energy cost that the mobile users experiences in participating in FL. To decide winners in the auction and maximize social welfare, we propose the primal-dual greedy auction mechanism. The proposed mechanism can guarantee three economic properties, namely, truthfulness, individual rationality and efficiency. Finally, numerical results are shown to demonstrate the performance effectiveness of our proposed mechanism.
D3R-Net: Dynamic Routing Residue Recurrent Network for Video Rain Removal

In this paper, we address the problem of video rain removal by considering rain occlusion regions, i.e., very low light transmittance for rain streaks. Different from additive rain streaks, in such occlusion regions, the details of backgrounds are completely lost. Therefore, we propose a hybrid rain model to depict both rain streaks and occlusions. Integrating the hybrid model and useful motion segmentation context information, we present a Dynamic Routing Residue Recurrent Network (D3R-Net). D3R-Net first extracts the spatial features by a residual network. Then, the spatial features are aggregated by recurrent units along the temporal axis. In the temporal fusion, the context information is embedded into the network in a “dynamic routing” way. A heap of recurrent units takes responsibility for handling the temporal fusion in given contexts, e.g., rain or non-rain regions. In the certain forward and backward processes, one of these recurrent units is mainly activated. Then, a context selection gate is employed to detect the context and select one of these temporally fused features generated by these recurrent units as the final fused feature. Finally, this last feature plays a role of “residual feature.” It is combined with the spatial feature and then used to reconstruct the negative rain streaks. In such a D3R-Net, we incorporate motion segmentation, which denotes whether a pixel belongs to fast moving edges or not, and rain type indicator, indicating whether a pixel belongs to rain streaks, rain occlusions, and non-rain regions, as the context variables. Extensive experiments on a series of synthetic and real videos with rain streaks verify not only the superiority of the proposed method over state of the art but also the effectiveness of our network design and its each component.
Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models

Pathological estimation of tumor necrosis after chemotherapy is essential for patients with osteosarcoma. This study reports the first fully automated tool to assess viable and necrotic tumor in osteosarcoma, employing advances in histopathology digitization and automated learning. We selected 40 digitized whole slide images representing the heterogeneity of osteosarcoma and chemotherapy response. With the goal of labeling the diverse regions of the digitized tissue into viable tumor, necrotic tumor, and non-tumor, we trained 13 machine-learning models and selected the top performing one (a Support Vector Machine) based on reported accuracy. We also developed a deep-learning architecture and trained it on the same data set. We computed the receiver-operator characteristic for discrimination of non-tumor from tumor followed by conditional discrimination of necrotic from viable tumor and found our models performing exceptionally well. We then used the trained models to identify regions of interest on image-tiles generated from test whole slide images. The classification output is visualized as a tumor-prediction map, displaying the extent of viable and necrotic tumor in the slide image. Thus, we lay the foundation for a complete tumor assessment pipeline from original histology images to tumor-prediction map generation. The proposed pipeline can also be adopted for other types of tumor.
Federated Visual Classification with Real-World Data Distribution

Federated Learning enables visual models to be trained on-device, bringing advantages for user privacy (data need never leave the device), but challenges in terms of data diversity and quality. Whilst typical models in the datacenter are trained using data that are independent and identically distributed (IID), data at source are typically far from IID. Furthermore, differing quantities of data are typically available at each device (imbalance). In this work, we characterize the effect these real-world data distributions have on distributed learning, using as a benchmark the standard Federated Averaging (FedAvg) algorithm. To do so, we introduce two new large-scale datasets for species and landmark classification, with realistic per-user data splits that simulate real-world edge learning scenarios. We also develop two new algorithms (FedVC, FedIR) that intelligently resample and reweight over the client pool, bringing large improvements in accuracy and stability in training. The datasets are made available online.
Invertible Denoising Network: A Light Solution for Real Noise Removal

Invertible networks have various benefits for image de-noising since they are lightweight, information-lossless, and memory-saving during back-propagation. However, applying invertible models to remove noise is challenging because the input is noisy, and the reversed output is clean, following two different distributions. We propose an invertible denoising network, InvDN, to address this challenge. InvDN transforms the noisy input into a low-resolution clean image and a latent representation containing noise. To discard noise and restore the clean image, InvDN replaces the noisy latent representation with another one sampled from a prior distribution during reversion. The de-noising performance of InvDN is better than all the existing competitive models, achieving a new state-of-the-art result for the SIDD dataset while enjoying less run time. Moreover, the size of InvDN is far smaller, only having 4.2% of the number of parameters compared to the most recently proposed DANet. Further, via manipulating the noisy latent representation, InvDN is also able to generate noise more similar to the original one. Our code is available at: https://github.com/Yang-Liu1082/InvDN.git.
Covert Spoofing Algorithm of UAV Based on GPS/INS-Integrated Navigation

A covert spoofing algorithm of an unmanned aerial vehicle (UAV) based on GPS/inertial-navigation-system-integrated navigation is analyzed in this paper. The proposed algorithm theoretically proves the fact that when the acceleration component of the counterfeit GPS signal is the difference between the UAV's current acceleration and the spoofing control input, the UAV can be covert spoofed. To avoid adverse consequences (detection or crash) caused by frequent changes in UAV flight during the GPS spoofing attacks, the proposed algorithm requires that the deception trajectory planned by the GPS spoofer is slowly changing relative to the reference trajectory, which is pre-set by the UAV. Simulation results have verified the correctness of the proposed covert spoofing algorithm of UAV. Moreover, the spoofing effect is more distinct when the deception trajectory planning is considered.
Identifying hypernyms in distributional semantic spaces

In this paper we apply existing directional similarity measures to identify hypernyms with a state-of-the-art distributional semantic model. We also propose a new directional measure that achieves the best performance in hypernym identification.
Real-time anomaly detection based on long short-Term memory and Gaussian Mixture Model

Anomaly detection is a long-standing problem in system designation. High-quality anomaly detection can benefit plenty of applications (e.g. system monitoring, disaster precaution and intrusion detection). Most of the existing anomalies detection algorithms are less competent for both effectiveness and real-time capability requirements simultaneously. Therefore, in this paper, the LGMAD, a real-time anomaly detection algorithm based on Long-Short Term Memory (LSTM) and Gaussian Mixture Model (GMM)is proposed. Specifically, we evaluate the real-time anomalies of each univariate sensing time-series via LSTM model, and then a Gaussian Mixture Model is adopted to give a multidimensional joint detection of possible anomalies. Both NAB dataset and self-made dataset are employed to verify our approach. Extensive experiments are conducted to demonstrate the superiority of LGMAD compared to existing anomaly detection algorithms.
Blockchain for Giving Patients Control Over Their Medical Records

Personal health records (PHRs) are valuable assets to individuals because they enable them to integrate and manage their medical data. A PHR is an electronic application through which patients can manage their health information. Giving patients control over their medical data offers an advantageous realignment of the doctor-patient dynamic. However, today's PHR management systems fall short of giving reliable, traceable, trustful, and secure patients control over their medical data, which poses serious threats to their authenticity and accuracy. Moreover, most of the current approaches and systems leveraged for managing PHR are centralized that not only make medical data sharing difficult but also pose a risk of single point of failure problem. In this paper, we propose Ethereum blockchain-based smart contracts to give patients control over their data in a manner that is decentralized, immutable, transparent, traceable, trustful, and secure. The proposed system employs decentralized storage of interplanetary file systems (IPFS) and trusted reputation-based re-encryption oracles to securely fetch, store, and share patients' medical data. We present algorithms along with their full implementation details. We evaluate the proposed smart contracts using two important performance metrics, such as cost and correctness. Furthermore, we provide security analysis and discuss the generalization aspects of our solution. We outline the limitations of the proposed approach. We make the smart contract source code publicly available on Github.
Video-Based Vehicle Counting Framework

The continuous development in the construction of transportation infrastructure has brought enormous pressure to traffic control. Accurate and detailed traffic flow information is valuable for an effective traffic control strategy. This paper proposes a video-based vehicle counting framework using a three-component process of object detection, object tracking, and trajectory processing to obtain the traffic flow information. First, a dataset for vehicle object detection (VDD) and a standard dataset for verifying the vehicle counting results (VCD) were established. The object detection was then completed by deep learning with VDD. Using this detection, a matching algorithm was designed to perform multi-object tracking in combination with a traditional tracking method. Trajectories of the moving objects were obtained using this approach. Finally, a trajectory counting algorithm based on encoding is proposed. The vehicles were counted according to the vehicle categories and their moving route to obtain detailed traffic flow information. The results demonstrated that the overall accuracy of our method for vehicle counting can reach more than 90%. The running rate of the proposed framework is 20.7 frames/s on the VCD. Therefore, the proposed vehicle counting framework is capable of acquiring reliable traffic flow information, which is likely applicable to intelligent traffic control and dynamic signal timing.
Cooperative Training of Descriptor and Generator Networks

This paper studies the cooperative training of two generative models for image modeling and synthesis. Both models are parametrized by convolutional neural networks (ConvNets). The first model is a deep energy-based model, whose energy function is defined by a bottom-up ConvNet, which maps the observed image to the energy. We call it the descriptor network. The second model is a generator network, which is a non-linear version of factor analysis. It is defined by a top-down ConvNet, which maps the latent factors to the observed image. The maximum likelihood learning algorithms of both models involve MCMC sampling such as Langevin dynamics. We observe that the two learning algorithms can be seamlessly interwoven into a cooperative learning algorithm that can train both models simultaneously. Specifically, within each iteration of the cooperative learning algorithm, the generator model generates initial synthesized examples to initialize a finite-step MCMC that samples and trains the energy-based descriptor model. After that, the generator model learns from how the MCMC changes its synthesized examples. That is, the descriptor model teaches the generator model by MCMC, so that the generator model accumulates the MCMC transitions and reproduces them by direct ancestral sampling. We call this scheme MCMC teaching. We show that the cooperative algorithm can learn highly realistic generative models.
IoT network slicing on virtual layers of homogeneous data for improved algorithm operation in smart buildings

With its strong coverage, low energy consumption, low cost and great connectivity, the Internet of Things technology has become the key technology in smart cities. However, faced with a large number of terminals, the rational allocation of limited resources, the topology and non-uniformity of smart buildings, the fusion of heterogeneous data become important trends in Internet of Things research. As a result, this paper proposes a novel technique for processing heterogeneous temperature data collected by an IoT network in a smart building and transforms them into homogeneous data that can be used as an input for monitoring and control algorithms in smart buildings, optimizing their performance. The proposed technique, called IoT slicing, combines complex networks and clusters in order to reduce algorithm input errors and improve the monitoring and control of a smart building. For validating the efficiency of the algorithm, it is proposed as a case study using the IoT slicing technique to improve the operation of an algorithm to self-correct outliers in data collected by IoT networks. The results of the case study confirm, irrefutably, the effectiveness of the proposed method.
Unsupervised Specific Emitter Identification Method Using Radio-Frequency Fingerprint Embedded InfoGAN

Machine learning approaches are becoming increasingly popular to improve the efficiency of specific emitter identification (SEI). However, in most non-cooperative SEI scenarios, supervised and semi-supervised learning approaches are often incompatible due to the lack of labeled datasets. To solve this challenge, an unsupervised SEI framework is proposed based on information maximized generative adversarial networks (InfoGANs) and radio frequency fingerprint embedding (RFFE). To enhance individual discriminability, a gray histogram is first constructed according to the bispectrum extracted from the received signal before being embedded into the proposed framework. In addition to the latent class input and the RFFE, the proposed InfoGAN incorporates a priori statistical characteristics of the wireless propagation channels in the form of a structured multimodal latent vector to further improve the GAN quality. The probabilistic distribution of the bispectrum is derived in closed-form and the convergence of the InfoGAN is analyzed to demonstrate the influence of the RFFE. Numerical results indicate that the proposed framework consistently outperforms state-of-the-art algorithms for unsupervised SEI applications, both in terms of evaluation score and classification accuracy.
Strong mixed-integer programming formulations for trained neural networks

We present strong mixed-integer programming (MIP) formulations for high-dimensional piecewise linear functions that correspond to trained neural networks. These formulations can be used for a number of important tasks, such as verifying that an image classification network is robust to adversarial inputs, or solving decision problems where the objective function is a machine learning model. We present a generic framework, which may be of independent interest, that provides a way to construct sharp or ideal formulations for the maximum of d affine functions over arbitrary polyhedral input domains. We apply this result to derive MIP formulations for a number of the most popular nonlinear operations (e.g. ReLU and max pooling) that are strictly stronger than other approaches from the literature. We corroborate this computationally, showing that our formulations are able to offer substantial improvements in solve time on verification tasks for image classification networks.
Solving a class of non-convex min-max games using iterative first order methods

Recent applications that arise in machine learning have surged significant interest in solving min-max saddle point games. This problem has been extensively studied in the convex-concave regime for which a global equilibrium solution can be computed efficiently. In this paper, we study the problem in the non-convex regime and show that an $\varepsilon$--first order stationary point of the game can be computed when one of the player’s objective can be optimized to global optimality efficiently. In particular, we first consider the case where the objective of one of the players satisfies the Polyak-{\L}ojasiewicz (PL) condition. For such a game, we show that a simple multi-step gradient descent-ascent algorithm finds an $\varepsilon$--first order stationary point of the problem in $\widetilde{\mathcal{O}}(\varepsilon^{-2})$ iterations. Then we show that our framework can also be applied to the case where the objective of the ``max-player is concave. In this case, we propose a multi-step gradient descent-ascent algorithm that finds an $\varepsilon$--first order stationary point of the game in $\widetilde{\cal O}(\varepsilon^{-3.5})$ iterations, which is the best known rate in the literature. We applied our algorithm to a fair classification problem of Fashion-MNIST dataset and observed that the proposed algorithm results in smoother training and better generalization.
Bitcoin and Blockchain: Security and Privacy

Blockchain is a technology that was proposed to enable the decentralized digital currency, Bitcoin. Since its inception, blockchain has been widely used in many other areas, including tracing sensor data and mitigating its duplication in IoT applications, the healthcare industry, and e-voting. In this article, we provide a comprehensive review and analysis of the major security and privacy issues of Bitcoin and blockchain, the major challenges, and opportunities in utilizing the technology. First, we present a comprehensive background of Bitcoin and the preliminary on security. Second, the major security threats and countermeasures of Bitcoin are investigated. We analyze the risk of double-spending attacks, evaluate the probability of success in performing the attacks, and derive the profitability for the attacker to perform such attacks. Third, we analyze the underlying Bitcoin peer-to-peer network security risks and Bitcoin storage security. We compare three types of Bitcoin wallets in terms of security, types of services, and their tradeoffs. Finally, we discuss the security and privacy features of alternative cryptocurrencies and present an overview of emerging technologies today. Our results can help Bitcoin users to determine a tradeoff between the risk of double-spending attempts and the transaction time delay or confidence before accepting transactions. These results can also assist miners to develop suitable strategies to get involved in the mining process and maximize their profits.
Deep Learning-Embedded Social Internet of Things for Ambiguity-Aware Social Recommendations

With the increasing demand of users for personalized social services, social recommendation (SR) has been an important concern in academia. However, current research on SR universally faces two main challenges. On the one hand, SR lacks the considerable ability of robust online data management. On the other hand, SR fails to take the ambiguity of preference feedback into consideration. To bridge these gaps, a deep learning-embedded social Internet of Things (IoT) is proposed for ambiguity-aware SR (SIoT-SR). Specifically, a social IoT architecture is developed for social computing scenarios to guarantee reliable data management. A deep learning-based graph neural network model that can be embedded into the model is proposed as the core algorithm to perform ambiguity-aware SR. This design not only provides proper online data sensing and management but also overcomes the preference ambiguity problem in SR. To evaluate the performance of the proposed SIoT-SR, two real-world datasets are selected to establish experimental scenarios. The method is assessed using three different metrics, selecting five typical methods as benchmarks. The experimental results show that the proposed SIoT-SR performs better than the benchmark methods by at least 10% and has good robustness.