Reviewerly/exHarmony huggingface dataset:
Analysis of Model Compression Using Knowledge Distillation

In the development of deep learning, several convolution neural network (CNN) models are designed to solve various tasks.However, these CNN models are complex and cumbersome to achieve state-of-the-art performance.The current CNN models remain to suffer from the problem of large models.Thus, model compression techniques are proposed to cope with the complex CNN models.Meanwhile, the selection of compressed model to suit the user requirement significantly contributes during the deployment process.This paper analyses two model compressions, namely the layerwise and the widthwise compression.The compression techniques are implemented in the MobileNetV1 model.Then, knowledge distillation is applied to compensate for the accuracy loss of the compressed model.We demonstrate the analysis of those compressed models from various perspectives and develop several suggestions on the trade-off between the performance and the compression rate.In addition, we also show that the feature that is learned by the compressed models using knowledge distillation has better representation compared to the vanilla model.Our experiment shows that the widthwise compression on MobileNetV1 achieves a compression rate of 42.27% and the layerwise compression achieves 32.42%, respectively.Furthermore, the improvement of the compressed models using knowledge distillation is notable for the widthwise compression with the increasing accuracy above 4.71%.
PEGANs: Phased Evolutionary Generative Adversarial Networks with Self-Attention Module

Generative adversarial networks have made remarkable achievements in generative tasks. However, instability and mode collapse are still frequent problems. We improve the framework of evolutionary generative adversarial networks (E-GANs), calling it phased evolutionary generative adversarial networks (PEGANs), and adopt a self-attention module to improve upon the disadvantages of convolutional operations. During the training process, the discriminator will play against multiple generators simultaneously, where each generator adopts a different objective function as a mutation operation. Every time after the specified number of training iterations, the generator individuals will be evaluated and the best performing generator offspring will be retained for the next round of evolution. Based on this, the generator can continuously adjust the training strategy during training, and the self-attention module also enables the model to obtain the modeling ability of long-range dependencies. Experiments on two datasets showed that PEGANs improve the training stability and are competitive in generating high-quality samples.
Has the pandemic made us more digitally literate?

Digital literacy has been included in the set of the eight key competences, which are necessary to enjoy life to the full in the twenty-first century. According to the previous studies, women tend to possess lower digital competence than men; the older the person, the lower the level of digital literacy. To date, Polish citizens in general have worse skills than the European average. This may lead to people being socially excluded and vulnerable to cybersecurity threats, especially in the times of the COVID-19 pandemic, which requires them to work, study and shop using the Internet. The study concerned Polish women who work at universities, as scientists and teachers. Their perceived level of their digital literacy has been studied in the broad campaign, along with their awareness of the cybersecurity matters. Then, the collected results were processed with an association rules mining algorithm, uncovering the factors related to the shifts in them.
Robust local-coordinate non-negative matrix factorization with adaptive graph for robust clustering

• A new robust adaptive local structure learning strategy. • A new orthogonal regularization term ensures the orthogonality of the factor matrix . • A new algorithmic framework optimizes the model caused by robust loss function. With its unique geometric properties, non-negative matrix factorization (NMF) has become one of the widely used clustering methods in the field of data mining. Regrettably, most existing NMF methods are sensitive to super-noise (super-outliers). This paper proposes a novel robust clustering method to address this issue. Based on the Hx loss function, this method establishes a novel robust adaptive local structure learning strategy, reducing the interference of noise (outliers) on data reconstruction and space exploration. In addition, a new orthogonal regularization term is incorporated into the model, ensuring the orthogonality of the factor matrix and enhancing the discriminant ability. Finally, we develop an efficient algorithm to solve the resultant model and analyze its convergence from theoretical and experimental aspects. Experimental results on random synthetic data sets and benchmark databases demonstrate that the proposed method outperforms the existing robust NMF methods in terms of spatial structure learning, discriminant power, and robustness.
Abnormality classification and localization using dual-branch whole-region-based CNN model with histopathological images

The task of classification and localization with detecting abnormalities in medical images is considered very challenging. Computer-aided systems have been widely employed to address this issue, and the proliferation of deep learning network architectures is proof of the outstanding performance reported in the literature. However, localizing abnormalities in regions of images that can support the confidence of classification continues to attract research interest. The difficulty of using digital histopathology images for this task is another drawback, which needs high-level deep learning models to address the situation. Successful pathology localization automation will support automatic acquisition planning and post-imaging analysis. In this paper, we address issues related to the combination of classification with image localization and detection through a dual branch deep learning framework that uses two different configurations of convolutional neural networks (CNN) architectures. Whole-image based CNN (WCNN) and region-based CNN (RCNN) architectures are systematically combined to classify and localize abnormalities in samples. A multi-class classification and localization of abnormalities are achieved using the method with no annotation-dependent images. In addition, seamless confidence and explanation mechanism is provided so that outcomes from WCNN and RCNN are mapped together for further analysis. Using images from both BACH and BreakHis databases, an exhaustive set of experiments was carried out to validate the performance of the proposed method in achieving classification and localization simultaneously. Obtained results showed that the system achieved a classification accuracy of 97.08%, a localization accuracy of 94%, and an area under the curve (AUC) of 0.10 for classification. Further findings from this study revealed that a multi-neural network approach could provide a suitable method for addressing the combinatorial problem of classification and localization anomalies in digital medical images. Lastly, the study's outcome offers means for automating the annotation of histopathology images and the support for human pathologists in locating abnormalities.
FedWalk

Node embedding aims to map nodes in the complex graph into low-dimensional representations. The real-world large-scale graphs and difficulties of labeling motivate wide studies of unsupervised node embedding problems. Nevertheless, previous effort mostly operates in a centralized setting where a complete graph is given. With the growing awareness of data privacy, data holders who can be represented by one vertex in the graph and are only its neighbors demand greater privacy protection. In this paper, we introduce FedWalk, a random-walk-based unsupervised node embedding algorithm that operates in such a node-level visibility graph with raw graph information remaining locally. FedWalk is designed to offer centralized competitive graph representation capability with data privacy protection and great communication efficiency. FedWalk instantiates the prevalent federated paradigm and contains three modules. We first design a hierarchical clustering tree (HCT) constructor to extract the structural feature of each node. A dynamic time warping algorithm seamlessly handles the structural heterogeneity across different nodes. Based on the constructed HCT, we then design a random walk generator, wherein a sequence encoder is designed to preserve privacy and a two-hop neighbor predictor is designed to save communication cost. The generated random walks are then used to update node embedding based on a SkipGram model. Extensive experiments on two large graphs demonstrate that FedWalk achieves competitive representativeness as a centralized node embedding algorithm does with only up to 1.8% Micro-F1 score and 4.4% Marco-F1 score loss while reducing about 6.7 times of inter-device communication per walk.
PECOS

Different from traditional machine learning tasks and benchmarks, real-world problems are usually accompanied by enormous output spaces, from hundred thousands of diseases in medical diagnosis, to millions of items and billions of websites in product and web search engines. Unfortunately, conventional machine learning tools and libraries are incapable of efficiently and accurately tackling large-scale output spaces. To address this issue, PECOS (Prediction for Enormous and Correlated Output Spaces) [11] is a state-of-the-art and open-sourced machine learning library1, which not only provides high-level and user-friendly interfaces of both linear and deep learning models, but also supplies considerable flexibility for solving diverse machine learning problems. Specifically, PECOS eases complicated semantic indexing for organizing enormous output spaces, thereby efficiently training models and deriving predictions by magnitude orders on correlated output labels. As a powerful and useful framework, PECOS has already been adopted in various real- world large-scale products like semantic search in Amazon [1], as well as achieved state-of-the-art on public extreme multi-label classification (XMC) benchmarks [2, 11, 12 ] and various downstream applications [3, 7, 9].
4SDrug: Symptom-based Set-to-set Small and Safe Drug Recommendation

Drug recommendation is an important task of AI for healthcare. To recommend proper drugs, existing methods rely on various clinical records (e.g., diagnosis and procedures), which are commonly found in data such as electronic health records (EHRs). However, detailed records as such are often not available and the inputs might merely include a set of symptoms provided by doctors. Moreover, existing drug recommender systems usually treat drugs as individual items, ignoring the unique requirements that drug recommendation has to be done on a set of items (drugs), which should be as small as possible and safe without harmful drug-drug interactions (DDIs).
A Blockchain-based platform for data management and sharing

Data management is the collection, processing, storing, and sharing of data. In today's dispensation, data sharing and collaborative data processing is a necessity for multi-partner organizations as it can lead to the discovery of new insight. Shared data is generally for the purpose of marketing, advertising, and other institutional decision-making reasons. A challenge however is that the collected data (mostly about individuals known as data subjects or producers) is disseminated among organizations without meaningful consent from the data subjects. Hence, data subjects are not aware of what is happening to their data regarding use and misuse. Furthermore, data subjects can hardly determine which third-party institutions have access to their data. In this paper, we opined that data should be managed in a manner that can persuade the trust of data subjects. To achieve this, data subjects should have the rights to be informed about details of their data. Thus, this paper proposes a cloud-based data management and sharing platform to enable data subjects to control who can access their data and consent to its collection and usage based on smart contracts. The proposed system leverages blockchain to enforce accountability, provenance, and auditability of all events. We implemented a dynamic consent management prototype on top of Ethereum blockchain to demonstrate the feasibility of the proposed work.
MIMO-GAN: Generative MIMO Channel Modeling

We propose generative channel modeling to learn statistical channel models from channel input-output measurements. Generative channel models can learn more complicated distributions and represent the field data more faithfully. They are tractable and easy to sample from, which can potentially speed up the simulation rounds. To achieve this, we leverage advances in generative adversarial network (GAN), which helps us learn an implicit distribution over stochastic MIMO channels from observed measurements. In particular, our approach MIMO-GAN implicitly models the wireless channel as a distribution of time-domain band-limited impulse responses. We evaluate MIMO-GAN on 3GPP TDL MIMO channels and observe high-consistency in capturing power, delay and spatial correlation statistics of the underlying channel. In particular, we observe MIMO-GAN achieve errors of under 3.57 ns average delay and -18.7 dB power.
Multi-cell Non-coherent Over-the-Air Computation for Federated Edge Learning

In this paper, we propose a framework where over-the-air computation (OAC) occurs in both uplink (UL) and downlink (DL), sequentially, in a multi-cell environment to address the latency and the scalability issues of federated edge learning (FEEL). To eliminate the channel state information (CSI) at the edge devices (EDs) and edge servers (ESs) and relax the time-synchronization requirement for the OAC, we use a non-coherent computation scheme, i.e., frequency-shift keying (FSK)-based majority vote (MV) (FSK-MV). With the proposed framework, multiple ESs function as the aggregation nodes in the UL and each ES determines the MVs independently. After the ESs broadcast the detected MVs, the EDs determine the sign of the gradient through another OAC in the DL. Hence, inter-cell interference is exploited for the OAC. In this study, we prove the convergence of the non-convex optimization problem for the FEEL with the proposed OAC framework. We also numerically evaluate the efficacy of the proposed method by comparing the test accuracy in both multi-cell and single-cell scenarios for both homogeneous and heterogeneous data distributions.
Cryptocurrencies and the IPE of money: an agenda for research

Despite the exploding interest in cryptocurrencies in the markets in recent years, the political economy study of them, in particular as money, remains underdeveloped. This seems quite surprising, given the otherwise extensive research on money and finance in the international political economy (IPE) literature. This essay calls for a greater engagement of IPE scholarship in analyzing cryptocurrencies as money. It argues first that theories of money—in particular, the “commodity theory of money” and the “state theory of money”—could provide valuable theoretical insights for the IPE study of cryptocurrencies as money. It then suggests how these insights can be incorporated into that research, drawing particular attention to analysis of the political dynamics surrounding cryptocurrencies’ development as money.
VLSG-SANet: A feature matching algorithm for remote sensing image registration

Finding a reliable correspondence between two feature point sets efficiently and accurately is the first step in many computer vision applications. It is a crucial prerequisite for many remote sensings, target recognition, and photogrammetry applications. However, in different application scenarios, images may undergo different degrees of deformation, which makes the predefined geometric model no longer appropriate. This article attempts to remove mismatches between the set of assumed matching points. In order to achieve this goal, we propose an effective matching method, called ”visualized local structure generation-Siamese attention” network (VLSG-SANet), which transform the task of eliminating mismatching of feature points into a dynamic visual similarity assessment. The VLSG-SANet can generate visual descriptors by using the local structure information about each feature point. We embed the attention module based on the Siamese network for the generated feature descriptors, allowing the network to adjust the descriptors dynamically. In order to prove the robustness and versatility of the VLSG-SANet algorithm, we have performed good experiments on open remote sensing data images and got good experimental results.
Occluded pedestrian detection through bi-center prediction in anchor-free network

To handle the occlusion issue, this paper proposes a simple but effective pedestrian detector who designs a novel bi-center prediction mechanism (namely Bi-Center) based on the anchor-free network. Specifically, we modify the training loss or ground truth of the two designed center prediction branches in the pedestrian detection head. In the first branch, the negative samples around centers of occluded pedestrians, which are mainly covered by the background region, are down-weighted in a novel weighted Focal Loss according to the occlusion levels. In the second branch, we further integrate the occlusion ratio into the Gaussian mask of center ground truth to improve the predicted probability of occluded pedestrian instances. We optimize the designed detection head of the network in an end-to-end fashion, and utilize the complementary outputs of the designed two center prediction branches to boost the detection performance. The proposed bi-center prediction mechanism forces the network to pay more attention to the occluded pedestrian instances. Experimental results on the challenging CityPersons, Caltech, and CrowdHuman benchmarks sufficiently validate the efficacy of our Bi-Center detector for occlusion handling in pedestrian detection.
Recent Advances in Fully Dynamic Graph Algorithms – A Quick Reference Guide

In recent years, significant advances have been made in the design and analysis of fully dynamic algorithms. However, these theoretical results have received very little attention from the practical perspective. Few of the algorithms are implemented and tested on real datasets, and their practical potential is far from understood. Here, we present a quick reference guide to recent engineering and theory results in the area of fully dynamic graph algorithms.
Dynamic self-attention with vision synchronization networks for video question answering

Video Question Answering (VideoQA) has gained increasing attention as an important task in understanding the rich spatio-temporal contents, i.e., the appearance and motion in the video. However, existing approaches mainly use the question to learn attentions over all the sampled appearance and motion features separately, which neglect two properties of VideoQA: (1) the answer to the question is often reflected on a few frames and video clips, and most video contents are superfluous; (2) appearance and motion features are usually concomitant and complementary to each other in time series. In this paper, we propose a novel VideoQA model, i.e., Dynamic Self-Attention with Vision Synchronization Networks (DSAVS), to address these problems. Specifically, a gated token selection mechanism is proposed to dynamically select the important tokens from appearance and motion sequences. These chosen tokens are fed into a self-attention mechanism to model the internal dependencies for more effective representation learning. To capture the correlation between the appearance and motion features, a vision synchronization block is proposed to synchronize the two types of vision features at the time slice level. Then, the visual objects can be correlated with their corresponding activities and the performance is further improved. Extensive experiments conducted on three public VideoQA data sets confirm the effectivity and superiority of our model compared with state-of-the-art methods.
Asynchronous federated learning system for human–robot touch interaction

Artificial intelligence and robotics are advancing at an incredible pace; however, there is a risk associated with the data privacy and personal information of users interacting with these systems and platforms. In this context, the federated learning approach emerged to enable large-scale, distributed learning without the need to transmit or store any information necessary to train the learning models. In a previous paper, we presented a system capable of detecting, locating, and classifying what kind of contact occurs between humans and one of our robots using innovative contact microphone technology. In this work we go further, improving the previously presented touch system with a multi-user, multi-robot, distributed, and scalable learning approach that is able to learn in a collaborative and incremental way while respecting the privacy of the user’s information. The system has been successfully evaluated in a real environment with 28 different users divided in 7 different groups. To assess the performance of our system with this federated learning approach, we compared it to the same distributed learning system without federated learning. That is, the control group for this comparison is a central node directly receiving all the training examples obtained by each robot locally. We found that in this context the inclusion of federated learning improves the results concerning traditional distributed learning.
A spatial attention guided deep learning system for prediction of pathological complete response using breast cancer histopathology images

Predicting pathological complete response (pCR) to neoadjuvant chemotherapy (NAC) in triple-negative breast cancer (TNBC) patients accurately is direly needed for clinical decision making. pCR is also regarded as a strong predictor of overall survival. In this work, we propose a deep learning system to predict pCR to NAC based on serial pathology images stained with hematoxylin and eosin and two immunohistochemical biomarkers (Ki67 and PHH3). To support human prior domain knowledge-based guidance and enhance interpretability of the deep learning system, we introduce a human knowledge-derived spatial attention mechanism to inform deep learning models of informative tissue areas of interest. For each patient, three serial breast tumor tissue sections from biopsy blocks were sectioned, stained in three different stains and integrated. The resulting comprehensive attention information from the image triplets is used to guide our prediction system for prognostic tissue regions.The experimental dataset consists of 26 419 pathology image patches of 1000×1000 pixels from 73 TNBC patients treated with NAC. Image patches from randomly selected 43 patients are used as a training dataset and images patches from the rest 30 are used as a testing dataset. By the maximum voting from patch-level results, our proposed model achieves a 93% patient-level accuracy, outperforming baselines and other state-of-the-art systems, suggesting its high potential for clinical decision making.The codes, the documentation and example data are available on an open source at: https://github.com/jkonglab/PCR_Prediction_Serial_WSIs_biomarkers.Supplementary data are available at Bioinformatics online.