OpenRaiser/Intern-Atlas huggingface dataset:
Blockchain Aided Privacy-Preserving Outsourcing Algorithms of Bilinear Pairings for Internet of Things Devices
Venue: arXiv cs.CR 2021
Year: 2021

Bilinear pairing is a fundamental operation that is widely used in cryptographic algorithms (e.g., identity-based cryptographic algorithms) to secure IoT applications. Nonetheless, the time complexity of bilinear pairing is $O(n^3)$, making it a very time-consuming operation, especially for resource-constrained IoT devices. Secure outsourcing of bilinear pairing has been studied in recent years to enable computationally weak devices to securely outsource the bilinear pairing to untrustworthy cloud servers. However, the state-of-art algorithms often require to pre-compute and store some values, which results in storage burden for devices. In the Internet of Things, devices are generally with very limited storage capacity. Thus, the existing algorithms do not fit the IoT well. In this paper, we propose a secure outsourcing algorithm of bilinear pairings, which does not require pre-computations. In the proposed algorithm, the outsourcer side's efficiency is significantly improved compared with executing the original bilinear pairing operation. At the same time, the privacy of the input and output is ensured. Also, we apply the Ethereum blockchain in our outsourcing algorithm to enable fair payments, which ensures that the cloud server gets paid only when he correctly accomplished the outsourced work. The theoretical analysis and experimental results show that the proposed algorithm is efficient and secure.
Architectural Patterns for the Design of Federated Learning Systems
Venue: arXiv cs.LG 2021
Year: 2021

Federated learning has received fast-growing interests from academia and industry to tackle the challenges of data hungriness and privacy in machine learning. A federated learning system can be viewed as a large-scale distributed system with different components and stakeholders as numerous client devices participate in federated learning. Designing a federated learning system requires software system design thinking apart from machine learning knowledge. Although much effort has been put into federated learning from the machine learning technique aspects, the software architecture design concerns in building federated learning systems have been largely ignored. Therefore, in this paper, we present a collection of architectural patterns to deal with the design challenges of federated learning systems. Architectural patterns present reusable solutions to a commonly occurring problem within a given context during software architecture design. The presented patterns are based on the results of a systematic literature review and include three client management patterns, four model management patterns, three model training patterns, and four model aggregation patterns. The patterns are associated to the particular state transitions in a federated learning model lifecycle, serving as a guidance for effective use of the patterns in the design of federated learning systems.
Detecting Log Anomalies with Multi-Head Attention (LAMA)
Venue: arXiv cs.LG 2021
Year: 2021

Anomaly detection is a crucial and challenging subject that has been studied within diverse research areas. In this work, we explore the task of log anomaly detection (especially computer system logs and user behavior logs) by analyzing logs' sequential information. We propose LAMA, a multi-head attention based sequential model to process log streams as template activity (event) sequences. A next event prediction task is applied to train the model for anomaly detection. Extensive empirical studies demonstrate that our new model outperforms existing log anomaly detection methods including statistical and deep learning methodologies, which validate the effectiveness of our proposed method in learning sequence patterns of log data.
Feasibility Study on Virtual Process Controllers as Basis for Future Industrial Automation Systems
Venue: arXiv cs.NI 2021
Year: 2021

Industry 4.0 offers many possibilities for creating highly efficient and flexible manufacturing. To create such advantages, highly automated and thus digitized processes and systems are required. Here, most technologies known from the office floor are basically suitable for these tasks, but cannot meet the high demands of industrial use cases. Therefore, they cannot replace industrial technologies and devices that have performed well over decades "out of the box". For this reason, many technologies known from the office floor are being investigated and adapted for industrial environments. An important task is the virtualization of process controls, as more and more devices use computation offloading, e.g. due to limited resources. In this paper we extend the work on our novel architecture that enables numerous use cases and meets industrial requirements by virtualizing process controllers. In addition, a testbed based on a factory scenario is proposed to evaluate the most important features of the presented architecture.
Multimodal Gait Recognition for Neurodegenerative Diseases
Venue: arXiv cs.CV 2021
Year: 2021

In recent years, single modality based gait recognition has been extensively explored in the analysis of medical images or other sensory data, and it is recognised that each of the established approaches has different strengths and weaknesses. As an important motor symptom, gait disturbance is usually used for diagnosis and evaluation of diseases; moreover, the use of multi-modality analysis of the patient's walking pattern compensates for the one-sidedness of single modality gait recognition methods that only learn gait changes in a single measurement dimension. The fusion of multiple measurement resources has demonstrated promising performance in the identification of gait patterns associated with individual diseases. In this paper, as a useful tool, we propose a novel hybrid model to learn the gait differences between three neurodegenerative diseases, between patients with different severity levels of Parkinson's disease and between healthy individuals and patients, by fusing and aggregating data from multiple sensors. A spatial feature extractor (SFE) is applied to generating representative features of images or signals. In order to capture temporal information from the two modality data, a new correlative memory neural network (CorrMNN) architecture is designed for extracting temporal features. Afterwards, we embed a multi-switch discriminator to associate the observations with individual state estimations. Compared with several state-of-the-art techniques, our proposed framework shows more accurate classification results.
Rankings for Bipartite Tournaments via Chain Editing
Venue: arXiv cs.MA 2021
Year: 2021

Ranking the participants of a tournament has applications in voting, paired comparisons analysis, sports and other domains. In this paper we introduce bipartite tournaments, which model situations in which two different kinds of entity compete indirectly via matches against players of the opposite kind; examples include education (students/exam questions) and solo sports (golfers/courses). In particular, we look to find rankings via chain graphs, which correspond to bipartite tournaments in which the sets of adversaries defeated by the players on one side are nested with respect to set inclusion. Tournaments of this form have a natural and appealing ranking associated with them. We apply chain editing -- finding the minimum number of edge changes required to form a chain graph -- as a new mechanism for tournament ranking. The properties of these rankings are investigated in a probabilistic setting, where they arise as maximum likelihood estimators, and through the axiomatic method of social choice theory. Despite some nice properties, two problems remain: an important anonymity axiom is violated, and chain editing is NP-hard. We address both issues by relaxing the minimisation constraint in chain editing, and characterise the resulting ranking methods via a greedy approximation algorithm.
Active learning for object detection in high-resolution satellite images
Venue: arXiv cs.CV 2021
Year: 2021

In machine learning, the term active learning regroups techniques that aim at selecting the most useful data to label from a large pool of unlabelled examples. While supervised deep learning techniques have shown to be increasingly efficient on many applications, they require a huge number of labelled examples to reach operational performances. Therefore, the labelling effort linked to the creation of the datasets required is also increasing. When working on defense-related remote sensing applications, labelling can be challenging due to the large areas covered and often requires military experts who are rare and whose time is primarily dedicated to operational needs. Limiting the labelling effort is thus of utmost importance. This study aims at reviewing the most relevant active learning techniques to be used for object detection on very high resolution imagery and shows an example of the value of such techniques on a relevant operational use case: aircraft detection.
Attitudes toward Open Access, Open Peer Review, and Altmetrics among Contributors to Spanish Scholarly Journals
Venue: arXiv cs.DL 2021
Year: 2021

This paper aims to gain a better understanding of the perspectives of contributors to Spanish academic journals regarding open access, open peer review, and altmetrics. It also explores how age, gender, professional experience, career history, and perception and use of social media influence authors opinions toward these developments in scholarly publishing. A sample of contributors (n-1254) to Spanish academic journals was invited to participate in a survey about the aforementioned topics. The response rate was 24 per cent (n-295). Contributors to Spanish scholarly journals hold a favourable opinion of open access but were more cautious about open peer review and altmetrics. Younger and female scholars were more reluctant to accept open peer review practices. A positive attitude toward social networks did not necessarily translate into enthusiasm for emerging trends in scholarly publishing. Despite this, ResearchGate users were more aware of altmetrics.
Towards a Smart Data Processing and Storage Model
Venue: arXiv cs.CL 2021
Year: 2021

In several domains it is crucial to store and manipulate data whose origin needs to be completely traceable to guarantee the consistency, trustworthiness and reliability on the data itself typically for ethical and legal reasons. It is also important to guarantee that such properties are also carried further when such data is composed and processed into new data. In this article we present the main requirements and theorethical problems that arise by the design of a system supporting data with such capabilities. We present an architecture for implementing a system as well as a prototype developed in Pharo.
Adaptive Immunity for Software: Towards Autonomous Self-healing Systems
Venue: arXiv cs.SE 2021
Year: 2021

Testing and code reviews are known techniques to improve the quality and robustness of software. Unfortunately, the complexity of modern software systems makes it impossible to anticipate all possible problems that can occur at runtime, which limits what issues can be found using testing and reviews. Thus, it is of interest to consider autonomous self-healing software systems, which can automatically detect, diagnose, and contain unanticipated problems at runtime. Most research in this area has adopted a model-driven approach, where actual behavior is checked against a model specifying the intended behavior, and a controller takes action when the system behaves outside of the specification. However, it is not easy to develop these specifications, nor to keep them up-to-date as the system evolves. We pose that, with the recent advances in machine learning, such models may be learned by observing the system. Moreover, we argue that artificial immune systems (AISs) are particularly well-suited for building self-healing systems, because of their anomaly detection and diagnosis capabilities. We present the state-of-the-art in self-healing systems and in AISs, surveying some of the research directions that have been considered up to now. To help advance the state-of-the-art, we develop a research agenda for building self-healing software systems using AISs, identifying required foundations, and promising research directions.
A Clinical Evaluation of a Low-Cost Strain Gauge Respiration Belt and Machine Learning to Detect Sleep Apnea
Venue: arXiv cs.LG 2021
Year: 2021

Sleep apnea is a serious and severely under-diagnosed sleep-related respiration disorder characterized by repeated disrupted breathing events during sleep. It is diagnosed via polysomnography which is an expensive test conducted in a sleep lab requiring sleep experts to manually score the recorded data. Since the symptoms of sleep apnea are often ambiguous, it is difficult for a physician to decide whether to prescribe polysomnography. In this study, we investigate whether helpful information can be obtained by collecting and automatically analysing sleep data using a smartphone and an inexpensive strain gauge respiration belt. We evaluate how accurately we can detect sleep apnea with wide variety of machine learning techniques with data from a clinical study with 49 overnight sleep recordings. With less than one hour of training, we can distinguish between normal and apneic minutes with an accuracy, sensitivity, and specificity of 0.7609, 0.7833, and 0.7217, respectively. These results can be achieved even if we train only on high-quality data from an entirely separate, clinically certified sensor, which has the potential to substantially reduce the cost of data collection. Data from a complete night can be analyzed in about one second on a smartphone.
Data Poisoning Attacks to Deep Learning Based Recommender Systems
Venue: arXiv cs.CR 2021
Year: 2021

Recommender systems play a crucial role in helping users to find their interested information in various web services such as Amazon, YouTube, and Google News. Various recommender systems, ranging from neighborhood-based, association-rule-based, matrix-factorization-based, to deep learning based, have been developed and deployed in industry. Among them, deep learning based recommender systems become increasingly popular due to their superior performance. In this work, we conduct the first systematic study on data poisoning attacks to deep learning based recommender systems. An attacker's goal is to manipulate a recommender system such that the attacker-chosen target items are recommended to many users. To achieve this goal, our attack injects fake users with carefully crafted ratings to a recommender system. Specifically, we formulate our attack as an optimization problem, such that the injected ratings would maximize the number of normal users to whom the target items are recommended. However, it is challenging to solve the optimization problem because it is a non-convex integer programming problem. To address the challenge, we develop multiple techniques to approximately solve the optimization problem. Our experimental results on three real-world datasets, including small and large datasets, show that our attack is effective and outperforms existing attacks. Moreover, we attempt to detect fake users via statistical analysis of the rating patterns of normal and fake users. Our results show that our attack is still effective and outperforms existing attacks even if such a detector is deployed.
BRDS: An FPGA-based LSTM Accelerator with Row-Balanced Dual-Ratio Sparsification
Venue: arXiv cs.AR 2021
Year: 2021

In this paper, first, a hardware-friendly pruning algorithm for reducing energy consumption and improving the speed of Long Short-Term Memory (LSTM) neural network accelerators is presented. Next, an FPGA-based platform for efficient execution of the pruned networks based on the proposed algorithm is introduced. By considering the sensitivity of two weight matrices of the LSTM models in pruning, different sparsity ratios (i.e., dual-ratio sparsity) are applied to these weight matrices. To reduce memory accesses, a row-wise sparsity pattern is adopted. The proposed hardware architecture makes use of computation overlapping and pipelining to achieve low-power and high-speed. The effectiveness of the proposed pruning algorithm and accelerator is assessed under some benchmarks for natural language processing, binary sentiment classification, and speech recognition. Results show that, e.g., compared to a recently published work in this field, the proposed accelerator could provide up to 272% higher effective GOPS/W and the perplexity error is reduced by up to 1.4% for the PTB dataset.
Waveform and Beamforming Design for Intelligent Reflecting Surface Aided Wireless Power Transfer: Single-User and Multi-User Solutions
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, we study the waveform and passive beamforming design for intelligent reflecting surface (IRS)-aided wireless power transfer (WPT). Generalized multi-user and low complexity single-user algorithms are derived based on alternating optimization (AO) framework to maximize the weighted sum output DC current, subject to transmit power constraints and passive beamforming phases unit modulus constraints. The input signal waveform and IRS passive beamforming phase shifts are jointly designed as a function of users' individual frequency-selective channel state information (CSI). The energy harvester nonlinearity is explored and two IRS deployment schemes, namely frequency selective IRS (FS-IRS) and frequency flat IRS (FF-IRS), are modeled and analyzed. This paper highlights the fact that IRS can provide an extra passive beamforming gain on output DC power over conventional WPT designs and significantly influence the waveform design by leveraging the benefit of passive beamforming, frequency diversity and energy harvester nonlinearity. Even though FF-IRS exhibits lower output DC current than FS-IRS, it still achieves substantially increased DC power over conventional WPT designs. Performance evaluations confirm the significant benefits of a joint waveform and passive beamforming design accounting for the energy harvester nonlinearity to boost the performance of single-user and multi-user WPT system.
Does double-blind peer-review reduce bias? Evidence from a top computer science conference
Venue: arXiv cs.CY 2021
Year: 2021

Peer review is widely regarded as essential for advancing scientific research. However, reviewers may be biased by authors' prestige or other characteristics. Double-blind peer review, in which the authors' identities are masked from the reviewers, has been proposed as a way to reduce reviewer bias. Although intuitive, evidence for the effectiveness of double-blind peer review in reducing bias is limited and mixed. Here, we examine the effects of double-blind peer review on prestige bias by analyzing the peer review files of 5027 papers submitted to the International Conference on Learning Representations (ICLR), a top computer science conference that changed its reviewing policy from single-blind peer review to double-blind peer review in 2018. We find that after switching to double-blind review, the scores given to the most prestigious authors significantly decreased. However, because many of these papers were above the threshold for acceptance, the change did not affect paper acceptance decisions significantly. Nevertheless, we show that double-blind peer review may have improved the quality of the selections by limiting other (non-author-prestige) biases. Specifically, papers rejected in the single-blind format are cited more than those rejected under the double-blind format, suggesting that double-blind review better identifies poorer quality papers. Interestingly, an apparently unrelated change - the change of rating scale from 10 to 4 points - likely reduced prestige bias significantly, to an extent that affected papers' acceptance. These results provide some support for the effectiveness of double-blind review in reducing prestige bias, while opening new research directions on the impact of peer review formats.
Transfer Learning Between Different Architectures Via Weights Injection
Venue: arXiv cs.LG 2021
Year: 2021

This work presents a naive algorithm for parameter transfer between different architectures with a computationally cheap injection technique (which does not require data). The primary objective is to speed up the training of neural networks from scratch. It was found in this study that transferring knowledge from any architecture was superior to Kaiming and Xavier for initialization. In conclusion, the method presented is found to converge faster, which makes it a drop-in replacement for classical methods. The method involves: 1) matching: the layers of the pre-trained model with the targeted model; 2) injection: the tensor is transformed into a desired shape. This work provides a comparison of similarity between the current SOTA architectures (ImageNet), by utilising TLI (Transfer Learning by Injection) score.
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
Venue: arXiv cs.CV 2021
Year: 2021

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different "views" of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC.