Reviewerly/exHarmony huggingface dataset:
DOTA: detect and omit weak attentions for scalable transformer acceleration

Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively.
Relevance, redundancy, and complementarity trade-off (RRCT): A principled, generic, robust feature-selection tool

We present a new heuristic feature-selection (FS) algorithm that integrates in a principled algorithmic framework the three key FS components: relevance, redundancy, and complementarity. Thus, we call it relevance, redundancy, and complementarity trade-off (RRCT). The association strength between each feature and the response and between feature pairs is quantified via an information theoretic transformation of rank correlation coefficients, and the feature complementarity is quantified using partial correlation coefficients. We empirically benchmark the performance of RRCT against 19 FS algorithms across four synthetic and eight real-world datasets in indicative challenging settings evaluating the following: (1) matching the true feature set and (2) out-of-sample performance in binary and multi-class classification problems when presenting selected features into a random forest. RRCT is very competitive in both tasks, and we tentatively make suggestions on the generalizability and application of the best-performing FS algorithms across settings where they may operate effectively.
Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation. We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4% ~165.3% improvement on cumulative performance while reducing 91.0%~99.5% unsafe configuration recommendations.
FedNI: Federated Graph Learning With Network Inpainting for Population-Based Disease Prediction

Graph Convolutional Neural Networks (GCNs) are widely used for graph analysis. Specifically, in medical applications, GCNs can be used for disease prediction on a population graph, where graph nodes represent individuals and edges represent individual similarities. However, GCNs rely on a vast amount of data, which is challenging to collect for a single medical institution. In addition, a critical challenge that most medical institutions continue to face is addressing disease prediction in isolation with incomplete data information. To address these issues, Federated Learning (FL) allows isolated local institutions to collaboratively train a global model without data sharing. In this work, we propose a framework, FedNI, to leverage network inpainting and inter-institutional data via FL. Specifically, we first federatively train missing node and edge predictor using a graph generative adversarial network (GAN) to complete the missing information of local networks. Then we train a global GCN node classifier across institutions using a federated graph learning platform. The novel design enables us to build more accurate machine learning models by leveraging federated learning and also graph learning approaches. We demonstrate that our federated model outperforms local and baseline FL methods with significant margins on two public neuroimaging datasets.
Computational Advantage from a Quantum Superposition of Qubit Gate Orders

In an ordinary quantum algorithm the gates are applied in a fixed order on the systems. The introduction of indefinite causal structures allows us to relax this constraint and control the order of the gates with an additional quantum state. It is known that this quantum-controlled ordering of gates can reduce the query complexity in deciding a property of black-box unitaries with respect to the best algorithm in which the gates are applied in a fixed order. However, all tasks explicitly found so far require unitaries that either act on unbounded dimensional quantum systems in the asymptotic limit (the limiting case of a large number of black-box gates) or act on qubits, but then involve only a few unitaries. Here we introduce tasks (i) for which there is a provable computational advantage of a quantum-controlled ordering of gates in the asymptotic case and (ii) that require only qubit gates and are therefore suitable to demonstrate this advantage experimentally. We study their solutions with the quantum n-switch and within the quantum circuit model and find that while the n-switch requires to call each gate only once, a causal algorithm has to call at least 2n-1 gates. Furthermore, the best known solution with a fixed gate ordering calls O[n log_{2}(n)] gates.
Joint Resource Allocation to Minimize Execution Time of Federated Learning in Cell-Free Massive MIMO

Due to its communication efficiency and privacy-preserving capability, federated learning (FL) has emerged as a promising framework for machine learning in 5G-and-beyond wireless networks. Of great interest is the design and optimization of new wireless network structures that support the stable and fast operation of FL. Cell-free massive multiple-input–multiple-output (CFmMIMO) turns out to be a suitable candidate, which allows each communication round in the iterative FL process to be stably executed within a large-scale coherence time. Aiming to reduce the total execution time of the FL process in CFmMIMO, this article proposes choosing only a subset of available users to participate in FL. An optimal selection of users with favorable link conditions would minimize the execution time of each communication round while limiting the total number of communication rounds required. Toward this end, we formulate a joint optimization problem of user selection, transmit power, and processing frequency, subject to a predefined minimum number of participating users to guarantee the quality of learning. We then develop a new algorithm that is proven to converge to the neighborhood of the stationary points of the formulated problem. Numerical results confirm that our proposed approach significantly reduces the FL total execution time over baseline schemes. The time reduction is more pronounced when the density of access point deployments is moderately low.
A Comprehensive Review of Denial of Service Attacks in Blockchain Ecosystem and Open Challenges

With the proliferation of the blockchain technology ecosystems such as mining pools, crypto exchanges, full Bitcoin nodes, wallets, and pool protocol servers in recent years, the denial of service (DoS) attack vector has become more prevalent, and the attacks are targeted to the peer-to-peer networks and blockchain users. Despite blockchain enhancing security with decentralized design, secured distributed storage, and privacy, it is still vulnerable to new attack threats. If an attempted DoS is successful on blockchain, the impact is most likely massive given the fact that it is predominantly used for finance applications. An extensive account of the current state-of-the-art for possible DoS and corresponding mitigation techniques is not discussed in the existing research. This paper analyzes and categorizes the existing state-of-the-art DoS attack methods, detection techniques, and mitigation solutions targeting blockchain peer-to-peer networks as well as conventional network crypto exchanges. The review of the prior research shows that the blockchain ecosystem can be a target to successfully perform DoS attacks in the future, and technological advancements in blockchain are needed to mitigate potential attacks.
Two-Photon Excitation Sets Limit to Entangled Photon Pair Generation from Quantum Emitters

Entangled photon pairs are key to many novel applications in quantum technologies. Semiconductor quantum dots can be used as sources of on-demand, highly entangled photons. The fidelity to a fixed maximally entangled state is limited by the excitonic fine-structure splitting. This work demonstrates that, even if this splitting is absent, the degree of entanglement cannot reach unity when the excitation pulse in a two-photon resonance scheme has a finite duration. The degradation of the entanglement has its origin in a dynamically induced splitting of the exciton states caused by the laser pulse itself. Hence, in the setting explored here, the excitation process limits the achievable concurrence for entangled photons generated in an optically excited four-level quantum emitter.
Lightweight Helmet Detection Algorithm Using an Improved YOLOv4

Safety helmet wearing plays a major role in protecting the safety of workers in industry and construction, so a real-time helmet wearing detection technology is very necessary. This paper proposes an improved YOLOv4 algorithm to achieve real-time and efficient safety helmet wearing detection. The improved YOLOv4 algorithm adopts a lightweight network PP-LCNet as the backbone network and uses deepwise separable convolution to decrease the model parameters. Besides, the coordinate attention mechanism module is embedded in the three output feature layers of the backbone network to enhance the feature information, and an improved feature fusion structure is designed to fuse the target information. In terms of the loss function, we use a new SIoU loss function that fuses directional information to increase detection precision. The experimental findings demonstrate that the improved YOLOv4 algorithm achieves an accuracy of 92.98%, a model size of 41.88 M, and a detection speed of 43.23 pictures/s. Compared with the original YOLOv4, the accuracy increases by 0.52%, the model size decreases by about 83%, and the detection speed increases by 88%. Compared with other existing methods, it performs better in terms of precision and speed.
Reflectionless programmable signal routers

We demonstrate experimentally that reflectionless scattering modes (RSMs), a generalized version of coherent perfect absorption, can be functionalized to perform reflectionless programmable signal routing. We achieve versatile programmability both in terms of operating frequencies and routing functionality with negligible reflection upon in-coupling, which avoids unwanted signal power echoes in radio frequency or photonic networks. We report in situ observations of routing functionalities like wavelength demultiplexing, including cases where multichannel excitation requires adapted coherent input wavefronts. All experiments are performed in the microwave domain based on the same irregularly shaped cavity with strong modal overlap that is massively parametrized by a 304-element-programmable metasurface. RSMs in our highly overdamped multiresonance transport problem are fundamentally intriguing because the simple critical coupling picture for reflectionless excitation of isolated resonances fails spectacularly. We show in simulation that the distribution of damping rates of scattering singularities broadens under strong absorption so that weakly damped zeros can be tuned toward functionalized RSMs.
Blockchain technology for next-generation society: current trends and future opportunities for smart era

Blockchain, a relatively novel technology, is often termed as the internet of value. Even though there are predictions that indicate that the future of blockchain is likely to be perilous, it has contributed remarkably and has already brought about revolutionary changes in terms of transactions and digital currencies. This paper is a survey and analysis of one of the fast-evolving technologies called Blockchain. It covers all essential information required for a beginner to venture into this complicated field while also covering necessary concepts that can be useful for experts; the first provides a general introduction and discusses the disruptive changes initiated by blockchain, the second discusses the unique value of blockchain and its general characteristics, the third presents an overview of industries with the greatest potential for disruptive changes, the fourth describes the four major blockchain applications with the highest prospective advantages, and the fifth part of the paper ends with a discussion on the most notable subset of innovative blockchain applications - Smart Contracts, Decentralized Autonomous Organizations (DAOs), and super safe networks - and their future implications. There is also a concluding section, which summarizes the chapter, describes the future of blockchain, and mentions the challenges to be overcome.
An Artificial Neural Network-Based Finite State Machine for Adaptive Scenario Selection in Serious Game

Serious game is one of the pedagogical media capable of transferring knowledge to its players.This game genre requires a support system that adaptively selects the appropriate scenario for players to increase their interest and comfort.Therefore, this study proposed an adaptive scenario selection (ASS) system using a finite state machine based on an artificial neural network (ANN).The game scenario is selected by ASS based on five player preferences, including work, hobbies/interests, origin, group members, and repetition.Furthermore, the multi-layer perceptron (MLP) architecture was used in the scenario selection process for the proposed ANN method.The experimental stage was carried out using the theme of travel in several tourism destinations in Batu City, East Java, Indonesia.The experimental results show that ASS succeeded in generating adaptive game scenario choices for players based on their preference data with an accuracy of 67.25%.
Should You Mask 15% in Masked Language Modeling?

Masked language models (MLMs) conventionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations; this masking rate has been widely used, regardless of model sizes or masking strategies. In this work, we revisit this important choice of MLM pre-training. We first establish that 15% is not universally optimal, and larger models should adopt a higher masking rate. Specifically, we find that masking 40% outperforms 15% for BERT-large size models on GLUE and SQuAD. Interestingly, an extremely high masking rate of 80% can still preserve 95% fine-tuning performance and most of the accuracy in linguistic probing, challenging the conventional wisdom about the role of the masking rate. We then examine the interplay between masking rates and masking strategies and find that uniform masking requires a higher masking rate compared to sophisticated masking strategies such as span or PMI masking. Finally, we argue that increasing the masking rate has two distinct effects: it leads to more corruption, which makes the prediction task more difficult; it also enables more predictions, which benefits optimization. Using this framework, we revisit BERT’s 80-10-10 corruption strategy. Together, our results contribute to a better understanding of MLM pre-training.
Bayes-Adaptive Interactive POMDPs

We introduce the Bayes-Adaptive Interactive Partially Observable Markov Decision Process (BA-IPOMDP), the first multiagent decision model that explicitly incorporates model learning. As in I-POMDPs, the BA-IPOMDP agent maintains beliefs over interactive states, which include the physical states as well as the other agents’ models. The BA-IPOMDP assumes that the state transition and observation probabilities are unknown, and augments the interactive states to include these parameters. Beliefs are maintained over this augmented interactive state space. This (necessary) state expansion exacerbates the curse of dimensionality, especially since each I-POMDP belief update is already a recursive procedure (because an agent invokes belief updates from other agents’ perspectives as part of its own belief update, in order to anticipate other agents’ actions). We extend the interactive particle filter to perform approximate belief update on BA-IPOMDPs. We present our findings on the multiagent Tiger problem.
Dual-View Ranking with Hardness Assessment for Zero-Shot Learning

Zero-shot learning (ZSL) is to build recognition models for previously unseen target classes which have no labeled data for training by transferring knowledge from some other related auxiliary source classes with abundant labeled samples to the target ones with class attributes as the bridge. The key is to learn a similarity based ranking function between samples and class labels using the labeled source classes so that the proper (unseen) class label for a test sample can be identified by the function. In order to learn the function, single-view ranking based loss is widely used which aims to rank the true label prior to the other labels for a training sample. However, we argue that the ranking can be performed from the other view, which aims to place the images belonging to a label before the images from the other classes. Motivated by it, we propose a novel DuAl-view RanKing (DARK) loss for zeroshot learning simultaneously ranking labels for an image by point-to-point metric and ranking images for a label by pointto-set metric, which is capable of better modeling the relationship between images and classes. In addition, we also notice that previous ZSL approaches mostly fail to well exploit the hardness of training samples, either using only very hard ones or using all samples indiscriminately. In this work, we also introduce a sample hardness assessment method to ZSL which assigns different weights to training samples based on their hardness, which leads to a more accurate and robust ZSL model. Experiments on benchmarks demonstrate that DARK outperforms the state-of-the-arts for (generalized) ZSL.
Identifying Known and Unknown Mobile Application Traffic Using a Multilevel Classifier

Due to the proliferation of mobile applications, mobile traffic identification plays a crucial role in understanding the network traffic. However, the pervasive unconcerned apps and the emerging apps pose great challenges to the mobile traffic identification method based on supervised machine learning, since such method merely identifies and discriminates several apps of interest. In this paper we propose a three-layer classifier using machine learning to identify mobile traffic in open-world settings. The proposed method has the capability of identifying traffic generated by unconcerned apps and zero-day apps; thus it can be applied in the real world. A self-collected dataset that contains 160 apps is used to validate the proposed method. The experimental results show that our classifier achieves over 98% precision and produces a much smaller number of false positives than that of the state of the art.
Prediction of two-dimensional topography of laser cladding based on neural network

The two-dimensional morphology of the cladding layer has an important influence on the quality of the cladding layer and the crack tendency. Using the powerful nonlinear processing ability of the single hidden layer feedforward neural network, a prediction model between the cladding technological parameters and the two-dimensional morphology of the cladding layer is established. Taking the cladding parameters as the input and the two-dimensional morphology of the cladding as the output, the experimental data is used to train the network to achieve a high-level mapping of the input and output. On this basis, the algorithm of extreme learning machine is used to optimize the single hidden layer feedforward neural network to overcome the problems of slow convergence speed, more network training parameters and easy local convergence in back-propagation algorithm. The results show that the relationship between the cladding process parameters and the two-dimensional morphology of the cladding layer can be roughly reflected by the back-propagation algorithm. However, the prediction results are not stable and the error rate is between 10% and 40%. The neural network optimized by the extreme learning machine is utilized to get a better prediction result. The error rate is 10–20%.
A Distributed Cooperative Co-evolutionary CMA Evolution Strategy for Global Optimization of Large-Scale Overlapping Problems

Distributed cooperative co-evolution (DCC) is an effective way to solve large-scale optimization problems.The cooperative co-evolution methodology can reduce the optimizing complexity by dividing a large-scale problem into small subcomponents, and distributed computation can accelerate the optimizing speed.However, existing DCC algorithms often encounter deficiency in overlapping problems that cannot be ideally divided due to unavoidable overlaps between subcomponents.To address this issue, this paper proposes an algorithm called distributed cooperative co-evolutionary covariance matrix adaptation evolution strategy (DCCMAES) to solve large-scale overlapping optimization problems.First, a new grouping scheme is introduced, where a variable, if needed, can be assigned into multiple subcomponents.Second, to address the conflicts caused by optimizing multiple copies of shared variables, a random orthogonal experiment method is proposed to generate global solutions.Moreover, three cooperation schemes are proposed to coordinate the optimizers in DCCMAES during evolution.The experiments on benchmark functions and a real-world application show that DCCMAES is promising for global optimization of large-scale overlapping problems.
Deep 2D Encoder-Decoder Convolutional Neural Network for Multiple Sclerosis Lesion Segmentation in Brain MRI

In this paper, we propose an automated segmentation approach based on a deep two-dimensional fully convolutional neural network to segment brain multiple sclerosis lesions from multimodal magnetic resonance images. The proposed model is made as a combination of two deep subnetworks. An encoding network extracts different feature maps at various resolutions. A decoding part upconvolves the feature maps combining them through shortcut connections during an upsampling procedure. To the best of our knowledge, the proposed model is the first slice-based fully convolutional neural network for the purpose of multiple sclerosis lesion segmentation. We evaluated our network on a freely available dataset from ISBI MS challenge with encouraging results from a clinical perspective.