Reviewerly/exHarmony huggingface dataset:
An EfficientNet-Based Weighted Ensemble Model for Industrial Machine Malfunction Detection Using Acoustic Signals

Detecting and preventing industrial machine failures are significant in the modern manufacturing industry because machine failures substantially increase both maintenance and manufacturing costs.Recently, state-of-the-art deep learning techniques that use acoustic signals have been widely applied to solve industrial machine malfunction detection problems in order to reduce maintenance and manufacturing costs.The authors of this research propose a deep learning-based industrial machine malfunction detection model that uses acoustic signals to classify normal and abnormal conditions of industrial machines.In particular, a weighted ensemble model based on EfficientNet-B0, B5, and B7 is considered to improve classification performance.Case studies involving an open dataset for Malfunctioning Industrial Machine Investigation and Inspection (MIMII) validate that the proposed EfficientNet-based weighted ensemble model provides better classification performance than individual classifiers and other ensemble models.
FPGA-Based Acceleration for Bayesian Convolutional Neural Networks

Neural networks (NNs) have demonstrated their potential in a variety of domains ranging from computer vision (CV) to natural language processing. Among various NNs, two-dimensional (2-D) and three-dimensional (3-D) convolutional NNs (CNNs) have been widely adopted for a broad spectrum of applications, such as image classification and video recognition, due to their excellent capabilities in extracting 2-D and 3-D features. However, standard 2-D and 3-D CNNs are not able to capture their model uncertainty which is crucial for many safety-critical applications, including healthcare and autonomous driving. In contrast, Bayesian CNNs (BayesCNNs), as a variant of CNNs, have demonstrated their ability to express uncertainty in their prediction via a mathematical grounding. Nevertheless, BayesCNNs have not been widely used in industrial practice due to their compute requirements stemming from sampling and subsequent forward passes through the whole network multiple times. As a result, these requirements significantly increase the amount of computation and memory consumption in comparison to standard CNNs. This article proposes a novel field-programmable gate array (FPGA)-based hardware architecture to accelerate both 2-D and 3-D BayesCNNs based on Monte Carlo dropout (MCD). Compared with other state-of-the-art accelerators for BayesCNNs, the proposed design can achieve up to four times higher energy efficiency and nine times better compute efficiency. An automatic framework capable of supporting partial Bayesian inference is proposed to explore the tradeoff between algorithm and hardware performance. Extensive experiments are conducted to demonstrate that our framework can effectively find the optimal implementations in the design space.
Cross-modal image retrieval with deep mutual information maximization

In this paper, we study the cross-modal image retrieval, where the inputs contain a source image plus some text that describes certain modifications to this image and the desired image. Prior work usually uses a three-stage strategy to tackle this task: 1) extracting the features of the inputs; 2) fusing the features of the source image and its modified text to obtain the fusion feature; 3) learning a similarity metric between the desired image and the source image plus modified text via deep metric learning. Since classical image/text encoders can learn useful representations and common pair-based loss functions of distance metric learning are enough for cross-modal retrieval, people usually improve retrieval accuracy by designing new fusion networks. However, these methods do not successfully handle the modality gap caused by the inconsistent feature distributions of different modalities, which greatly influences the feature fusion and the similarity learning. To alleviate this problem, we apply the contrastive self-supervised learning method Deep InfoMax (DIM) [1] to our approach to bridge this gap by enhancing the dependence between the text, the image, and their fusion. Specifically, our method narrows the modality gap between the text modality and the image modality by maximizing mutual information between their semantically inconsistent representations. Moreover, we seek an effective common subspace for the semantically consistent features of the fusion and the desired images by utilizing Deep InfoMax between the low-level layer of the image encoder and the high-level layer of the fusion network. Extensive experiments on three large-scale benchmarks show that we have bridged the modality gap between different modalities and achieve the state-of-the-art retrieval performance.
Deterministic Time-Bin Entanglement between a Single Photon and an Atomic Ensemble

Hybrid matter-photon entanglement is the building block for quantum networks. It is very favorable if the entanglement can be prepared with a high probability. In this Letter, we report the deterministic creation of entanglement between an atomic ensemble and a single photon by harnessing the Rydberg blockade. We design a scheme that creates entanglement between a single photon's temporal modes and the Rydberg levels that host a collective excitation, using a process of cyclical retrieving and patching. The hybrid entanglement is tested via retrieving the atomic excitation as a second photon and performing correlation measurements, which suggest an entanglement fidelity of 87.8%. Our source of matter-photon entanglement will enable the entangling of remote quantum memories with much higher efficiency.
Interpretable Mining of Influential Patterns from Sparse Web

Big data are everywhere. World Wide Web is an example of these big data. It has become a vast data production and consumption platform, at which threads of data evolve from multiple devices, by different human interactions, over worldwide locations, under divergent distributed settings. Embedded in these big web data is implicit, previously unknown and potentially useful information and knowledge that awaited to be discovered. This calls for web intelligence solutions, which make good use of data science and data mining (especially, web mining) to discover useful knowledge and important information from the web. As a web mining task, web structure mining aims to examine incoming and outgoing links on web pages and make recommendations of frequently referenced web pages to web surfers. As another web mining task, web usage mining aims to examine web surfer patterns and make recommendations of frequently visited pages to web surfers. While the size of the web is huge, the connection among all web pages may be sparse. In other words, the number of vertex nodes (i.e., web pages) on the web is huge, the number of directed edges (i.e., incoming and outgoing hyperlinks between web pages) may be small. This leads to a sparse web. In this paper, we present a solution for interpretable mining of influential patterns from sparse web. In particular, we represent web structure and usage information by bitmaps to capture connections to web pages. Due to the sparsity of the web, we compress the bitmaps, and use them in mining influential patterns (e.g., popular web pages). For explainability of the mining process, we ensure the compressed bitmaps are interpretable. Evaluation on real-life web data demonstrates the effectiveness, interpretability and practicality of our solution for interpretable mining of influential patterns from sparse web.
FedOComp: Two-Timescale Online Gradient Compression for Over-the-Air Federated Learning

Federated learning (FL) is a machine learning framework, where multiple distributed edge Internet of Things (IoT) devices collaboratively train a model under the orchestration of a central server while keeping the training data distributed on the IoT devices. FL can mitigate the privacy risks and costs from data collection in traditional centralized machine learning. However, the deployment of standard FL is hindered by the expense of the communication of the gradients from the devices to the server. Hence, many gradient compression methods have been proposed to reduce the communication cost. However, the existing methods ignore the structural correlations of the gradients and, therefore, lead to a large compression loss which will decelerate the training convergence. Moreover, many of the existing compression schemes do not enable over-the-air aggregation and, hence, require huge communication resources. In this work, we propose a gradient compression scheme, named FedOComp, which leverages the correlations of the stochastic gradients in FL systems for efficient compression of the high-dimension gradients with over-the-air aggregation. The proposed design can achieve a smaller deceleration of the training convergence compared to other gradient compression methods since the compression kernel exploits the structural correlations of the gradients. It also directly enables over-the-air aggregation to save communication resources. The derived convergence analysis and simulation results further illustrate that under the same power cost, the proposed scheme has a much faster convergence rate and higher test accuracy compared to existing baselines.
RemoteCoDe: Robotic Embodiment for Enhancing Peripheral Awareness in Remote Collaboration Tasks

Collaborative design activities are often centered around physical artifacts. Depending on the design activity, this can be the model of a building, paper crafts, carving artwork, or a new circuit to be debugged and evaluated. In a typical setting, collaborators are seated around a table and divide their attention between the design artifact under review, at least one laptop supporting measurements and information foraging, and of course their collaborators. Although these activities involve complex sets of tools and configurations, people can easily work together when they are present in the same space. This is because the physical presence of a partner affords peripheral awareness to inform where the partner's attention is and what they are doing. This peripheral awareness allows collaborators to coordinate actions and manage coupling to achieve a shared task. For example, it is quite easy to know when your partner switches their focus from a breadboard to you as a request to start a face to face discussion.
Blockchain-Based Self-Sovereign Identity: Survey, Requirements, Use-Cases, and Comparative Study

Identity is at the heart of digital transformation. Successful digital transformation requires confidence in and protection of digital identities. On the Internet, however, there is no unique and standard identity layer. Consequently, a variety of digital identities have emerged over years, leading to privacy risks, security vulnerabilities, risks for identity owners, and liability for identity issuers and those relying on digital identities to grant access to goods and services. Self-Sovereign Identity (SSI) and similar forms of identity management on the blockchain distributed ledger are novel technologies that recognize the need to keep user identity privately stored in user-owned devices, securely verified by identity issuers, and only revealed to verifiers as needed. There is limited academic literature defining the prerequisite SSI functional and non-functional requirements and comparing SSI technologies. Often those SSI technologies reviewed in the literature lack behind current advances. We present the first work that compiles a comprehensive list of functional and non-functional requirements of SSI and compares an extensive number of existing SSI/blockchain-based identity management solutions with respect to these requirements. Our work sheds light on the state-of-the-art SSI development and paves the way for future, more informed analysis and development of novel identity management and SSI solutions.
Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation

The performance of multilingual pretrained models is highly dependent on the availability of monolingual or parallel text present in a target language.Thus, the majority of the world's languages cannot benefit from recent progress in NLP as they have no or limited textual data.To expand possibilities of using NLP technology in these under-represented languages, we systematically study strategies that relax the reliance on conventional language resources through the use of bilingual lexicons, an alternative resource with much better language coverage.We analyze different strategies to synthesize textual or labeled data using lexicons, and how this data can be combined with monolingual or parallel text when available.For 19 under-represented languages across 3 tasks, our methods lead to consistent improvements of up to 5 and 15 points with and without extra monolingual text respectively.Overall, our study highlights how NLP methods can be adapted to thousands more languages that are under-served by current technology. 1
Effective and efficient structure learning with pruning and model averaging strategies

Learning the structure of a Bayesian Network (BN) with score-based solutions involves exploring the search space of possible graphs and moving towards the graph that maximises a given objective function. Some algorithms offer exact solutions that guarantee to return the graph with the highest objective score, while others offer approximate solutions in exchange for reduced computational complexity. This paper describes an approximate BN structure learning algorithm, which we call Model Averaging Hill-Climbing (MAHC), that combines two novel strategies with hill-climbing search. The algorithm starts by pruning the search space of graphs, where the pruning strategy can be viewed as an aggressive version of the pruning strategies that are typically applied to combinatorial optimisation structure learning problems. It then performs model averaging in the hill-climbing search process and moves to the neighbouring graph that maximises the objective function, on average, for that neighbouring graph and over all its valid neighbouring graphs. Comparisons with other algorithms spanning different classes of learning suggest that the combination of aggressive pruning with model averaging is both effective and efficient, particularly in the presence of data noise.
Computational Efficiency of Multi-Step Learning Echo State Networks for Nonlinear Time Series Prediction

The echo state network (ESN) is a representative model for reservoir computing, which has been mainly used for temporal pattern recognition. Recent studies have shown that multi-reservoir ESN models constructed with multiple reservoirs can enhance the potential of the ESN-based approach. In the present study, we investigate computational performance and efficiency of the <i>multi-step learning ESN</i> which is one of the multi-reservoir ESN models and characterized by step-by-step learning processes. We show that the time complexity of the training algorithm of the multi-step learning ESN is equal to or smaller than that of the standard ESN. Our numerical experiments demonstrate that the multi-step learning ESN can achieve better or comparable performance with much less computational time compared to the standard ESN in nonlinear time series prediction tasks. Moreover, we reveal how the model architecture of the multi-step learning ESN is effective in comparison with other possible variant models. The step-by-step learning is applicable to general multi-reservoir systems and hardware for enhancement of their computational ability and efficiency.
M-FasterSeg: An efficient semantic segmentation network based on neural architecture search

Image semantic segmentation is one of the key technologies for intelligent systems to understand natural scenes. As one of the important research directions in the field of visual intelligence, this technology has a wide range of application scenarios in the fields of mobile robots, drones, and intelligent driving. However, in practical applications, there may be problems such as inaccurate prediction of semantic labels, loss of segmented objects and background edge information. This paper proposes an improved semantic segmentation network that combines self-attention module and neural architecture search (NAS) method. The method first uses the NAS method to find a semantic segmentation network with multiple resolution branches. During the search process, the searched network structure is adjusted by combining the self-attention module, and then combined with the semantic segmentation networks searched by different branches to integrate into two semantic segmentation network models with different complexity, and finally integrate two network models with different complexity according to the current general teacher–student framework. The input image will first pass through the high complexity model to obtain more accurate parameters, which will affect the training weight of the student network, then pass the image into the low-complexity model to get the final predicted result. The experimental results on the Cityscapes dataset show that the accuracy of the algorithm is 69.8 %, the inference speed is 166.4 FPS, and the actual image segmentation speed is 48/s. It can optimize edge segmentation for better performance in complex scenes and achieve a good balance between real-time performance and accuracy in practical applications.
PokeBNN: A Binary Pursuit of Lightweight Accuracy

Optimization of Top-1 ImageNet promotes enormous networks that may be impractical in inference settings. Binary neural networks (BNNs) have the potential to significantly lower the compute intensity but existing models suffer from low quality. To overcome this deficiency, we propose Poke- Conv, a binary convolution block which improves quality of BNNs by techniques such as adding multiple residual paths, and tuning the activation function. We apply it to ResNet-50 and optimize ResNet's initial convolutional layer which is hard to binarize. We name the resulting network family PokeBNN <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Poke/pnki/is pronounced similarly to pocket. PokeConv, PokeBNN, and Pokemon are abbreviations of Pocket Convolution, Pocket Binary Neural Network, and Pocket Monster, respectively.. These techniques are chosen to yield favorable improvements in both top-1 accuracy and the network's cost. In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardware- and energy-inspired cost metric for quantized and binarized networks. We also identify a need to optimize an under-explored hyper-parameter controlling the binarization gradient approximation. We establish a new, strong state-of-the-art (SOTA) on top-1 accuracy together with commonly-used CPU64 cost, ACE cost and network size metrics. ReActNet-Adam [33], the previous SOTA in BNNs, achieved a 70.5% top-1 accuracy with 7.9 ACE. A small variant of PokeBNN achieves 70.5% top-1 with 2.6 ACE, more than 3x reduction in cost; a larger PokeBNN achieves 75.6% top-1 with 7.8 ACE, more than 5% improvement in accuracy without increasing the cost. PokeBNN implementation in JAX/Flax [6, 18] and re-production instructions are open sourced. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> Source code and reproduction instructions are available in AQT repos-itory: github.com/google/aqt.
Metrics for Saliency Map Evaluation of Deep Learning Explanation Methods

Due to the black-box nature of deep learning models, there is a recent development of solutions for visual explanations of CNNs. Given the high cost of user studies, metrics are necessary to compare and evaluate these different methods. In this paper, we critically analyze the Deletion Area Under Curve (DAUC) and Insertion Area Under Curve (IAUC) metrics proposed by Petsiuk et al. (2018). These metrics were designed to evaluate the faithfulness of saliency maps generated by generic methods such as Grad-CAM or RISE. First, we show that the actual saliency score values given by the saliency map are ignored as only the ranking of the scores is taken into account. This shows that these metrics are insufficient by themselves, as the visual appearance of a saliency map can change significantly without the ranking of the scores being modified. Secondly, we argue that during the computation of DAUC and IAUC, the model is presented with images that are out of the training distribution which might lead to unexpected behavior of the model being explained. To complement DAUC/IAUC, we propose new metrics that quantify the sparsity and the calibration of explanation methods, two previously unstudied properties. Finally, we give general remarks about the metrics studied in this paper and discuss how to evaluate them in a user study.
Learning to recommend method names with global context

In programming, the names for the program entities, especially for the methods, are the intuitive characteristic for understanding the functionality of the code. To ensure the readability and maintainability of the programs, method names should be named properly. Specifically, the names should be meaningful and consistent with other names used in related contexts in their codebase. In recent years, many automated approaches are proposed to suggest consistent names for methods, among which neural machine translation (NMT) based models are widely used and have achieved state-of-the-art results. However, these NMT-based models mainly focus on extracting the code-specific features from the method body or the surrounding methods, the project-specific context and documentation of the target method are ignored. We conduct a statistical analysis to explore the relationship between the method names and their contexts. Based on the statistical results, we propose GTNM, a Global Transformer-based Neural Model for method name suggestion, which considers the local context, the project-specific context, and the documentation of the method simultaneously. Experimental results on java methods show that our model can outperform the state-of-the-art results by a large margin on method name suggestion, demonstrating the effectiveness of our proposed model.