Reviewerly/exHarmony huggingface dataset:
Associative Memory Based Experience Replay for Deep Reinforcement Learning

Experience replay is an essential component in deep reinforcement learning (DRL), which stores the experiences and generates experiences for the agent to learn in real time. Recently, prioritized experience replay (PER) has been proven to be powerful and widely deployed in DRL agents. However, implementing PER on traditional CPU or GPU architectures incurs significant latency overhead due to its frequent and irregular memory accesses. This paper proposes a hardware-software co-design approach to design an associative memory (AM) based PER, AMPER, with an AM-friendly priority sampling operation. AMPER replaces the widely-used time-costly tree-traversal-based priority sampling in PER while preserving the learning performance. Further, we design an in-memory computing hardware architecture based on AM to support AMPER by leveraging parallel in-memory search operations. AMPER shows comparable learning performance while achieving 55× to 270× latency improvement when running on the proposed hardware compared to the state-of-the-art PER running on GPU.
Pathwise CVA regressions with oversimulated defaults

Abstract We consider the computation by simulation and neural net regression of conditional expectations, or more general elicitable statistics, of functionals of processes . Here an exogenous component Y (Markov by itself) is time‐consuming to simulate, while the endogenous component X (jointly Markov with Y ) is quick to simulate given Y , but is responsible for most of the variance of the simulated payoff. To address the related variance issue, we introduce a conditionally independent, hierarchical simulation scheme, where several paths of X are simulated for each simulated path of Y . We analyze the statistical convergence of the regression learning scheme based on such block‐dependent data. We derive heuristics on the number of paths of Y and, for each of them, of X , that should be simulated. The resulting algorithm is implemented on a graphics processing unit (GPU) combining Python/CUDA and learning with PyTorch. A CVA case study with a nested Monte Carlo benchmark shows that the hierarchical simulation technique is key to the success of the learning approach.
Large-Scale Image Retrieval with Deep Attentive Global Features

How to obtain discriminative features has proved to be a core problem for image retrieval. Many recent works use convolutional neural networks to extract features. However, clutter and occlusion will interfere with the distinguishability of features when using convolutional neural network (CNN) for feature extraction. To address this problem, we intend to obtain high-response activations in the feature map based on the attention mechanism. We propose two attention modules, a spatial attention module and a channel attention module. For the spatial attention module, we first capture the global information and model the relation between channels as a region evaluator, which evaluates and assigns new weights to local features. For the channel attention module, we use a vector with trainable parameters to weight the importance of each feature map. The two attention modules are cascaded to adjust the weight distribution for the feature map, which makes the extracted features more discriminative. Furthermore, we present a scale and mask scheme to scale the major components and filter out the meaningless local features. This scheme can reduce the disadvantages of the various scales of the major components in images by applying multiple scale filters, and filter out the redundant features with the MAX-Mask. Exhaustive experiments demonstrate that the two attention modules are complementary to improve performance, and our network with the three modules outperforms the state-of-the-art methods on four well-known image retrieval datasets.
Shakespearean Magnitudes

The most conspicuous characteristic of The Chronicle History of Henry the fift is its brevity. The first edition of Shakespeare’s play, a quarto printed in 1600, contains only 12,272 words of dialogue. By comparison, The Comedy of Errors—the shortest of Shakespeare’s canonical texts—has 14,369 spoken words.1 The shortest playtext of undisputed Shakespearean authorship contains 17 percent more words of dialogue than The Chronicle History. The brevity of that quarto is even more remarkable if we take genre into account. The shortest of Shakespeare’s canonical “History” texts, King John, contains 20,386 words of dialogue: 66 percent more than The Chronicle History. Statistically, there is only one chance in 500,000 that The Chronicle History belongs to the same set as Shakespeare’s ten Folio histories, and only one chance in 250 that it belongs to the same set as the thirty-six canonical playtexts.2 Most significantly, the text of The Life of Henry the Fift published in the 1623 Folio—reproduced in all subsequent editions of Shakespeare’s works—contains 25,577 words of dialogue: more than double (208 percent) the number in the quarto. This contrast in magnitude between the two versions of Henry V is greater than any other gap between two versions of a Shakespeare play.3 In this disparity between its two texts, Henry V is unique. But The Life is not: it is not the longest of Shakespeare’s texts (the second quarto of Hamlet), not even the longest of Shakespeare’s English history plays (Folio Richard III). Within Shakespeare’s dramatic universe, the singularity of Henry V depends on the unparalleled brevity of the quarto. Why is this text uniquely brief?
Evaluation of the Learning Effect of VR on Engineering Education – Case Study in Machine Elements

Like the development of new products, the training of new engineers is also subject to constant change. The transfer of knowledge is increasingly taking place digitally, for example with the help of virtual reality. The research work presented here investigated the learning effect of using VR-supported teaching methods in engineering education. For this purpose, a VR-supported teaching scenario was designed and tested by a student test group. The learning effect was then checked with the help of a survey. It was shown that VR makes it possible to link theoretical knowledge and practical application. VR is therefore a good alternative to purely online teaching, but cannot replace face-to-face teaching. Future research activities should answer the question of how existing didactic methods can be implemented well and efficiently in VR.
Differentially private transferrable deep learning with membership-mappings

Despite a recent surge of research interest in privacy and transferrable deep learning, optimizing the tradeoff between privacy requirements and performance of machine learning models remains a challenge. This motivates the development of an approach that optimizes both privacy-preservation mechanism and learning of the deep models for achieving a robust performance. This paper considers the problem of semi-supervised transfer and multi-task learning under differential privacy framework. An alternative conception of deep autoencoder, referred to as Conditionally Deep Membership-Mapping Autoencoder (CDMMA), is considered for transferrable deep learning. Under practice-oriented settings, an analytical solution for the learning of CDMMA can be derived by means of variational optimization. The paper proposes a transfer and multi-task learning approach that combines CDMMA with a tailored noise adding mechanism to transfer knowledge from source to target domain in a privacy-preserving manner.
Blockchain technology for protection of biomedical documents in healthcare society

Blockchain is around a decade-old technology that initially came with Bitcoin to protect digital monetary transactions with consensus and mining incentives. Later on, blockchain technology is adopted by different domains, including banking, healthcare and governance, to mention a few. The Healthcare industry is massive in a world where data security is indispensable. Since biomedical research documents are content-sensitive with evidence-based research, it is essential to see data integrity and non-repudiation besides other security attributes. Blockchain is the technology found suitable for achieving the desired level of security for biomedical documents. This paper investigates the present state-of-the-art on the application of Blockchain to biomedical databases. Our contribution is to analyse and compare existing methods with a tabulated study. Based on specific investigations, we conclude that biomedical databases are protected through blockchain technology, making those data more reliable and helping to provide effective diagnosis and treatment to the patients in the hospital.
Explicit Occlusion Reasoning for Multi-person 3D Human Pose Estimation

Occlusion poses a great threat to monocular multi-person 3D human pose estimation due to large variability in terms of the shape, appearance, and position of occluders. While existing methods try to handle occlusion with pose priors/constraints, data augmentation, or implicit reasoning, they still fail to generalize to unseen poses or occlusion cases and may make large mistakes when multiple people are present. Inspired by the remarkable ability of humans to infer occluded joints from visible cues, we develop a method to explicitly model this process that significantly improves bottom-up multi-person human pose estimation with or without occlusions. First, we split the task into two subtasks: visible keypoints detection and occluded keypoints reasoning, and propose a Deeply Supervised Encoder Distillation (DSED) network to solve the second one. To train our model, we propose a Skeleton-guided human Shape Fitting (SSF) approach to generate pseudo occlusion labels on the existing datasets, enabling explicit occlusion reasoning. Experiments show that explicitly learning from occlusions improves human pose estimation. In addition, exploiting feature-level information of visible joints allows us to reason about occluded joints more accurately. Our method outperforms both the state-of-the-art top-down and bottom-up methods on several benchmarks. The code is available for research purposes https://github.com/qihao067/HUPOR .
Medication recommendation system for online pharmacy using an adaptive user interface

This article proposes a prototype of a user-adaptive system for helping patients to obtain their ambulatory prescribed medications when purchasing online in a more convenient manner than traditional methods, and the adoption of artificial intelligence to achieve improvements. The system developed simulates an online pharmacy with an introductory adaptive user interface using Bayesian user modeling for predicting the medication needs of patients. This program is used to show its step-by-step design and functioning. The introductory adaptive user interface was developed on Visual C++ of Microsoft Visual Studio. The patient model acquisition and application implementing the learning and inference was performed with a Bayesian Network. The Bayesian network was elaborated with the GeNIe Modeler software, Version 2.3.R4, provided by BayesFusion, LLC. Synthetic data from a synthetically generated dataset of anonymous patients was used. The performance of the system was evaluated through simulations using testing data from the synthetic dataset. The Accuracy of predictions was analyzed. The Average accuracy was estimated with the average correct recommendations of medications, for different numbers of purchased medications per session. The Average accuracy increased with the number of purchased medications, from 86.3529% up to 92.5303%. The Average wrong recommendations decreased with the increase in the number of purchased medications, from an average of 3.4117 up to 1.5686. The system quickly and consistently attained high accuracy in predicting the medication categories needed by the patients, potentially being able to save time and effort for the patients by relying on the system's recommendations.
Electricity and Carbon-aware Task Scheduling in Geo-distributed Internet Data Centers

Besides the enormous power consumption, carbon emission from data centers is also a concern for cloud service providers (CSPs). Based on cloud computing technology, the spatial-temporal transfer of workloads between geographically distributed internet data centers (IDCs) gives CSPs the potential to reduce their carbon footprint. However, workloads in real-world production are highly heterogeneous, which vary in resource demand, duration et al. Without ignoring the workload heterogeneity, we proposed an integer linear programming formulation of the task scheduling problem for geo-distributed IDCs. Extensive numerical studies are performed using real task information, electricity price, and marginal emission rate. Results show that CSP can better meet emission reduction demand and optimize its electricity expenses by spatial-temporal task scheduling.
Anomaly Detection Using Hybrid Learning for Industrial IoT

The industrial internet of things (IIoT) enhances industrial and manufacturing operations by using smart sensors and actuators. However, it is hampered due to the energy efficiency requirements, real time performance requirements in a dynamic environment, and maintaining the security of applications. Security is a serious issue nowadays and is mostly caused by abnormal traffic of some nodes. For detecting those abnormalities, there are two basic machine learning approaches, namely Federated and Centralized Learning. Centralized Learning has better performance, but it has a privacy issue since edge devices send data to the server. On the other hand, Federate Learning obviates privacy issues, but it has less performance due to the resource limitation of edge devices. In this study, a typical hybrid learning based abnormality detection framework has been proposed in which edge devices undertake Federated Learning with a limited number of datasets and the edge server will use the periodically collected aggregated data from edge devices. For security reasons, edge devices share their data after a certain period of time when the time value of the data has declined. We have used Long Short Term Memory (LSTM) Autoencoders with two different datasets (a smaller for edge devices and a larger for the edge server) for the demonstration. The experimental result shows that the size of the dataset affects the predicting performance and resource utilization in an anomaly detection model.
Hardly Perceptible Trojan Attack Against Neural Networks with Bit Flips

The security of deep neural networks (DNNs) has attracted increasing attention due to their widespread use in various applications. Recently, the deployed DNNs have been demonstrated to be vulnerable to Trojan attacks, which manipulate model parameters with bit flips to inject a hidden behavior and activate it by a specific trigger pattern. However, all existing Trojan attacks adopt noticeable patch-based triggers (e.g., a square pattern), making them perceptible to humans and easy to be spotted by machines. In this paper, we present a novel attack, namely hardly perceptible Trojan attack (HPT). HPT crafts hardly perceptible Trojan images by utilizing the additive noise and per-pixel flow field to tweak the pixel values and positions of the original images, respectively. To achieve superior attack performance, we propose to jointly optimize bit flips, additive noise, and flow field. Since the weight bits of the DNNs are binary, this problem is very hard to be solved. We handle the binary constraint with equivalent replacement and provide an effective optimization algorithm. Extensive experiments on CIFAR-10, SVHN, and ImageNet datasets show that the proposed HPT can generate hardly perceptible Trojan images, while achieving comparable or better attack performance compared to the state-of-the-art methods. The code is available at: https://github.com/jiawangbai/HPT .
Deep API learning revisited

Understanding the correct API usage sequences is one of the most important tasks for programmers when they work with unfamiliar libraries. However, programmers often encounter obstacles to finding the appropriate information due to either poor quality of API documentation or ineffective query-based searching strategy. To help solve this issue, researchers have proposed various methods to suggest the sequence of APIs given natural language queries representing the information needs from programmers. Among such efforts, Gu et al. adopted a deep learning method, in particular an RNN Encoder-Decoder architecture, to perform this task and obtained promising results on common APIs in Java. In this work, we aim to reproduce their results and apply the same methods for APIs in Python. Additionally, we compare the performance with a more recent Transformer-based method, i.e., CodeBERT, for the same task. Our experiment reveals a clear drop in performance measures when careful data cleaning is performed. Owing to the pretraining from a large number of source code files and effective encoding technique, CodeBERT outperforms the method by Gu et al., to a large extent.
Identifying the Effects of Immersion on Design for Additive Manufacturing Evaluation of Designs of Varying Manufacturability

Abstract The demand for additive manufacturing (AM) continues to grow as more industries look to integrate the technology into their product development. However, there is a deficit of designers skilled to innovate with this technology due to challenges in supporting designers with tools and education for their development in design for AM (DfAM). There is a need to introduce intuitive tools and knowledge to enable future designers to DfAM. Immersive virtual reality (VR) shows promise to serve as an intuitive tool for DfAM to aid designers during design evaluation. The goal of this research is to, therefore, identify the effects of immersion in design evaluation and study how evaluating designs for DfAM between mediums that vary in immersion, affects the results of the DfAM evaluation and the mental effort experienced from evaluating the designs. Our findings suggest that designers can use immersive and non-immersive mediums for DfAM evaluation without experiencing significant differences in the outcomes of the evaluation and the cognitive load experienced from conducting the evaluation. The findings from this work thus have implications for how industries can customize product and designer-talent development using modular design evaluation systems that leverage capabilities in immersive and non-immersive DfAM evaluation.
The Mediating Effect of Dynamic Capabilities on the Relationship Between Strategic Foresight and Strategic Renewal: Evidence from Islamic Banks in Jordan

Purpose: The present study aimed to examine the mediating effect that dynamic capabilities may have on the relationship between strategic foresight and strategic renewal. Design/methodology/ approach: The present study relied on quantitative and qualitative methods. More specifically, the authors used a cross-sectional design and survey strategy. Three hundred and three participants were selected randomly from four Islamic banks operating in Jordan completed the study questionnaire. A structural equation modeling technique was employed to test the study hypotheses by using SmartPLS 3 software package. Findings: The results demonstrated that strategic foresight and dynamic capabilities have a significant and positive effect on strategic renewal. Also, the results illustrated that the dynamic capabilities have a partial mediating effect on the relationship between strategic foresight and strategic renewal. Research limitations: The results of the present study relate to Islamic banks in Jordan; hence the ability to generalize the findings to other sectors is limited. Another limitation is respondent bias because the study design relies heavily on the participants’ perceptions. Practical implementation: Used properly, strategic foresight allows managers to see how events in the external environment influence their organizations’ performance. Thus, managers can use strategic foresight for making corrections in plans, policies, strategies, and objectives to get performance back on track using their organizations’ dynamic capabilities. Originality: Despite the amount of research and studies that have been devoted to understanding strategic foresight, dynamic capabilities, and strategic renewal over the last decades, the relationship between these three constructs has never been studied collectively. Therefore, the present study fills this gap in the literature.
Chinese Electronic Medical Record Named Entity Recognition based on BERT-WWM-IDCNN-CRF

Named entity recognition is a basic work of natural language processing. Aiming at the disadvantages of traditional BERT pre-training, such as random MASK of single word, the large number of unregistered words and the phenomenon of multiple meanings of one word in electronic medical record, the BERT model of Whole Word Masking and the network model of dilated convolutional network (IDCNN) were designed. In the word embedding layer, the BERT pre-training model of whole word covering is introduced to enhance the semantic representation of word vector. Then, IDCNN is used to extract sentence features to the maximum extent. Finally, conditional random field (CRF) is used to output tag sequences with the maximum probability. The experimental results show that this model is more suitable for Chinese electronic medical record entity recognition, and its recognition accuracy, recall rate and F1 value are all above 91%, which are better than the traditional method.
AdaBest: Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation

In Federated Learning (FL), a number of clients or devices collaborate to train a model without sharing their data. Models are optimized locally at each client and further communicated to a central hub for aggregation. While FL is an appealing decentralized training paradigm, heterogeneity among data from different clients can cause the local optimization to drift away from the global objective. In order to estimate and therefore remove this drift, variance reduction techniques have been incorporated into FL optimization recently. However, these approaches inaccurately estimate the clients’ drift and ultimately fail to remove it properly. In this work, we propose an adaptive algorithm that accurately estimates drift across clients. In comparison to previous works, our approach necessitates less storage and communication bandwidth, as well as lower compute costs. Additionally, our proposed methodology induces stability by constraining the norm of estimates for client drift, making it more practical for large scale FL. Experimental findings demonstrate that the proposed algorithm converges significantly faster and achieves higher accuracy than the baselines across various FL benchmarks.