Reviewerly/exHarmony huggingface dataset:
A Mean-field Analysis of Deep ResNet and Beyond: Towards Provable Optimization Via Overparameterization From Depth

Training deep neural networks with stochastic gradient descent (SGD) can often achieve zero training loss on real-world tasks although the optimization landscape is known to be highly non-convex. To understand the success of SGD for training deep neural networks, this work presents a mean-field analysis of deep residual networks, based on a line of works that interpret the continuum limit of the deep residual network as an ordinary differential equation when the network capacity tends to infinity. Specifically, we propose a new continuum limit of deep residual networks, which enjoys a good landscape in the sense that every local minimizer is global. This characterization enables us to derive the first global convergence result for multilayer neural networks in the mean-field regime. Furthermore, without assuming the convexity of the loss landscape, our proof relies on a zero-loss assumption at the global minimizer that can be achieved when the model shares a universal approximation property. Key to our result is the observation that a deep residual network resembles a shallow network ensemble, i.e. a two-layer network. We bound the difference between the shallow network and our ResNet model via the adjoint sensitivity method, which enables us to apply existing mean-field analyses of two-layer networks to deep networks. Furthermore, we propose several novel training schemes based on the new continuous model, including one training procedure that switches the order of the residual blocks and results in strong empirical performance on the benchmark datasets.
A new approach for microscopic diagnosis of malaria parasites in thick blood smears using pre-trained deep learning models

One of the deadly endemic diseases in sub-Saharan Africa is malaria. Its prevalence is promoted by lack of sufficient expertise to carry out accurate and timely diagnosis using the standard microscopy method. Where lab technicians are available, the results are usually subjective due to variations in expert judgement. To address this challenge, prompt interventions to improve disease control are needed. The emerging technologies of machine learning that can learn complex image patterns have accelerated research in medical image analysis. In this study, on a dataset of thick blood smear images, we evaluate and compare performance of three pre-trained deep learning architectures namely; faster regional convolutional neural network (faster R-CNN), single-shot multi-box detector (SSD) and RetinaNet through a Tensorflow object detection API. Data augmentation method was applied to optimise performance of the meta architectures. The possibility for mobile phone detector deployment was also investigated. The results revealed that faster R-CNN was the best trained model with a mean average precision of over 0.94 and SSD, was the best model for mobile deployment. We therefore deduce that faster R-CNN is best suited for obtaining high rates of accuracy in malaria detection while SDD is best suited for mobile deployment.
Open quantum entanglement: a study of two atomic system in static patch of de Sitter space

Abstract In this work, our prime objective is to study non-locality and long range effect of two body correlation using quantum entanglement from various information theoretic measure in the static patch of de Sitter space using a two body Open Quantum System (OQS). The OQS is described by a system of two entangled atoms, surrounded by a thermal bath, which is modelled by a massless probe scalar field. Firstly, we partially trace over the bath field and construct the Gorini Kossakowski Sudarshan Lindblad (GSKL) master equation, which describes the time evolution of the reduced subsystem density matrix. This GSKL master equation is characterized by two components, these are-Spin chain interaction Hamiltonian and the Lindbladian. To fix the form of both of them, we compute the Wightman functions for probe massless scalar field. Using this result alongwith the large time equilibrium behaviour we obtain the analytical solution for reduced density matrix. Further using this solution we evaluate various entanglement measures, namely Von-Neumann entropy, R $$e'$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:msup><mml:mi>e</mml:mi><mml:mo>′</mml:mo></mml:msup></mml:math> nyi entropy, logarithmic negativity, entanglement of formation, concurrence and quantum discord for the two atomic subsystem on the static patch of De-Sitter space. Finally, we have studied violation of Bell-CHSH inequality, which is the key ingredient to study non-locality in primordial cosmology.
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge

Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information. However, that knowledge exists only within the latent parameters of the model, inaccessible to inspection and interpretation, and even worse, factual information memorized from the training corpora is likely to become stale as the world changes. Knowledge stored as parameters will also inevitably exhibit all of the biases inherent in the source materials. To address these problems, we develop a neural language model that includes an explicit interface between symbolically interpretable factual information and subsymbolic neural knowledge. We show that this model dramatically improves performance on two knowledge-intensive question-answering tasks. More interestingly, the model can be updated without re-training by manipulating its symbolic representations. In particular this model allows us to add new facts and overwrite existing ones in ways that are not possible for earlier models.
SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning

Off-policy deep reinforcement learning (RL) has been successful in a range of challenging domains. However, standard off-policy RL algorithms can suffer from several issues, such as instability in Q-learning and balancing exploration and exploitation. To mitigate these issues, we present SUNRISE, a simple unified ensemble method, which is compatible with various off-policy RL algorithms. SUNRISE integrates two key ingredients: (a) ensemble-based weighted Bellman backups, which re-weight target Q-values based on uncertainty estimates from a Q-ensemble, and (b) an inference method that selects actions using the highest upper-confidence bounds for efficient exploration. By enforcing the diversity between agents using Bootstrap with random initialization, we show that these different ideas are largely orthogonal and can be fruitfully integrated, together further improving the performance of existing off-policy RL algorithms, such as Soft Actor-Critic and Rainbow DQN, for both continuous and discrete control tasks on both low-dimensional and high-dimensional environments. Our training code is available at this https URL.
Foundations, Properties, and Security Applications of Puzzles

Cryptographic algorithms have been used not only to create robust ciphertexts but also to generate cryptograms that, contrary to the classic goal of cryptography, are meant to be broken. These cryptograms, generally called puzzles, require the use of a certain amount of resources to be solved, hence introducing a cost that is often regarded as a time delay—though it could involve other metrics as well, such as bandwidth. These powerful features have made puzzles the core of many security protocols, acquiring increasing importance in the IT security landscape. The concept of a puzzle has subsequently been extended to other types of schemes that do not use cryptographic functions, such as CAPTCHAs, which are used to discriminate humans from machines. Overall, puzzles have experienced a renewed interest with the advent of Bitcoin, which uses a CPU-intensive puzzle as proof of work. In this article, we provide a comprehensive study of the most important puzzle construction schemes available in the literature, categorizing them according to several attributes, such as resource type, verification type, and applications. We have redefined the term puzzle by collecting and integrating the scattered notions used in different works, to cover all the existing applications. Moreover, we provide an overview of the possible applications, identifying key requirements and different design approaches. Finally, we highlight the features and limitations of each approach, providing a useful guide for the future development of new puzzle schemes.
Tempered Sigmoid Activations for Deep Learning with Differential Privacy

Because learning sometimes involves sensitive data, machine learning algorithms have been extended to offer differential privacy for training data. In practice, this has been mostly an afterthought, with privacy-preserving models obtained by re-running training with a different optimizer, but using the model architectures that already performed well in a non-privacy-preserving setting. This approach leads to less than ideal privacy/utility tradeoffs, as we show here. To improve these tradeoffs, prior work introduces variants of differential privacy that weaken the privacy guarantee proved to increase model utility. We show this is not necessary and instead propose that utility be improved by choosing activation functions designed explicitly for privacy-preserving training. A crucial operation in differentially private SGD is gradient clipping, which along with modifying the optimization path (at times resulting in not-optimizing a single objective function), may also introduce both significant bias and variance to the learning process. We empirically identify exploding gradients arising from ReLU may be one of the main sources of this. We demonstrate analytically and experimentally how a general family of bounded activation functions, the tempered sigmoids, consistently outperform the currently established choice: unbounded activation functions like ReLU. Using this paradigm, we achieve new state-of-the-art accuracy on MNIST, FashionMNIST, and CIFAR10 without any modification of the learning procedure fundamentals or differential privacy analysis. While the changes we make are simple in retrospect, the simplicity of our approach facilitates its implementation and adoption to meaningfully improve state-of-the-art machine learning while still providing strong guarantees in the original framework of differential privacy.
RemNet: remnant convolutional neural network for camera model identification

Camera model identification (CMI) has gained significant importance in image forensics as digitally altered images are becoming increasingly commonplace. In this paper, a novel convolutional neural network (CNN) architecture is proposed for CMI with emphasis given on the preprocessing task considered to be inevitable for removing the scene content that heavily obscures the camera model fingerprints. Unlike the conventional approaches where fixed filters are used for preprocessing, the proposed remnant blocks, when coupled with a classification block and trained end-to-end minimizing the classification loss, learn to suppress the unnecessary image contents dynamically. This helps the classification block extract more robust camera model-specific features for CMI from the remnant of the image. The whole network, called RemNet, consisting of a preprocessing block and a shallow classification block, when trained on 18 models from the Dresden database, shows 100% accuracy for 16 camera models with an overall accuracy of 97.59% on test images from unseen devices, outperforming the state-of-the-art deep CNNs used in CMI. Furthermore, the proposed remnant blocks, when cascaded with the existing deep CNNs, e.g., ResNet, DenseNet, boost their performances by a large margin. The proposed approach proves to be very robust in identifying the source camera models, even if the original images are post-processed. It also achieves an overall accuracy of 95.11% on the IEEE Signal Processing Cup 2018 dataset, which indicates its generalizability.
Virtual Reality Potentiates Emotion and Task Effects of Alpha/Beta Brain Oscillations

The progress of technology has increased research on neuropsychological emotion and attention with virtual reality (VR). However, direct comparisons between conventional two-dimensional (2D) and VR stimulations are lacking. Thus, the present study compared electroencephalography (EEG) correlates of explicit task and implicit emotional attention between 2D and VR stimulation. Participants (n = 16) viewed angry and neutral faces with equal size and distance in both 2D and VR, while they were asked to count one of the two facial expressions. For the main effects of emotion (angry vs. neutral) and task (target vs. nontarget), established event related potentials (ERP), namely the late positive potential (LPP) and the target P300, were replicated. VR stimulation compared to 2D led to overall bigger ERPs but did not interact with emotion or task effects. In the frequency domain, alpha/beta-activity was larger in VR compared to 2D stimulation already in the baseline period. Of note, while alpha/beta event related desynchronization (ERD) for emotion and task conditions were seen in both VR and 2D stimulation, these effects were significantly stronger in VR than in 2D. These results suggest that enhanced immersion with the stimulus materials enabled by VR technology can potentiate induced brain oscillation effects to implicit emotion and explicit task effects.
Effective Self-Recovery and Tampering Localization Fragile Watermarking for Medical Images

It is increasingly important to achieve security and anti-counterfeiting for medical images.The integrity and verifiability of medical images before they are reviewed for potential medical diagnoses is imperative.This paper presents an effective self-embedding fragile watermarking scheme for medical images.Both self-recovery information and authentication code for each block are generated in advance and then embedded into other blocks separately with the help of the turtle shell and embedding table.In the tampering detection and localization phase, a three-level detection and localization strategy is employed to achieve higher accuracy for tampering localization.In the self-recovery phase, a two-hierarchical restoration method is exploited to achieve higher image quality in the recovered images.Several malicious attacks were tested to demonstrate the performance of the proposed scheme in terms of tampering detection and image recovery.Simulation results confirmed that the tampering detection rate can be up to 99.83% and the average PSNR of a recovered image can be up to 42.11 dB, both of which are higher than those of the Shehab et al.'s method.
Artificial intelligence in pathology: an overview

Artificial intelligence (AI) is at the forefront of modern technology and emerging uses within the healthcare sector are now being realised. Pathology will be a key area where the impact of AI will be felt. With more and more laboratories making the transition to digital pathology this will provide the key infrastructure in which to deploy these tools and their use will start to become a reality in diagnostic practice. The potential of AI in pathology is to create image analysis tools which could either be used for diagnostic support or to derive novel insights into disease biology, in addition to those achievable with a human observer. Some examples providing diagnostic support currently exist for a limited, but expanding number of applications, such as tumour detection, automated tumour grading, immunohistochemistry scoring, and predicting mutation status. There are a number of challenges to consider, not least the validation and regulatory framework for these tools. In this article, we set out an overview of AI in histopathology, discuss its potential workflow applications, and give key examples of the potential for AI in clinical practice. Considerations for the implementation of AI in practice are also explored.
A Responsible Internet to Increase Trust in the Digital World

Abstract Policy makers in regions such as Europe are increasingly concerned about the trustworthiness and sovereignty of the foundations of their digital economy, because it often depends on systems operated or manufactured elsewhere. To help curb this problem, we propose the novel notion of a responsible Internet, which provides higher degrees of trust and sovereignty for critical service providers (e.g., power grids) and all kinds of other users by improving the transparency, accountability, and controllability of the Internet at the network-level. A responsible Internet accomplishes this through two new distributed and decentralized systems. The first is the Network Inspection Plane (NIP), which enables users to request measurement-based descriptions of the chains of network operators (e.g., ISPs and DNS and cloud providers) that handle their data flows or could potentially handle them, including the relationships between them and the properties of these operators. The second is the Network Control Plane (NCP), which allows users to specify how they expect the Internet infrastructure to handle their data (e.g., in terms of the security attributes that they expect chains of network operators to have) based on the insights they gained from the NIP. We discuss research directions and starting points to realize a responsible Internet by combining three currently largely disjoint research areas: large-scale measurements (for the NIP), open source-based programmable networks (for the NCP), and policy making (POL) based on the NIP and driving the NCP. We believe that a responsible Internet is the next stage in the evolution of the Internet and that the concept is useful for clean slate Internet systems as well.
Quantum control based on machine learning in an open quantum system

Designing robust control schemes in n-level open quantum system is significant for quantum computation. Here, we investigate two quantum control strategies based on supervised machine learning to suppress the quantum noise in an open quantum system. One is controlling state distance and the other is governing the average of a Hermitian operator. In this process, the dynamics of the system is mapped to a neural network where the control fields correspond to the weights. Besides, the system is transformed into the coherence Bloch space without using superoperator thus the complications are reduced largely. As an example, the two control protocols are demonstrated in a two-level and four-level systems, respectively. By applying these examples, the results show that the state of the system transfers to the target state and the average of a Hermitian operator to its minimum value in a given time despite disturbed by various types of noise.
Two-Factor Authentication Scheme for Mobile Money: A Review of Threat Models and Countermeasures

The proliferation of digital financial innovations like mobile money has led to the rise in mobile subscriptions and transactions. It has also increased the security challenges associated with the current two-factor authentication (2FA) scheme for mobile money due to the high demand. This review paper aims to determine the threat models in the 2FA scheme for mobile money. It also intends to identify the countermeasures to overcome the threat models. A comprehensive literature search was conducted from the Google Scholar and other leading scientific databases such as IEEE Xplore, MDPI, Emerald Insight, Hindawi, ACM, Elsevier, Springer, and Specific and International Journals, where 97 papers were reviewed that focused on the topic. Descriptive research papers and studies related to the theme were selected. Three reviewers extracted information independently on authentication, mobile money system architecture, mobile money access, the authentication scheme for mobile money, various attacks on the mobile money system (MMS), threat models in the 2FA scheme for mobile money, and countermeasures. Through literature analysis, it was found that the threat models in the 2FA scheme for mobile money were categorised into five, namely, attacks against privacy, attacks against authentication, attacks against confidentiality, attacks against integrity, and attacks against availability. The countermeasures include use of cryptographic functions (e.g., asymmetric encryption function, symmetric encryption function, and hash function) and personal identification (e.g., number-based and biometric-based countermeasures). This review study reveals that the current 2FA scheme for mobile money has security gaps that need to be addressed since it only uses a personal identification number (PIN) and a subscriber identity module (SIM) to authenticate users, which are susceptible to attacks. This work, therefore, will help mobile money service providers (MMSPs), decision-makers, and governments that wish to improve their current 2FA scheme for mobile money.
Simple Iterative Method for Generating Targeted Universal Adversarial Perturbations

Deep neural networks (DNNs) are vulnerable to adversarial attacks. In particular, a single perturbation known as the universal adversarial perturbation (UAP) can foil most classification tasks conducted by DNNs. Thus, different methods for generating UAPs are required to fully evaluate the vulnerability of DNNs. A realistic evaluation would be with cases that consider targeted attacks; wherein the generated UAP causes the DNN to classify an input into a specific class. However, the development of UAPs for targeted attacks has largely fallen behind that of UAPs for non-targeted attacks. Therefore, we propose a simple iterative method to generate UAPs for targeted attacks. Our method combines the simple iterative method for generating non-targeted UAPs and the fast gradient sign method for generating a targeted adversarial perturbation for an input. We applied the proposed method to state-of-the-art DNN models for image classification and proved the existence of almost imperceptible UAPs for targeted attacks; further, we demonstrated that such UAPs can be easily generated.
SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Driving

Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Despite more than a decade of research and development, the problem of how to competently interact with diverse road users in diverse scenarios remains largely unsolved. Learning methods have much to offer towards solving this problem. But they require a realistic multi-agent simulator that generates diverse and competent driving interactions. To meet this need, we develop a dedicated simulation platform called SMARTS (Scalable Multi-Agent RL Training School). SMARTS supports the training, accumulation, and use of diverse behavior models of road users. These are in turn used to create increasingly more realistic and diverse interactions that enable deeper and broader research on multi-agent interaction. In this paper, we describe the design goals of SMARTS, explain its basic architecture and its key features, and illustrate its use through concrete multi-agent experiments on interactive scenarios. We open-source the SMARTS platform and the associated benchmark tasks and evaluation metrics to encourage and empower research on multi-agent learning for autonomous driving. Our code is available at https://github.com/huawei-noah/SMARTS.