Reviewerly/exHarmony huggingface dataset:
Off-Policy Evaluation via Off-Policy Classification

In this work, we consider the problem of model selection for deep reinforcement learning (RL) in real-world environments. Typically, the performance of deep RL algorithms is evaluated via on-policy interactions with the target environment. However, comparing models in a real-world environment for the purposes of early stopping or hyperparameter tuning is costly and often practically infeasible. This leads us to examine off-policy policy evaluation (OPE) in such settings. We focus on OPE of value-based methods, which are of particular interest in deep RL with applications like robotics, where off-policy algorithms based on Q-function estimation can often attain better sample complexity than direct policy optimization. Furthermore, existing OPE metrics either rely on a model of the environment, or the use of importance sampling (IS) to correct for the data being off-policy. However, for high-dimensional observations, such as images, models of the environment can be difficult to fit and value-based methods can make IS hard to use or even ill-conditioned, especially when dealing with continuous action spaces. In this paper, we focus on the specific case of MDPs with continuous action spaces and sparse binary rewards, which is representative of many important real-world applications. We propose an alternative metric that relies on neither models nor IS, by framing OPE as a positive-unlabeled (PU) classification problem. We experimentally show that this metric outperforms baselines on a number of tasks. Most importantly, it can reliably predict the relative performance of different policies in a number of generalization scenarios, including the transfer to the real-world of policies trained in simulation for an image-based robotic manipulation task.
Some New Trends of Deep Learning Research

Deep learning has been attracting increasing attention in the recent decade throughout science and engineering due to its wide range of successful applications. In real problems, however, most implementation stages for applying deep learning still require inevitable manual interventions, which naturally conducts difficulty in its availability to general users with less expertise and also deviates from the intelligence of humans. It is thus a challenging while critical issue to enhance the level of automation across all elements of the entire deep learning framework, like input amelioration, model designing and learning, and output adjustment. This paper tries to list several representative issues of this research topic, and briefly describe their recent research progress and some related works proposed along this research line. Some specific challenging problems have also been presented.
A Rigorous Theory of Conditional Mean Embeddings

Conditional mean embeddings (CMEs) have proven themselves to be a powerful tool in many machine learning applications. They allow the efficient conditioning of probability distributions within the corresponding reproducing kernel Hilbert spaces by providing a linear-algebraic relation for the kernel mean embeddings of the respective joint and conditional probability distributions. Both centered and uncentered covariance operators have been used to define CMEs in the existing literature. In this paper, we develop a mathematically rigorous theory for both variants, discuss the merits and problems of each, and significantly weaken the conditions for applicability of CMEs. In the course of this, we demonstrate a beautiful connection to Gaussian conditioning in Hilbert spaces.
An evolutionary strategy for finding effective quantum 2-body Hamiltonians of p-body interacting systems

Embedding p-body interacting models onto the 2-body networks implemented on commercial quantum annealers is a relevant issue. For highly interacting models, requiring a number of ancilla qubits, that can be sizable and make unfeasible (if not impossible) to simulate such systems. In this manuscript, we propose an alternative to minor embedding, developing a new approximate procedure based on genetic algorithms, allowing to decouple the p-body in terms of 2-body interactions. A set of preliminary numerical experiments demonstrates the feasibility of our approach for the ferromagnetic p-spin model and paves the way towards the application of evolutionary strategies to more complex quantum models.
Pupil’s representation about the socio-psychological qualities of teachers in the process of effective meaning transmission

In the article, the authors raise the question of the relationship between the pupils’ perception of the teacher and effective meaning transmission. The study identified socio-psychological qualities and non-verbal patterns of teacher behavior that influence the effectiveness of meaning from the point of view of students. Students prefer to come into contact, with a teacher who has characteristics such as sincerity, leadership, compassion, restraint, responsibility, kindness, cordiality and others. It is established that among the nonverbal patterns of behavior that contribute to the effectiveness of meaning, pupils are considered “gestures appropriate to the situation communication”, “communication distance, which is regulated in accordance with the situation” and “the manifestation of the ability to listen and speak”.
Grasping Control Method of Manipulator Based on Binocular Vision Combining Target Detection and Trajectory Planning

In order to make the manipulator more accurately identify and grasp the target object, this paper proposes a manipulator control method based on binocular vision. First, the rotation relation between lever arms of manipulator is modeled by the D-H method and the model is simplified to improve the efficiency of the rotation operation. Then, apply the Canny edge detection algorithm based on binocular vision to identify and locate the target, and calculate the position of the target in the visual coordinate system. Next, the position of the target in the manipulator coordinate system is obtained according to the coordinate conversion. Finally, plan the movement trajectory of manipulator according to the inverse kinematics solution and motion trajectory planning algorithm, and drive the manipulator to capture the target. The experimental results show that the manipulator can accurately identify and capture the target with proposed control method.
XRA-Net Framework for Visual Sentiments Analysis

The exponential growth of social media has motivated people to express themselves in various forms. Visual media is one of the most effective and popular ways of conveying sentiments or opinions on the web as people keeps on uploading millions of photos on famous social networking sites. Hence, Visual Sentiment Analysis is instrumental in monitoring an overview of the broader public consensus behind a specific topic or issue. This work proposes a deep learning-based architecture XRA-Net (Xception Residual Attention based Network) for visual sentiment analysis. Moreover, the performance of the XRA-Net architecture is evaluated on the publicly available real-world Twitter I dataset, which is further composed of three subsets of the dataset: 3-agree, 4-agree, and 5-agree. The accuracy achieved on these datasets are: 79.2%, 81.2%, and 86.4% respectively, which shows that the proposed architecture has outperformed the state-of-the-art results on all the three subsets of Twitter I dataset as it can focus on the most informative features in an input image, which boosts the visual sentiment analysis process.
PointRend: Image Segmentation as Rendering

We present a new method for efficient high-quality image segmentation of objects and scenes. By analogizing classical computer graphics methods for efficient rendering with over- and undersampling challenges faced in pixel labeling tasks, we develop a unique perspective of image segmentation as a rendering problem. From this vantage, we present the PointRend (Point-based Rendering) neural network module: a module that performs point-based segmentation predictions at adaptively selected locations based on an iterative subdivision algorithm. PointRend can be flexibly applied to both instance and semantic segmentation tasks by building on top of existing state-of-the-art models. While many concrete implementations of the general idea are possible, we show that a simple design already achieves excellent results. Qualitatively, PointRend outputs crisp object boundaries in regions that are over-smoothed by previous methods. Quantitatively, PointRend yields significant gains on COCO and Cityscapes, for both instance and semantic segmentation. PointRend's efficiency enables output resolutions that are otherwise impractical in terms of memory or computation compared to existing approaches. Code has been made available at this https URL.
Optimistic Exploration even with a Pessimistic Initialisation

Optimistic initialisation is an effective strategy for efficient exploration in reinforcement learning (RL). In the tabular case, all provably efficient model-free algorithms rely on it. However, model-free deep RL algorithms do not use optimistic initialisation despite taking inspiration from these provably efficient tabular algorithms. In particular, in scenarios with only positive rewards, Q-values are initialised at their lowest possible values due to commonly used network initialisation schemes, a pessimistic initialisation. Merely initialising the network to output optimistic Q-values is not enough, since we cannot ensure that they remain optimistic for novel state-action pairs, which is crucial for exploration. We propose a simple count-based augmentation to pessimistically initialised Q-values that separates the source of optimism from the neural network. We show that this scheme is provably efficient in the tabular setting and extend it to the deep RL setting. Our algorithm, Optimistic Pessimistically Initialised Q-Learning (OPIQ), augments the Q-value estimates of a DQN-based agent with count-derived bonuses to ensure optimism during both action selection and bootstrapping. We show that OPIQ outperforms non-optimistic DQN variants that utilise a pseudocount-based intrinsic motivation in hard exploration tasks, and that it predicts optimistic estimates for novel state-action pairs.
Smart Intra-query Fault Tolerance for Massive Parallel Processing Databases

Abstract Intra-query fault tolerance has increasingly been a concern for online analytical processing, as more and more enterprises migrate data analytical systems from mainframes to commodity computers. Most massive parallel processing (MPP) databases do not support intra-query fault tolerance. They may suffer from prolonged query latency when running on unreliable commodity clusters. While SQL-on-Hadoop systems can utilize the fault tolerance support of low-level frameworks, such as MapReduce and Spark, their cost-effectiveness is not always acceptable. In this paper, we propose a smart intra-query fault tolerance (SIFT) mechanism for MPP databases. SIFT achieves fault tolerance by performing checkpointing, i.e., materializing intermediate results of selected operators. Different from existing approaches, SIFT aims at promoting query success rate within a given time. To achieve its goal, it needs to: (1) minimize query rerunning time after encountering failures and (2) introduce as less checkpointing overhead as possible. To evaluate SIFT in real-world MPP database systems, we implemented it in Greenplum. The experimental results indicate that it can improve success rate of query processing effectively, especially when working with unreliable hardware.
Deformable Kernels: Adapting Effective Receptive Fields for Object Deformation

Convolutional networks are not aware of an object's geometric variations, which leads to inefficient utilization of model and data capacity. To overcome this issue, recent works on deformation modeling seek to spatially reconfigure the data towards a common arrangement such that semantic recognition suffers less from deformation. This is typically done by augmenting static operators with learned free-form sampling grids in the image space, dynamically tuned to the data and task for adapting the receptive field. Yet adapting the receptive field does not quite reach the actual goal -- what really matters to the network is the *effective* receptive field (ERF), which reflects how much each pixel contributes. It is thus natural to design other approaches to adapt the ERF directly during runtime. In this work, we instantiate one possible solution as Deformable Kernels (DKs), a family of novel and generic convolutional operators for handling object deformations by directly adapting the ERF while leaving the receptive field untouched. At the heart of our method is the ability to resample the original kernel space towards recovering the deformation of objects. This approach is justified with theoretical insights that the ERF is strictly determined by data sampling locations and kernel values. We implement DKs as generic drop-in replacements of rigid kernels and conduct a series of empirical studies whose results conform with our theories. Over several tasks and standard base models, our approach compares favorably against prior works that adapt during runtime. In addition, further experiments suggest a working mechanism orthogonal and complementary to previous works.
SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization

Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantly outperforms prior art of detectors. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: https://github.com/tensorflow/tpu/tree/master/models/official/detection.
Implementation of Blockchain Technology in Insurance Contracts Against Natural Hazards: A Methodological Multi-Disciplinary Approach

Abstract Risk insurance for disasters plays a relevant part in the implementation of risk reduction strategies during the pre-disaster phase. This is essential to support risk management towards decreasing the marginal risk allowing policy holders to transfer risk to avoid considerable financial loads from the costs incurred during the recovery phase in a post-disaster phase. There is evidence that the introduction of an integrated risk insurance strategy for community resilience planning is still lacking. Thus, this undermines the possibility to have proper optimized holistic risk management; on the one hand this strengthens pre-disaster risk mitigation measures, mostly relying on mitigative infrastructural solutions, and on the other hand it better defines risk prevention strategies mostly connected to land planning and urban development. This paper will show how insurance markets can play a key role towards mitigating the economic consequences of natural and climate change disasters, and how essential it is to better quantify the beneficial effects and costs of engineer-based mitigative solutions. In this context, the legal framework into which the actuarial quantitative model can be implemented will support the creation of an integrated multidisciplinary approach with potential implementation on a novel platform capable of collecting and processing information from different sources and dimensions such as blockchain technology. The scientific community is, in fact, increasingly interested in implementing blockchain technology to overcome problems linked to the contractual dimension of natural disaster risk insurance which can be interpreted as a sort of smart contracting. Through a study that involved four distinct areas, namely: law, environmental engineering, insurance and IT, this paper proposes a specific multidisciplinary methodology to achieve the drafting and implementation of a digital insurance contract on a blockchain platform against natural hazards. This paper proposes the basis to advance a quantitative concept to optimize the impact of catastrophe risk insurance onto the community resilience; in fact providing a key synergy for definition of pre-disaster conditions.
Anisotropic Denoising of 3D Point Clouds by Aggregation of Multiple Surface-Adaptive Estimates

3D point clouds commonly contain positional errors which can be regarded as noise. We propose a point cloud denoising algorithm based on aggregation of multiple anisotropic estimates computed on local coordinate systems. These local estimates are adaptive to the shape of the surface underlying the point cloud, leveraging an extension of the Local Polynomial Approximation (LPA) - Intersection of Confidence Intervals (ICI) technique to 3D point clouds. The adaptivity due to LPA-ICI is further strengthened by the dense aggregation with data-driven weights. Experimental results demonstrate state-of-the-art restoration quality of both sharp features and smooth areas.
Quantum process capability

Physical processes in the quantum regime possess non-classical properties of quantum mechanics. However, methods for quantitatively identifying such processes are still lacking. Accordingly, in this study, we develop a framework for characterizing and quantifying the ability of processes to cause quantum-mechanical effects on physical systems. We start by introducing a new concept, referred to as quantum process capability, to evaluate the effects of an experimental process upon a prescribed quantum specification. Various methods are then introduced for measuring such a capability. It is shown that the methods are adapted to quantum process tomography for implementation of process capability measure and applicable to all physical processes that can be described using the general theory of quantum operations. The utility of the proposed framework is demonstrated through several examples, including processes of entanglement, coherence, and superposition. The formalism proposed in this study provides a generic approach for the identification of dynamical processes in quantum mechanics and facilitates the general classification of quantum-information processing.
Educational management of innovative activity of teachers as an object of pedagogical research

Introduction. To achieve the effectiveness of innovation processes, a new approach to ensuring the participation of teachers in innovation activities is needed, aimed at the effectiveness and manageability of these processes. In this context, the idea of developing ways of educational management of innovative activity of a University teacher arises.. Materials and methods. The main methods were the analysis of sources on the studied problem, including normative documents, scientific articles on the studied problem, as well as generalization and systematization of the obtained data. Results. Examines approaches to the concept of "educational management" is defined by specificity of educational management innovation, justify the choice of methods of implementation of the educational management of innovative activity of the teaching staff of the universities. Educational management of innovation activity of the teaching staff of the University should include several processes: management of innovations in the University, management of environmental factors, management of personal and professional resources, the process of overcoming professional difficulties of innovation (consulting). Discussion and Conclusions. Innovative activity of the pedagogical staff of the University is provided by involvement as a way of educational management aimed at overcoming professional difficulties. We associate the further direction of the research with the definition of systemic components of involvement).
Blockchain for Machine to Machine Interaction in Industry 4.0

Internet of Things (IoT) has already widely adopted for industrial applications in tasks such as automation, diagnostics, and management of industrial machines and the processes inherent in the formation of supply chains. Such devices have unique characteristics, such as reduced processing and memory, low bandwidth for data transmission and data collection, and limited autonomy, as they are usually battery-powered.
Improved hybrid layered image compression using deep learning and traditional codecs

Recently deep learning-based methods have been applied in image compression and achieved many promising results. In this paper, we propose an improved hybrid layered image compression framework by combining deep learning and the traditional image codecs. At the encoder, we first use a convolutional neural network (CNN) to obtain a compact representation of the input image, which is losslessly encoded by the FLIF codec as the base layer of the bit stream. A coarse reconstruction of the input is obtained by another CNN from the reconstructed compact representation. The residual between the input and the coarse reconstruction is then obtained and encoded by the H.265/HEVC-based BPG codec as the enhancement layer of the bit stream. Experimental results using the Kodak and Tecnick datasets show that the proposed scheme outperforms the state-of-the-art deep learning-based layered coding scheme and traditional codecs including BPG in both PSNR and MS-SSIM metrics across a wide range of bit rates, when the images are coded in the RGB444 domain.
Accurate Hand Detection from Single-Color Images by Reconstructing Hand Appearances

Hand detection is a crucial pre-processing procedure for many human hand related computer vision tasks, such as hand pose estimation, hand gesture recognition, human activity analysis, and so on. However, reliably detecting multiple hands from cluttering scenes remains to be a challenging task because of complex appearance diversities of dexterous human hands (e.g., different hand shapes, skin colors, illuminations, orientations, and scales, etc.) in color images. To tackle this problem, an accurate hand detection method is proposed to reliably detect multiple hands from a single color image using a hybrid detection/reconstruction convolutional neural networks (CNN) framework, in which regions of hands are detected and appearances of hands are reconstructed in parallel by sharing features extracted from a region proposal layer, and the proposed model is trained in an end-to-end manner. Furthermore, it is observed that the generative adversarial network (GAN) could further boost the detection performance by generating more realistic hand appearances. The experimental results show that the proposed approach outperforms the state-of-the-art on public challenging hand detection benchmarks.