Reviewerly/exHarmony huggingface dataset:
PNS: Population-Guided Novelty Search for Reinforcement Learning in Hard Exploration Environments

Reinforcement Learning (RL) has made remarkable achievements, but it still suffers from inadequate exploration strategies, sparse reward signals, and deceptive reward functions. To alleviate these problems, a Population-guided Novelty Search (PNS) parallel learning method is proposed in this paper. In PNS, the population is divided into multiple sub-populations, each of which has one chief agent and several exploring agents. The chief agent evaluates the policies learned by exploring agents and shares the optimal policy with all sub-populations. The exploring agents learn their policies in collaboration with the guidance of the optimal policy and, simultaneously, upload their policies to the chief agent. To balance exploration and exploitation, the Novelty Search (NS) is employed in every chief agent to encourage policies with high novelty while maximizing per-episode performance. We apply PNS to the twin delayed deep deterministic (TD3) policy gradient algorithm. The effectiveness of PNS to promote exploration and improve performance in continuous control domains is demonstrated in the experimental section. Notably, PNS-TD3 achieves rewards that far exceed the SOTA methods in environments with sparse or delayed reward signals. We also demonstrate that PNS enables robotic agents to learn control policies directly from pixels for sparse-reward manipulation in both simulated and real-world settings.
Software Effort Estimation Using Genetic Algorithms with the Variance-Accounted-For (VAF) and the Manhattan Distance

The cost and effort for developing software projects gain a growing interest in recent years. Defining these parameters is considered as a valuable goal in achieving efficiency in developing the projects. Implementing the COCOMO model in effort estimation helps the project developers to allocate the resources efficiently. But there lies a main problem to optimize the constants in the COCOMO model. In this study, we present a way to optimize these constants using genetic algorithm by comparing two different methods in calculating the fitness function. Identifying the efficient method among them will help the optimization of COCOMO parameters and makes the effort estimation more efficient.
Workload Prediction for Cloud Resource Provisioning using Time Series Data

Analyzing and interpreting the fluctuating real-time workload by cloud providers is a challenging task for providing cloud services to end-users. Allocating computing resources (e.g., container machine) for running applications at right time solves the problem of the unavailability of service, maximizes response time, and minimizes throughput. Predictive analysis of data is important to identify the future trends and it also enables cloud providers to allocate the computing resources according to the demand for managing the workload. Predictive analysis has applications in different fields such as weather forecasting, stock price prediction, bio-informatics, and traffic over the Internet network (Cloud Computing). They use this analysis to assess the risk and predict future effect to avoid the different types of losses in the cloud computing area such as energy consumption, service delay, and customer loss. One of the methods to do predictive analysis is by using time series data. This paper presents the analysis of time series forecasting using the traditional classical Model (ARIMA) to predict the real-time fluctuating workload over the network. The predicted accuracy of model is measured using three metrics—RMSE, MSE, and MAE.
Open Data Settings: A Conceptual Framework Explored Through the Map Room Project

In recent years, researchers have sought more effective ways of making data open, for purposes of accountability, engagement, and reuse. Often, such efforts focus on making existing data sets available to broad audiences. The expression data set itself suggests something discrete, complete, and easily transferable. But data are none of those things. In this paper, we argue that open data projects could benefit from a more contextual understanding of what open means. Instead of focusing on open data sets, researchers can seek to create and understand open data settings: contexts in which things of public significance can be presented as evidence. We share our experiences creating and analyzing open data settings for the Map Room Project, a research through design initiative that establishes local spaces for collaborative data exploration and mapping. Our contribution is to offer a conceptual framework through which researchers, as well as designers, might think about the openness of data settings. This framework emerged from our situational analysis of comparative empirical case studies. In Map Rooms, we found that open can mean accessible, inclusive, or indeterminate. Three practices of contextualization, which we call configuring, convening, and claim-making, shape these dimensions of openness by defining all of the following: where data can work, who is empowered to use them, and what can count as data.
ASTRON: Action-Based Spatio-Temporal Robot Navigation

To achieve the tasks provided by a user, it is necessary for robots to have a plan that fully exploits their functionalities in an environment.The objective of this study is to realize robot task planning in real space for effectively use of the robot's functions The plan is formed by deriving a feasible action sequence by interpreting the instructions within the scope of the action possibilities of the robots and the changes in them.In this paper, we first propose an action graph as a novel environmental representation approach to facilitate the understanding of the robot's action possibility in real space.In the action graph, the action possibility is represented by nodes, which describe the spatial position to perform each feasible action, and edges, which describe the feasible actions, based on the subsystem-level affordance and the arrangement of objects in the environment.We also propose an action-based spatio-temporal robot navigation (ASTRON), which focuses on robot navigation tasks.ASTRON enables the robots to determine a feasible action sequence that utilizes their functions by interpreting the instructions based on the action graph.The effectiveness of the proposed method was evaluated through simulations and actual machine experiments in a coffee shop environment.In the actual machine experiments, the proposed method was applied to robots with different subsystem configurations.The experimental results demonstrated that the proposed method could plan the feasible action sequence to complete the tasks by considering the environmental state and the subsystem configurations of the robot.
Compositional Verification of Smart Contracts Through Communication Abstraction

Solidity smart contracts are programs that manage up to \(2^{160}\) users on a blockchain. Verifying a smart contract relative to all users is intractable due to state explosion. Existing solutions either restrict the number of users to under-approximate behaviour, or rely on manual proofs. In this paper, we present local bundles that reduce contracts with arbitrarily many users to sequential programs with a few representative users. Each representative user abstracts concrete users that are locally symmetric to each other relative to the contract and the property. Our abstraction is semi-automated. The representatives depend on communication patterns, and are computed via static analysis. A summary for the behaviour of each representative is provided manually, but a default summary is often sufficient. Once obtained, a local bundle is amenable to sequential static analysis. We show that local bundles are relatively complete for parameterized safety verification, under moderate assumptions. We implement local bundle abstraction in SmartACE, and show order-of-magnitude speedups compared to a state-of-the-art verifier.
Random Transformation of image brightness for adversarial attack

Deep neural networks (DNNs) are vulnerable to adversarial examples, which are crafted by adding small, human-imperceptible perturbations to the original images, but make the model output inaccurate predictions. Before DNNs are deployed, adversarial attacks can thus be an important method to evaluate and select robust models in safety-critical applications. However, under the challenging black-box setting, the attack success rate, i.e., the transferability of adversarial examples, still needs to be improved. Based on image augmentation methods, this paper found that random transformation of image brightness can eliminate overfitting in the generation of adversarial examples and improve their transferability. In light of this phenomenon, this paper proposes an adversarial example generation method, which can be integrated with Fast Gradient Sign Method (FGSM)-related methods to build a more robust gradient-based attack and to generate adversarial examples with better transferability. Extensive experiments on the ImageNet dataset have demonstrated the effectiveness of the aforementioned method. Whether on normally or adversarially trained networks, our method has a higher success rate for black-box attacks than other attack methods based on data augmentation. It is hoped that this method can help evaluate and improve the robustness of models.
Structure-aware Mathematical Expression Recognition with Sequence-Level Modeling

Mathematical expression recognition (MER) aims to convert an image of mathematical expressions into a Latex sequence. In practice, the task of MER is challenging, since 1) the images of mathematical expressions often contain complex structure relationships, e.g., fractions, matrixes, and subscripts; 2) the generated Latex sequences can be very complex and they have to satisfy strict syntax rules. Existing methods, however, often ignore the complex dependence among image regions, resulting in poor feature representation. In addition, they may fail to capture the rigorous relations among different formula symbols as they consider MER as a common language generation task. To address these issues, we propose a Structure-Aware Sequence-Level (SASL) model for MER. First, to better represent and recognize the visual content of formula images, we propose a structure-aware module to capture the relationship among different symbols. Meanwhile, the sequence-level modeling helps the model to concentrate on the generation of entire sequences. To make the problem feasible, we cast the generation problem into a Markov decision process (MDP) and seek to learn a Latex sequence generating policy. Based on MDP, we learn SASL by maximizing the matching score of each image-sequence pair to obtain the generation policy. Extensive experiments on the IM2LATEX-100K dataset verify the effectiveness and superiority of the proposed method.
Two-Round Adaptively Secure MPC from Isogenies, LPN, or CDH

We present a new framework for building round-optimal (two-round) adaptively secure MPC. We show that a relatively weak notion of OT that we call indistinguishability OT with receiver oblivious sampleability (\(\mathsf {r\hbox {-}iOT}\)) is enough to build two-round, adaptively secure MPC against malicious adversaries in the CRS model. We then show how to construct \(\mathsf {r\hbox {-}iOT}\) from CDH, LPN, or isogeny-based assumptions that can be viewed as group actions (such as CSIDH and CSI-FiSh). This yields the first constructions of two-round adaptively secure MPC against malicious adversaries from CDH, LPN, or isogeny-based assumptions. We further extend our non-isogeny results to the plain model, achieving (to our knowledge) the first construction of two-round adaptively secure MPC against semi-honest adversaries in the plain model from LPN.
Neuromorphology in-sensor computing architecture based on an optical Fourier transform

We propose an object recognition architecture relying on a neural network algorithm in optical sensors. Precisely, by applying the high-speed and low-power Fourier transform operation in the optical domain, we can transfer the high-cost part of the traditional convolutional neural network algorithm to the sensor side to achieve faster computing speed. An optical neuron unit (ONU) consisting of transition metal sulfide (TMD) material is fabricated for a vivid validation of this architecture. Using the embedded gate pair structure inside our ONU, TMD materials can be electrically doped at different levels, forming an in-plane PN junction, which allows for effective manipulation of light response to imitate biological nerve synapses. The results demonstrate that our ONU could reach the ability of optic neurons, providing experimental support for future in-sensor computing architecture.
Multi-User Drone Flight Training in Mixed Reality

The development of services and applications involving drones is promoting the growth of the unmanned-aerial-vehicle industry. Moreover, the supply of low-cost compact drones has greatly contributed to the popularization of drone flying. However, flying first-person-view (FPV) drones requires considerable experience because the remote pilot views a video transmitted from a camera mounted on the drone. In this paper, we propose a remote training system for FPV drone flying in mixed reality. Thereby, beginners who are inexperienced in FPV drone flight control can practice under the guidance of remote experts.
FISAR: Forward Invariant Safe Reinforcement Learning with a Deep Neural Network-Based Optimizer

This paper investigates reinforcement learning with constraints, which are indispensable in safety-critical environments. To drive the constraint violation to decrease monotonically, we take the constraints as Lyapunov functions and impose new linear constraints on the policy parameters&#x2019; updating dynamics. As a result, the original safety set can be forward-invariant. However, because the new guaranteed-feasible constraints are imposed on the updating dynamics instead of the original policy parameters, classic optimization algorithms are no longer applicable. To address this, we propose to learn a generic deep neural network (DNN)-based optimizer to optimize the objective while satisfying the linear constraints. The constraint-satisfaction is achieved via projection onto a polytope formulated by multiple linear inequality constraints, which can be solved analytically with our newly designed metric. To the best of our knowledge, this is the first DNN-based optimizer for constrained optimization with the forward invariance guarantee. We show that our optimizer trains a policy to decrease the constraint violation and maximize the cumulative reward monotonically. Results on numerical constrained optimization and obstacle-avoidance navigation validate the theoretical findings.
The Fast Simulation Architecture Construction for Integrated Electric Transmission and Distribution Power Grid Based On Big Data Platform

This paper studies constructing methods and key technologies of the fast simulation architecture for integrated electric transmission and distribution power grid based on big data platform, aiming at the problems about difficult communication, poor collaboration, low reusability of services and applications, and time-consuming preparation in the fast integrated simulation system. Specifically, this paper focuses on the creation technology of simulating scenarios, the design of dispatcher storage system for distributed tasks, the management of service based on micro service framework, and the simulation data mart, etc. Besides, a business flow and data flow of fast simulation are realized. Comparing with the traditional DTS process, this method makes full use of simulation resource, also improves the speed and quality of simulation process and achieves a better human-computer interaction experience. In addition, this method enables the reusability for services and applications. The feasibility and effectiveness is validated.
PROBLEMS OF DIFFERENTIAL AND TOPOLOGICAL DIAGNOSTICS. PART 5. THE CASE OF TRAJECTORIAL MEASUREMENTS WITH ERROR

Proposed work is the fifth work of the cycle on differential and topological diagnostics. The article gives an estimate of the errors method of direction fields in the case of not accurate trajectory measurements, but trajectory measurements with an error are limited by the modulus of a given a smooth function of time, and in case this error is a random variable distributed according to the normal law with fixed parameters. We show that in these more complex cases, you can specify the best number of required trajectory measurements, inwhich the proposed algorithms of diagnostics will work constructively, and the malfunction will be determined unambiguously.
A Predicting Model For Accounting Fraud Based On Ensemble Learning

Accounting fraud, usually difficult to detect, can cause significant harm to stakeholders and serious damage to the market. Effective methods of accounting fraud detection are needed for the prevention and governance of accounting fraud.In this study, we develop a novel accounting fraud prediction model using XGBoost, a powerful ensemble learning approach. We respectively select 12 financial ratios, 28 raw accounting numbers and 99 raw accounting numbers available from Chinese listed firms’ financial statements, as the model input. To assess the performance of fraud prediction models, we select two evaluation metrics - AUC and NDCG@k, and two benchmark models - the Dechow et al. (2011) logistic regression model based on financial ratios, and the Bao et al. (2020) AdaBoost model based on raw accounting numbers.Results show that: 1) our XGBoost-based prediction model outperforms two benchmark models by a large margin whatever model inputs and evaluation metrics; 2) the XGBoost-based prediction model with raw accounting numbers input outperforms the one with financial ratios input; 3) the XGoost-based prediction model with 99 raw accounting numbers input outperforms the one with 28 raw accounting numbers input.
Interrogating Alexa

This paper reports on a preliminary comparative study of Alexa, Siri, and Google Assistant voice assistants (VA) that explores the origins of answers provided on each platform in an attempt to determine the extent that these origins influence responses. Questions were selected from Text Recognition (TREC) 2017 Live Question Answering (QA) Track Data, a collection of pre-assessed questions that are part of the National Institute of Standards and Technology (NIST) TREC QA track. Responses were collected as voice memos and screen captures, then analyzed to determine the origins of each answer or set of answers provided. Results indicate that the origins of answers are different search engines, and that algorithm-centered processes in each voice assistant result in vast differences in answers to questions. Because the online search results provided as answers by voice assistants are influenced by content and structured data, technical communicators and UX practitioners can help ensure that voice assistants are able to provide accurate, complete, and ethical responses to users’ questions.
Small Object Detection and Tracking from Aerial Imagery

Object detection and tracking from airborne imagery draws attention to the parallel development of UAV systems and computer vision technologies. Aerial imagery has its own unique challenges that differ from the training set of modern-day object detectors, since it is made of images of larger areas compared to the regular datasets and the objects are very small on the contrary. These problems do not allow us to use common object detection models. The main purpose of this paper is to make modifications to the Faster-RCNN (FRCNN) model, then leverage it for small object detection and tracking from the aerial imagery. It is aimed to use both spatial and temporal information from the image sequence, as appearance information alone is insufficient. The anchors in the Region Proposal Network (RPN) stage will be adjusted for small objects. Also, intersection over union (IoU) is optimized for small objects. After improving detection performance, The DeepSORT algorithm is inserted right after the Region of Interest (ROI Head) to track the objects. The results show that the proposed model has good performance on the VisDrone-2019 dataset. Detection performance becomes considerably better than the original FRCNN and the algorithms that are evaluated in the VisDrone-2019 VID challenge. After completing the proposed modifications, the AP-AP50 values reached 14.07-29.41 from 8.08-18.70, which means approximately 75% improvement.
Γ-profilometry: a new paradigm for precise optical metrology

We show that the shape of a surface can be unambiguously determined from investigating the coherence function of a wave-field reflected by the surface and without the requirement of a reference wave. Spatio-temporal sampling facilitates the identification of temporal shifts of the coherence function that correspond to finite height differences of the surface. Evaluating these finite differences allows for the reconstruction of the surface using a numerical integration procedure. Spatial sampling of the coherence function is provided by a shear interferometer whereas temporal sampling is achieved by means of a Soleil-Babinet compensator. This low coherence profiling method allows to determine the shape of an object with sub-micrometer resolution and over a large unambiguity range, although it does not require any isolation against mechanical vibration. The approach therefore opens up a new avenue for precise, rugged optical metrology suitable for industrial in-line applications.
Declarative Goal Mediation in Smart Environments

Smart environments enabled by the Internet of Things aim at improving our daily lives by automatically tuning ambient parameters and by achieving energy savings through self-managing cyber-physical systems. Commercial solutions, however, only permit setting simple target goals on those parameters and do not mediate between conflicting goals among different users and/or system administrators, nor across different IoT verticals. In this article, we propose a declarative approach (and its open-source Prolog prototype) to represent smart environments, user-set goals and customisable mediation policies to reconcile contrasting goals across multiple IoT systems.
Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models

The potential benefits of model-free reinforcement learning to real robotics systems are limited by its uninformed exploration that leads to slow convergence, lack of data-efficiency, and unnecessary interactions with the environment. To address these drawbacks we propose a method that combines reinforcement and imitation learning by shaping the reward function with a state-and-action-dependent potential that is trained from demonstration data, using a generative model. We show that this accelerates policy learning by specifying high-value areas of the state and action space that are worth exploring first. Unlike the majority of existing methods that assume optimal demonstrations and incorporate the demonstration data as hard constraints on policy optimization, we instead incorporate demonstration data as advice in the form of a reward shaping potential trained as a generative model of states and actions. In particular, we examine both normalizing flows and Generative Adversarial Networks to represent these potentials. We show that, unlike many existing approaches that incorporate demonstrations as hard constraints, our approach is unbiased even in the case of suboptimal and noisy demonstrations. We present an extensive range of simulations, as well as experiments on the Franka Emika 7DOF arm, to demonstrate the practicality of our method.