Reviewerly/exHarmony huggingface dataset:
Improving Sample Complexity Bounds for Actor-Critic Algorithms

The actor-critic (AC) algorithm is a popular method to find an optimal policy in reinforcement learning. In the infinite horizon scenario, the finite-sample convergence rate for the AC and natural actor-critic (NAC) algorithms has been established recently, but under independent and identically distributed (i.i.d.) sampling and single-sample update at each iteration. In contrast, this paper characterizes the convergence rate and sample complexity of AC and NAC under Markovian sampling, with mini-batch data for each iteration, and with actor having general policy class approximation. We show that the overall sample complexity for a mini-batch AC to attain an $\epsilon$-accurate stationary point improves the best known sample complexity of AC by an order of $\mathcal{O}(\epsilon^{-1}\log(1/\epsilon))$, and the overall sample complexity for a mini-batch NAC to attain an $\epsilon$-accurate globally optimal point improves the existing sample complexity of NAC by an order of $\mathcal{O}(\epsilon^{-1}/\log(1/\epsilon))$. Moreover, the sample complexity of AC and NAC characterized in this work outperforms that of policy gradient (PG) and natural policy gradient (NPG) by a factor of $\mathcal{O}((1-\gamma)^{-3})$ and $\mathcal{O}((1-\gamma)^{-4}\epsilon^{-1}/\log(1/\epsilon))$, respectively. This is the first theoretical study establishing that AC and NAC attain orderwise performance improvement over PG and NPG under infinite horizon due to the incorporation of critic.
Multi-granularity generative adversarial nets with reconstructive sampling for image inpainting

Image inpainting based on deep neural network has drawn more attention with the development of deep learning, then Generative adversarial nets(GANs) which is a combination of multiple deep networks has been applied to image inpainting and achieved top-level performance. However, GANs-based inpainting method always suffers from unstable training and vanishing gradient. In this paper, we present an image inpainting method based on GANs with reconstructive sampling and multi-granularity generative adversarial strategy. The key idea is to solve unstable training and vanishing gradient by proposed reconstructive sampling, in which we sample from reconstructive distribution than low-dimension noise. It has been proved that reconstructive sampling is effective to avoid unstable training and gradient vanish. Then, multi-granularity generative adversarial strategy, which is decomposed into two steps, is adopted to make inpainted image more continuous and realistic in texture structure and vision, respectively. Extensive experiments show that ours brings substantial improvements over other state-of-the-art image inpainting algorithms in distortion and perception evaluation. Besides, comparisons on stability with baselines show that our method gains better stability during image inpainting.
ESA-Stream: Efficient Self-Adaptive Online Data Stream Clustering

Many big data applications produce a massive amount of high-dimensional, real-time, and evolving streaming data. Clustering such data streams with both effectiveness and efficiency are critical for these applications. Although there are well-known data stream clustering algorithms that are based on the popular online-offline framework, these algorithms still face some major challenges. Several critical questions are still not answer satisfactorily: How to perform dimensionality reduction effectively and efficiently in the online dynamic environment? How to enable the clustering algorithm to achieve complete real-time online processing? How to make algorithm parameters learn in a self-supervised or self-adaptive manner to cope with high-speed evolving streams? In this paper, we focus on tackling these challenges by proposing a fully online data stream clustering algorithm (called ESA-Stream) that can learn parameters online dynamically in a self-adaptive manner, speedup dimensionality reduction, and cluster data streams effectively and efficiently in an online and dynamic environment. Experiments on a wide range of synthetic and real-world data streams show that ESA-Stream outperforms state-of-the-art baselines considerably in both effectiveness and efficiency.
ERRANT: Realistic emulation of radio access networks

Mobile devices drastically changed how people use the Internet. We use smartphones to access a heterogeneous catalog of web services such as news, social networks, audio/video streaming. Differently from wired connections, mobile networks do not offer the same kind of performance stability yet. Thus, service providers have to handle different network scenarios, e.g., 3G or 4G, while promising good end-users’ quality of experience (QoE). To ensure that QoE is adequate, it is necessary to thoroughly test applications with a wide range of possible network conditions. For this, network emulation is of vital importance as it allows a tester to run experiments with a wide range of network conditions. However, when it comes to mobile networks, the variety of technical characteristics, coupled with the opaque network configurations, makes realistic emulation a challenging task. Most of the freely available emulation tools rely on a simple emulation, offering limited variability performances for each network condition. In this paper, we propose ERRANT, EmulatoR of Radio Access NeTworks, an open-source tool that emulates mobile networks with a high level of realism, following a data-driven approach. We use a large-scale dataset composed of 100 k speed test measurements collected from 4 network operators in 2 countries. We create 32 different network profiles based on different countries, operators, radio access technologies, and signal qualities. For each profile, we obtain both typical behavior and variability for latency, download and upload bandwidth. We use the profiles to create models by means of the Kernel Density Estimation. Then, ERRANT employs the tc-netem Linux tool and the models for emulation. In this way, ERRANT offers realistic network emulation, in which both typical behavior and network variability are accurately recreated. We validate ERRANT models with an independent dataset of HTTP downloads performed on the same mobile networks as of the profiles. Results show the effectiveness of ERRANT in the emulation of real mobile networks in terms of average behavior and obtained variability. We also show the limitations of a simple emulation, and of other freeware approaches versus ERRANT. Finally, we show two practical use cases to demonstrate the benefits of a dynamic emulation in understanding the performance of web browsing and video streaming. To run new measurement campaigns and create new models, we provide guidelines along with the required open-source code.
A Mixed CVaR-Based Stochastic Information Gap Approach for Building Optimal Offering Strategies of a CSP Plant in Electricity Markets

The development of the concentrating solar power (CSP) plant as a new dispatchable resource that can participate in the electricity markets as an independent power producer and coordinate intermittent renewables has attracted much attention recently. In this work, optimal offering strategies of a price-taker CSP plant in the day-ahead (DA) and real-time (RT) electricity markets are addressed considering non-stochastic uncertainties (NSUs) from the thermal production of the CSP plant and stochastic uncertainties (SUs) from the market prices as well as the risk attitude of the CSP plant concerned. A hybrid stochastic information gap approach (SIGA) integrating the well-established information gap decision theory with the mixed conditional value at risk (CVaR) is proposed to hedge the revenue risk against NSUs and SUs in the offering problem based on the risk preference of the decision maker. A two-stage architecture is utilized for framing the DA and RT offering problems, where the first-stage co-optimizes offering strategies in the DA and RT markets, while the second-stage determines the actual RT hourly offering strategy in a rolling horizon manner. Case studies show that the SIGA can make optimal offering strategies against the non-stochastic thermal production and stochastic market prices given the risk attitude of the CSP plant. Comparisons also demonstrate that the SIGA could be an effective tool to manage coexistent NSUs and SUs.
Spatial prediction of oil and gas distribution using Tree Augmented Bayesian network

Accurate prediction of the spatial distribution of hydrocarbon accumulations underlies risk reduction and improvement of exploration work. Determining the location of oil and gas reserves is a comprehensive and complex problem riddled with uncertainty. This paper applies Bayesian network, which is an effective tool for uncertainty knowledge expression and reasoning, to predict the spatial distribution of oil and gas resources, which is the first attempt of its kind. Taking the first member of Dongying Formation (Ed1) of Nanpu Depression in Bohai Bay Basin for example, 222 exploratory wells, basin simulation, seismic interpretation, and other data were synergized to train the Tree Augmented Bayesian network (TAN) structure, then the hydrocarbon-bearing posterior probability of the target layer is calculated based on the trained TAN model, and ultimately the probability map of hydrocarbon spatial distribution prediction in Ed1 was obtained. Prediction results indicate that oilfields are found mostly in areas with high resource potential and high hydrocarbon-bearing probability. Notably, the northern part of No.2 structural belt has a high petroleum bearing probability and no oilfields have ever been found there, which can be considered as the next exploration direction. The application results show that Bayesian network serves to capture essential spatial characteristics of hydrocarbon accumulations and accurately predict the spatial distribution of oil and gas resources, which can help to visualize the geological risk, optimize the drilling strategy, and thus contribute to exploration success.
Introduction to Data Science: Data Analysis and Prediction Algorithms With R

The textbook belongs to the Data Science series and presents a modern approach to statistical evaluations via powerful abilities of the R language. The monograph is organized in six parts and thirt...
Sequential Dispersive Measurement of a Superconducting Qubit

We present a superconducting device that realizes the sequential measurement of a transmon qubit. The device disables common limitations of dispersive readout such as Purcell effect or transients in the cavity mode by turning on and off the coupling to the measurement channel on demand. The qubit measurement begins by loading a readout resonator that is coupled to the qubit. After an optimal interaction time with negligible loss, a microwave pump releases the content of the readout mode by upconversion into a measurement line in a characteristic time as low as 10~ns, which is 400 times shorter than the lifetime of the readout resonator. A direct measurement of the released field quadratures demonstrates a readout fidelity of $97.5~\%$ in a total measurement time of $220~\mathrm{ns}$. The Wigner tomography of the readout mode allows us to characterize the non-Gaussian nature of the readout mode and its dynamics.
Spectral Clustering by Subspace Randomization and Graph Fusion for High-Dimensional Data

Subspace clustering has been gaining increasing attention in recent years due to its promising ability in dealing with high-dimensional data. However, most of the existing subspace clustering methods tend to only exploit the subspace information to construct a single affinity graph (typically for spectral clustering), which often lack the ability to go beyond a single graph to explore multiple graphs built in various subspaces in high-dimensional space. To address this, this paper presents a new spectral clustering approach based on subspace randomization and graph fusion (SC-SRGF) for high-dimensional data. In particular, a set of random subspaces are first generated by performing random sampling on the original feature space. Then, multiple K-nearest neighbor (K-NN) affinity graphs are constructed to capture the local structures in the generated subspaces. To fuse the multiple affinity graphs from multiple subspaces, an iterative similarity network fusion scheme is utilized to achieve a unified graph for the final spectral clustering. Experiments on twelve real-world high-dimensional datasets demonstrate the superiority of the proposed approach. The MATLAB source code is available at https://www.researchgate.net/publication/338864134.
Towards context-aware collaborative filtering by learning context-aware latent representations

Contexts have been proven to be an important source of information that can significantly improve the performance of collaborative filtering (CF), e.g., for recommendation. Most context-aware approaches that are basing on latent factor models assume that contexts share the same latent space with users and items. However such a strategy does not always make sense, e.g., the influence of contextual information may be overestimated. In this paper, we propose a generic framework to learn context-aware latent representations for context-aware collaborative filtering without imposing contexts into latent space of users and items. Contextual contents are combined via a function to produce the contextual influence factor, which is then combined with each latent factor to derive latent representations. We instantiate the generic framework using biased Matrix Factorization for rating prediction task and Bayesian Personalized Ranking (BPR) for item recommendation tasks. Stochastic Gradient Descent (SGD) based optimization procedures are developed to fit the two context-aware models by jointly learning the weight of each context and latent factors of users and items. Experiments conducted on three real-world datasets demonstrate that our context-aware CF model significantly outperforms not only the base models but also the representative context-aware models.
Architectural and Behavioral Analysis for Cyber Security

The asymmetric nature and ever-increasing degree of sophistication of cyber threats drive the need for assurance of critical infrastructure and systems. Current approaches that can help counter these cyber threats include the application of tools with the ability to analyze system behavior in its most general form and in the presence of wide classes of threats-at design time. In this paper we describe our tool for incorporating cyber security resiliency analysis and recommendations in the system design process that are automated, scalable, provide rich feedback, specify trade-offs and are easy to use by system architects. The architecture models and design knowledge are captured in formalisms that are expressive and amenable to automated reasoning, analysis, and analysis for cyber security. The two main components developed are Model-Based Architectural Analysis and Cyber-resiliency Verifier. The Model-Based Architectural Analysis identifies threats against the architecture and the potential tradeoffs of their mitigations. Threats are identified from attack patterns from Mitre's Common Attack Pattern Enumeration and Classification (CAPEC) and defenses are suggested based on controls from NIST's Security and Privacy Controls list. After the threats and defenses are identified, an attack-defense tree is generated along with cutsets that describe attack scenarios and associated defenses. The identified threats as well as the attack-defense trees are visualized to provide a concise and informative view of the analysis. There may be multiple ways of addressing a threat and the system architect can decide which defense to implement. After the architecture has been revised to be secure, we then consider cyber-resiliency behavioral properties of the system. For this we abstract threat models in terms of an instrumentor that incorporates the effects of the threats. This allows us to aggregate classes of threats with the same effect so that they can be addressed at the effects level as opposed to treating them individually. The cyber-resiliency properties are then verified by an extension of the proven model checker Kind 2 which identifies the system components responsible for a property violation and provides localized diagnostic information. This enables the system architect to rework the vulnerable portions of the design to discharge required resiliency obligations. In case of resiliency property violations, the tool can recommend placement of runtime monitors sufficient to discharge specific proof obligations. Finally, our tool automatically generates test cases and procedure to check the conformance between the design and implementation.
Vehicle detection based on remote sensing image of Yolov3

The recognition and classification of remote sensing satellite images have unique value and significance in many fields. Based on the yolov3 deep learning neural network, this paper first adjusts the network model to make it suitable for small target detection on remote sensing images, then uses k-means algorithm to calculate the grid size of Yolo network model suitable for vehicles, and then uses yolov3 to train the data which is the satellite remote sensing image set, The network model for vehicle detection in the satellite remote sensing images is obtained, and the model is tested. Finally, the vehicle detection model suitable for remote sensing image is obtained. Through the research and analysis of the experimental results, it can be seen that this method can effectively detect the vehicles in the remote sensing image and ensure the high detection accuracy of the experimental results.
On the Versatility of Open Logical Relations

Abstract Logical relations are one among the most powerful techniques in the theory of programming languages, and have been used extensively for proving properties of a variety of higher-order calculi. However, there are properties that cannot be immediately proved by means of logical relations, for instance program continuity and differentiability in higher-order languages extended with real-valued functions. Informally, the problem stems from the fact that these properties are naturally expressed on terms of non-ground type (or, equivalently, on open terms of base type), and there is no apparent good definition for a base case (i.e. for closed terms of ground types). To overcome this issue, we study a generalization of the concept of a logical relation, called open logical relation , and prove that it can be fruitfully applied in several contexts in which the property of interest is about expressions of first-order type. Our setting is a simply-typed $$\lambda $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>λ</mml:mi> </mml:math> -calculus enriched with real numbers and real-valued first-order functions from a given set, such as the one of continuous or differentiable functions. We first prove a containment theorem stating that for any collection of real-valued first-order functions including projection functions and closed under function composition, any well-typed term of first-order type denotes a function belonging to that collection. Then, we show by way of open logical relations the correctness of the core of a recently published algorithm for forward automatic differentiation. Finally, we define a refinement-based type system for local continuity in an extension of our calculus with conditionals, and prove the soundness of the type system using open logical relations.
Multi-USV System Cooperative Underwater Target Search Based on Reinforcement Learning and Probability Map

Unmanned surface vehicle (USV) is a robotic system with autonomous planning, driving, and navigation capabilities. With the continuous development of applications, the missions faced by USV are becoming more and more complex, so it is difficult for a single USV to meet the mission requirements. Compared with a single USV, a multi-USV system has some outstanding advantages such as fewer perceptual constraints, larger operation ranges, and stronger operation capability. In the search mission about multiple stationary underwater targets by a multi-USV system in the environment with obstacles, we propose a novel cooperative search algorithm (CSBDRL) based on reinforcement learning (RL) method and probability map method. CSBDRL is composed of the environmental sense module and policy module, which are organized by the “divide and conquer” policy-based architecture. The environmental sense module focuses on providing environmental sense values by using the probability map method. The policy module focuses on learning the optimal policy by using RL method. In CSBDRL, the mission environment is modeled and the corresponding reward function is designed to effectively explore the environment and learning policies. We test CSBDRL in the simulation environment and compare it with other methods. The results prove that compared with other methods, CSBDRL makes the multi-USV system have a higher search efficiency, which can ensure targets are found more quickly and accurately while ensuring the USV avoids obstacles in time during the mission.
Practical Card-Based Protocol for Three-Input Majority

We present a card-based protocol for computing a three-input majority using six cards. The protocol essentially consists of performing a simple XOR protocol two times. Compared to the existing protocols, our protocol does not require private operations other than choosing cards.
High-Dimensional Frequency-Encoded Quantum Information Processing with Passive Photonics and Time-Resolving Detection

In this Letter, we propose a new approach to process high-dimensional quantum information encoded in a photon frequency domain. In contrast to previous approaches based on nonlinear optical processes, no active control of photon energy is required. Arbitrary unitary transformation and projection measurement can be realized with passive photonic circuits and time-resolving detection. A systematic circuit design for a quantum frequency comb with arbitrary size has been given. The criteria to verify quantum frequency correlation has been derived. By considering the practical condition of the detector's finite response time, we show that high-fidelity operation can be readily realized with current device performance. This work will pave the way towards scalable and high-fidelity quantum information processing based on high-dimensional frequency encoding.
Application Chang's Extent Analysis Method for Ranking Barriers in the E-Learning Model Based on Multi-Stakeholder Decision Making

E-learning success model is a crucial topic, especially in the context of the Fourth Industry with artificial intelligence and virtual reality in developing countries.In practice, it is a combination of technology and education; human and system form an integrated digital learning world.E-learning has many outstanding advantages in providing flexible training and equal education to everyone, which has dramatically changed the self-study process due to the ability to customize and effectively meet the learners' demands.However, e-learning is still inferior to traditional learning and teaching in recent years.This paper explores the key barriers to e-learning success based on multi-stakeholder perspectives.The Chang's Extent Analysis Method under fuzzy logic was applied to rank the components of e-learning success model in Vietnam by defining the key barriers to its development.The incorporation of fuzzy set theory in the Analytical Hierarchical Process is superior in representing human judgments.Seven criteria were selected after justifying the experts' opinions in the in-depth interviews.The findings revealed the important order of infrastructure and technology, human resources, setting, evaluation, management and execution, support and pedagogical aspect.The results are significant for the proper action plan to promote e-learning development under the resource constraint.