Miracle-dz/UniFAR-FTUs huggingface dataset:
Contrasting Centralized and Decentralized Critics in Multi-Agent Reinforcement Learning

Centralized Training for Decentralized Execution, where agents are trained offline using centralized information but execute in a decentralized manner online, has gained popularity in the multi-agent reinforcement learning community. In particular, actor-critic methods with a centralized critic and decentralized actors are a common instance of this idea. However, the implications of using a centralized critic in this context are not fully discussed and understood even though it is the standard choice of many algorithms. We therefore formally analyze centralized and decentralized critic approaches, providing a deeper understanding of the implications of critic choice. Because our theory makes unrealistic assumptions, we also empirically compare the centralized and decentralized critic methods over a wide set of environments to validate our theories and to provide practical advice. We show that there exist misconceptions regarding centralized critics in the current literature and show that the centralized critic design is not strictly beneficial, but rather both centralized and decentralized critics have different pros and cons that should be taken into account by algorithm designers.
Efficient, Fair, and Incentive-Compatible Healthcare Rationing

During the COVID-19 pandemic, fair and efficient rationing of healthcare resources has emerged as an important issue that has been discussed by medical experts, policy-makers, and the general public. We consider a healthcare rationing problem where medical units are to be allocated to patients. Each unit is reserved for one of several categories and the patients have different priorities for the categories. We present an allocation rule that respects the priorities, complies with the eligibility requirements, allocates the largest feasible number of units, and does not incentivize agents to hide that they qualify through a category. Moreover, the rule is polynomial-time computable. To the best of our knowledge, it is the first known rule with the aforementioned properties.
Continuous Monitoring of Blood Pressure with Evidential Regression

Photoplethysmogram (PPG) signal-based blood pressure (BP) estimation is a promising candidate for modern BP measurements, as PPG signals can be easily obtained from wearable devices in a non-invasive manner, allowing quick BP measurement. However, the performance of existing machine learning-based BP measuring methods still fall behind some BP measurement guidelines and most of them provide only point estimates of systolic blood pressure (SBP) and diastolic blood pressure (DBP). In this paper, we present a cutting-edge method which is capable of continuously monitoring BP from the PPG signal and satisfies healthcare criteria such as the Association for the Advancement of Medical Instrumentation (AAMI) and the British Hypertension Society (BHS) standards. Furthermore, the proposed method provides the reliability of the predicted BP by estimating its uncertainty to help diagnose medical condition based on the model prediction. Experiments on the MIMIC II database verify the state-of-the-art performance of the proposed method under several metrics and its ability to accurately represent uncertainty in prediction.
Uncovering the Missing Link between Molecular Electrochemistry and Electrocatalysis: Mechanism of the Reduction of Benzyl Chloride at Silver Cathodes

Centre National de la Recherche Scientifique, the Universite Pierre et Marie Curie, the Ecole Normale Superieure (Unite Mixte de Recherche 8640); Centre National de la Recherche Scientifique, the Universite Pierre et Marie Curie, the Ecole Normale Superieure (Laboratoire International Associe XiamENS); Agence Nationale de la Recherche Chaire d'Excellence "MicroNanoChem" [ANR-10-CHEX-012-01]; National Science Foundation of China [21061120456]; Universite degli Studi di Milano (PUR Funds)
Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud

Point cloud upsampling is vital for the quality of the mesh in three-dimensional reconstruction. Recent research on point cloud upsampling has achieved great success due to the development of deep learning. However, the existing methods regard point cloud upsampling of different scale factors as independent tasks. Thus, the methods need to train a specific model for each scale factor, which is both inefficient and impractical for storage and computation in real applications. To address this limitation, in this article, we propose a novel method called “Meta-PU” to first support point cloud upsampling of arbitrary scale factors with a single model. In the Meta-PU method, besides the backbone network consisting of residual graph convolution (RGC) blocks, a meta-subnetwork is learned to adjust the weights of the RGC blocks dynamically, and a farthest sampling block is adopted to sample different numbers of points. Together, these two blocks enable our Meta-PU to continuously upsample the point cloud with arbitrary scale factors by using only a single model. In addition, the experiments reveal that training on multiple scales simultaneously is beneficial to each other. Thus, Meta-PU even outperforms the existing methods trained for a specific scale factor only.
Quantum Algorithmic Information Theory

This paper extends quantum information theory into the algorithmic sphere, using randomness and information conservation inequalities. We show for an overwhelming majority of pure states, the self classical algorithmic information induced by a measurement will be negligible. Puriﬁcation of two states increases information. Decoherence causes quantum states to lose information with all other quantum states.
Deep Reinforcement Learning with Dynamic Optimism

In recent years, deep off-policy actor-critic algorithms have become a dominant approach to reinforcement learning for continuous control. This comes after a series of breakthroughs to address function approximation errors, which previously led to poor performance. These insights encourage the use of pessimistic value updates. However, this discourages exploration and runs counter to theoretical support for the efﬁcacy of optimism in the face of uncertainty. So which approach is best? In this work, we show that the optimal degree of optimism can vary both across tasks and over the course of learning. Inspired by this insight, we introduce a novel deep actor-critic algo-rithm, Dynamic Optimistic and Pessimistic Estimation (DOPE) to switch between optimistic and pessimistic value learning online by formulating the selection as a multi-arm bandit problem. We show in a series of challenging continuous control tasks that DOPE outperforms existing state-of-the-art methods, which rely on a ﬁxed degree of optimism. Since our changes are simple to implement, we believe these insights can be extended to a number of off-policy algorithms.
Heptylprodigiosin induces apoptosis in Jurkat leukemia T cells via CD95 death receptor and evades anti- apoptotic Bcl-2 and Bcl-x L proteins

metabolite isolated from a marine bacterium Pseudovibrio denitrificans strain Z143-1. In this study, we investigated the mechanism by which HPDG induces cytotoxicity in human leukemia Jurkat T cells. HPDG-induced apoptosis (classic apoptosis) was shown by G0/G1 arrest and DNA fragmentation that is concentrationand time-dependent in CD95 (Fas/Apo-1)sensitive Jurkat T cells (S-Jurkat). The MTT cytotoxicity assay is consistent with the apoptosis assay showing an IC50 of ~2 μM in S-Jurkat T cells. HPDG-mediated apoptosis in S-Jurkat T cells is dependent on caspase activation based on a caspase 3-like activity assay and the time-dependent immunoblot analysis of caspases. Flow cytometry analysis of CD95-deficient JurkatR, H Jurkat/FADD-/-, Jurkat/caspase 8-/-, and vector control cells (Jurkat/A3), showed that HPDG-induced apoptosis requires the CD95 receptor, FADD, and caspase 8. Immunoblot analysis and a mitochondrial dissipation assay revealed delayed activation of the intrinsic pathway. HPDG-treated Jurkat/Neo, Jurkat/Bcl-2+/+ and Jurkat/Bcl-xL+/+ cells showed that HPDG-induced apoptosis is independent of Bcl-2 and Bcl-xL overexpression, suggesting that HPDG is unaffected by the protective mechanism of these anti-apoptotic proteins in Jurkat T cells. This is the first report of the apoptotic mechanism of HPDG that may have clinical utility as a potent chemosensitizer to tumor cells with CD95 expression, dysfunctional apoptosome activity, and overexpressed Bcl-2 and Bcl-xL proteins.
Propagation Kinetics in Radical Polymerization Studied via Pulsed Laser Techniques

Pulsed laser techniques have enormously improved the quality by which rate coefficients of radical polymerization may be determined. The specific and most important feature of the various types of pulsed laser techniques consists in the almost instantaneous production of a significant radical concentration and in the ease by which radical concentration may be controlled by applying further laser pulses at pre-selected delay times. The pulsed laser polymerization – size exclusion chromatography (PLP-SEC) experiment is extremely valuable for the determination of reliable propagation rate coefficients, kp. The present article briefly reviews recent results from PLP-SEC studies directed toward the solvent dependence of kp in aqueous systems, into kp in copolymerization reactions, and into measurements of propagation rate coefficients in systems containing two types of radicals, as is the case with acrylates, where secondary chain-end radicals may undergo 1,5-H shift reactions (backbiting steps) to produce tertiary midchain radicals. By a propagation step, the tertiary radicals are transformed back to secondary ones.
State-Aware Variational Thompson Sampling for Deep Q-Networks

Thompson sampling is a well-known approach for balancing exploration and exploitation in reinforcement learning. It requires the posterior distribution of value-action functions to be maintained; this is generally intractable for tasks that have a high dimensional state-action space. We derive a variational Thompson sampling approximation for DQNs which uses a deep network whose parameters are perturbed by a learned variational noise distribution. We interpret the successful NoisyNets method \cite{fortunato2018noisy} as an approximation to the variational Thompson sampling method that we derive. Further, we propose State Aware Noisy Exploration (SANE) which seeks to improve on NoisyNets by allowing a non-uniform perturbation, where the amount of parameter perturbation is conditioned on the state of the agent. This is done with the help of an auxiliary perturbation module, whose output is state dependent and is learnt end to end with gradient descent. We hypothesize that such state-aware noisy exploration is particularly useful in problems where exploration in certain \textit{high risk} states may result in the agent failing badly. We demonstrate the effectiveness of the state-aware exploration method in the off-policy setting by augmenting DQNs with the auxiliary perturbation module.
Estimate The Efficiency Of Multiprocessor’s Cash Memory Work Algorithms

Many computer systems for calculating the proper organization of memory are among the most critical issues. Using a tier cache memory (along with branching prediction) is an effective means of increasing modern multi-core processors' performance. Designing high-performance processors is a complex task and requires preliminary verification and analysis of the model level, usually used in analytical and simulation modeling. The refinement of extreme programming is an unfortunate challenge. Few experts disagree with the synthesis of access points. This article demonstrates that Internet QoS and 16-bit architectures are always incompatible, but it's the same situation for write-back caches. The solution to this problem can be implemented by analyzing simulation models of different complexity in combination with the analytical evaluation of individual algorithms. This work is devoted to designing a multi-parameter simulation model of a multi-process for evaluating the performance of cache memory algorithms and the optimality of the structure. Optimization of the structures and algorithms of the cache memory allows you to accelerate the interaction of the memory process and improve the performance of the entire system.
Unruh effect of detectors with quantized center of mass

The Unruh eﬀect is the prediction that particle detectors accelerated through the vacuum get excited by the apparent presence of radiation quanta — a fundamental quantum phenomenon in the presence of acceleration. Prior treatments of the Unruh eﬀect, that presume a classically prescribed trajectory, do not account for the quantum dynamics of the detector’s center-of-mass. Here, we study more realistic detectors whose center of mass is a quantized degree of freedom being accelerated by an external classical ﬁeld. We investigate the detector’s recoil due to the emission of Unruh quanta. Vice versa, we also study the recoil’s impact on the emission of Unruh quanta and the excitation of the detector. We ﬁnd that the recoil due to the emission of Unruh quanta may be a relevant experimental signature of the Unruh eﬀect.
Humanacgan: Conditional Generative Adversarial Network with Human-Based Auxiliary Classifier and its Evaluation in Phoneme Perception

We propose a conditional generative adversarial network (GAN) incorporating humans’ perceptual evaluations. A deep neural network (DNN)-based generator of a GAN can represent a real-data distribution accurately but can never represent a human-acceptable distribution, which are ranges of data in which humans accept the naturalness regardless of whether the data are real or not. A Human-GAN was proposed to model the human-acceptable distribution. A DNN-based generator is trained using a human-based discriminator, i.e., humans’ perceptual evaluations, instead of the GAN’s DNN-based discriminator. However, the HumanGAN cannot represent conditional distributions. This paper proposes the HumanACGAN, a theoretical extension of the HumanGAN, to deal with conditional human-acceptable distributions. Our HumanACGAN trains a DNN-based conditional generator by regarding humans as not only a discriminator but also an auxiliary classifier. The generator is trained by deceiving the human-based discriminator that scores the unconditioned naturalness and the human-based classifier that scores the class-conditioned perceptual acceptability. The training can be executed using the backpropagation algorithm involving humans’ perceptual evaluations. Our experimental results in phoneme perception demonstrate that our HumanACGAN can successfully train this conditional generator.
Design of Dynamic Experiments for Black-Box Model Discrimination

Diverse domains of science and engineering require and use mechanistic mathematical models, e.g. systems of differential algebraic equations. Such models often contain uncertain parameters to be estimated from data. Consider a dynamic model discrimination setting where we wish to chose: (i) what is the best mechanistic, time-varying model and (ii) what are the best model parameter estimates. These tasks are often termed model discrimination/selection/validation/verification. Typically, several rival mechanistic models can explain data, so we incorporate available data and also run new experiments to gather more data. Design of dynamic experiments for model discrimination helps optimally collect data. For rival mechanistic models where we have access to gradient information, we extend existing methods to incorporate a wider range of problem uncertainty and show that our proposed approach is equivalent to historical approaches when limiting the types of considered uncertainty. We also consider rival mechanistic models as dynamic black boxes that we can evaluate, e.g. by running legacy code, but where gradient or other advanced information is unavailable. We replace these black-box models with Gaussian process surrogate models and thereby extend the model discrimination setting to additionally incorporate rival black-box model. We also explore the consequences of using Gaussian process surrogates to approximate gradient-based methods.
Intensity-SLAM: Intensity Assisted Localization and Mapping for Large Scale Environment

Simultaneous Localization And Mapping (SLAM) is a task to estimate the robot location and to reconstruct the environment based on observation from sensors such as LIght Detection And Ranging (LiDAR) and camera. It is widely used in robotic applications such as autonomous driving and drone delivery. Traditional LiDAR-based SLAM algorithms mainly leverage the geometric features from the scene context, while the intensity information from LiDAR is ignored. Some recent deep-learning-based SLAM algorithms consider intensity features and train the pose estimation network in an end-to-end manner. However, they require significant data collection effort and their generalizability to environments other than the trained one remains unclear. In this letter we introduce intensity features to a SLAM system. And we propose a novel full SLAM framework that leverages both geometry and intensity features. The proposed SLAM involves both intensity-based front-end odometry estimation and intensity-based back-end optimization. Thorough experiments are performed including both outdoor autonomous driving and indoor warehouse robot manipulation. The results show that the proposed method outperforms existing geometric-only LiDAR SLAM methods.
An isolated logarithmic layer

Abstract To isolate the multiscale dynamics of the logarithmic layer of wall-bounded turbulent flows, a novel numerical experiment is conducted in which the mean tangential Reynolds stress is eliminated except in a subregion corresponding to the typical location of the logarithmic layer in channels. Various statistical comparisons against channel flow databases show that, despite some differences, this modified flow system reproduces the kinematics and dynamics of natural logarithmic layers well, even in the absence of a buffer and an outer zone. This supports the previous idea that the logarithmic layer has its own autonomous dynamics. In particular, the results suggest that the mean velocity gradient and the wall-parallel scale of the largest eddies are determined by the height of the tallest momentum-transferring motions, which implies that the very large-scale motions of wall-bounded flows are not an intrinsic part of the logarithmic-layer dynamics. Using a similar set-up, an isolated layer with a constant total stress, which represents the logarithmic layer without a driving force, is simulated and examined.
MSPM: A modularized and scalable multi-agent reinforcement learning-based system for financial portfolio management

Financial portfolio management (PM) is one of the most applicable problems in reinforcement learning (RL) owing to its sequential decision-making nature. However, existing RL-based approaches rarely focus on scalability or reusability to adapt to the ever-changing markets. These approaches are rigid and unscalable to accommodate the varying number of assets of portfolios and increasing need for heterogeneous data input. Also, RL agents in the existing systems are ad-hoc trained and hardly reusable for different portfolios. To confront the above problems, a modular design is desired for the systems to be compatible with reusable asset-dedicated agents. In this paper, we propose a multi-agent RL-based system for PM (MSPM). MSPM involves two types of asynchronously-updated modules: Evolving Agent Module (EAM) and Strategic Agent Module (SAM). An EAM is an information-generating module with a Deep Q-network (DQN) agent, and it receives heterogeneous data and generates signal-comprised information for a particular asset. An SAM is a decision-making module with a Proximal Policy Optimization (PPO) agent for portfolio optimization, and it connects to multiple EAMs to reallocate the corresponding assets in a financial portfolio. Once been trained, EAMs can be connected to any SAM at will, like assembling LEGO blocks. With its modularized architecture, the multi-step condensation of volatile market information, and the reusable design of EAM, MSPM simultaneously addresses the two challenges in RL-based PM: scalability and reusability. Experiments on 8-year U.S. stock market data prove the effectiveness of MSPM in profit accumulation by its outperformance over five different baselines in terms of accumulated rate of return (ARR), daily rate of return (DRR), and Sortino ratio (SR). MSPM improves ARR by at least 186.5% compared to constant rebalanced portfolio (CRP), a widely-used PM strategy. To validate the indispensability of EAM, we back-test and compare MSPMs on four different portfolios. EAM-enabled MSPMs improve ARR by at least 1341.8% compared to EAM-disabled MSPMs.
Academic Source Code Plagiarism Detection by Measuring Program Behavioral Similarity

Source code plagiarism is a long-standing issue in tertiary computer science education. Many source code plagiarism detection tools have been proposed to aid in the detection of source code plagiarism. However, existing detection tools are not robust to pervasive plagiarism-hiding transformations and can be inaccurate in the detection of plagiarised source code. This article presents BPlag, a behavioural approach to source code plagiarism detection. BPlag is designed to be both robust to pervasive plagiarism-hiding transformations and accurate in the detection of plagiarised source code. Greater robustness and accuracy is afforded by analyzing the behavior of a program, as behavior is perceived to be the least susceptible aspect of a program impacted upon by plagiarism-hiding transformations. BPlag applies symbolic execution to analyses execution behavior and represents a program in a novel graph-based format. Plagiarism is then detected by comparing these graphs and evaluating similarity scores. BPlag is evaluated for robustness, accuracy and efficiency against five commonly used source code plagiarism detection tools. It is then shown that BPlag is more robust to plagiarism-hiding transformations and more accurate in the detection of plagiarised source code, but is less efficient than the compared tools.
Normal Cayley digraphs of cyclic groups with CI-property

Abstract A Cayley (di)graph of a group G is called normal if the right regular representation of G is normal in the full automorphism group of and a CI-(di)graph if for every Cayley (di)graph implies that there is such that We call a group G an NDCI-group or NCI-group if all normal Cayley digraphs or graphs of G are CI-digraphs or CI-graphs, respectively. We prove that a cyclic group of order n is an NDCI-group if and only if and an NCI-group if and only if either n = 8 or
Deep optical cooling of coupled nuclear spin-spin and quadrupole reservoirs in a GaAs/(Al,Ga)As quantum well

The selective cooling of $^{75}$As spins by optical pumping followed by adiabatic demagnetization in the rotating frame is realized in a nominally undoped GaAs/(Al,Ga)As quantum well. The rotation of 6 kG strong Overhauser field at the $^{75}$As Larmor frequency of 5.5 MHz is evidenced by the dynamic Hanle effect. Despite the presence of the quadrupole induced nuclear spin splitting, it is shown that the rotating $^{75}$As magnetization is uniquely determined by the spin temperature of coupled spin-spin and quadrupole reservoirs. The dependence of heat capacity of these reservoirs on the external magnetic field direction with respect to crystal and structure axes is investigated. The lowest nuclear spin temperature achieved is 0.54 $\mu$K, which is the record low value for semiconductors and semiconductor nanostructures.