Miracle-dz/UniFAR-FTUs huggingface dataset:
Sample Efficiency in Sparse Reinforcement Learning: Or Your Money Back

Sparse rewards present a difficult problem in reinforcement learning and may be inevitable in certain domains with complex dynamics such as real-world robotics. Hindsight Experience Replay (HER) is a recent replay memory development that allows agents to learn in sparse settings by altering memories to show them as successful even though they may not be. While, empirically, HER has shown some success, it does not provide guarantees around the makeup of samples drawn from an agent's replay memory. This may result in minibatches that contain only memories with zero-valued rewards or agents learning an undesirable policy that completes HER-adjusted goals instead of the actual goal. In this paper, we introduce Or Your Money Back (OYMB), a replay memory sampler designed to work with HER. OYMB improves training efficiency in sparse settings by providing a direct interface to the agent's replay memory that allows for control over minibatch makeup, as well as a preferential lookup scheme that prioritizes real-goal memories before HER-adjusted memories. We test our approach on five tasks across three unique environments. Our results show that using HER in combination with OYMB outperforms using HER alone and leads to agents that learn to complete the real goal more quickly.
The wide-field photometric system of the Nanshan One-meter Telescope

The Nanshan One-meterWide-field Telescope is a prime focus system that is located at Nanshan Station of Xinjiang Astronomical Observatories. The field of view was designed to 1.5° × 1.5°, and the Johnson-Cousins UBVRI system was chosen as the main filter set. The telescope has been providing observation services for astronomers since September 2013. Variable source searching and time-domain surveys are the main scientific goals. The system’s test results are reported including linearity, dark current, bias, readout noise and gain of the CCD camera. The accurate instrumental calibration coefficients in UBVRI bands were driven with Landolt standard stars during photometric nights. Finally, the limiting magnitudes are given with signal-to-noise ratios and various exposure times for observers.
Temporal variation in microhabitat use and spatial relations in the benthic fish community of a stream

-Patterns of microhabitat use and overlap among five benthic fish species in a fifth-order stream were investigated during two same-year sampling periods characterized by high vs. low discharges. Microhabitat use patterns for Cottus carolinae (banded sculpin), Etheostoma bellum (orangefin darter), E. blennioides (greenside darter), E. maculatum (spotted darter) and E. zonale (banded darter) were similar to literature reports. During high flow, most segregation occurred along a depth/substrate roughness gradient while in low flows segregation was more common along a velocity gradient. Less segregation in microhabitat use during low flow was probably due to differences in spatial overlap. Five species pairs exhibited high spatial overlap (>0.60) during high flow, whereas one pair exhibited this pattern during low flow. Overall, there was a decrease in spatial breadth (13.91 vs. 10.57) and overlap (0.52 vs. 0.36) of microhabitat use during the low flow period.
Collapsing Geometry with Ricci Curvature Bounded Below and Ricci Flow Smoothing

We survey some recent developments in the study of collapsing Riemannian manifolds with Ricci curvature bounded below, especially the locally bounded Ricci covering geometry and the Ricci flow smoothing techniques. We then prove that if a Calabi-Yau manifold is sufficiently volume collapsed with bounded diameter and sectional curvature, then it admits a Ricci-flat Kähler metrictogether with a compatible pure nilpotent Killing structure: this is related to an open question of Cheeger, Fukaya and Gromov.
Revisiting dynamics of quantum causal structures -- when can causal order evolve?

Recently, there has been substantial interest in studying the dynamics of quantum theory beyond that of states, in particular, the dynamics of channels, measurements, and higher-order transformations. Ref. [Phys. Rev. X 8.1 (2018): 011047] pursues this using the process matrix formalism, together with a definition of the possible dynamics of such process matrices, and focusing especially on the question of evolution of causal structures. One of its major conclusions is a strong theorem saying that, within the formalism, under continuous and reversible transformations, the causal order between operations must be preserved. Here we find a surprising result: if one is to take into account a full picture of the physical evolution of operations within the standard quantum-mechanical formalism, then one can actually draw the opposite conclusion. That is, we show that under certain continuous and reversible dynamics the causal order between operations is not necessarily preserved. We moreover identify and analyse the root of this apparent contradiction, specifically, that the commonly accepted and widely applied framework of higher-order processes, whilst mathematically sound, is not always the appropriate one for describing interesting physical dynamics. Finally we show how to reconcile the elements of the whole picture following the intuition based on entanglement processing by local operations and classical communication.
Deep geometric representations for modeling effects of mutations on protein-protein binding affinity

Modeling the impact of amino acid mutations on protein-protein interaction plays a crucial role in protein engineering and drug design. In this study, we develop GeoPPI, a novel structure-based deep-learning framework to predict the change of binding affinity upon mutations. Based on the three-dimensional structure of a protein, GeoPPI first learns a geometric representation that encodes topology features of the protein structure via a self-supervised learning scheme. These representations are then used as features for training gradient-boosting trees to predict the changes of protein-protein binding affinity upon mutations. We find that GeoPPI is able to learn meaningful features that characterize interactions between atoms in protein structures. In addition, through extensive experiments, we show that GeoPPI achieves new state-of-the-art performance in predicting the binding affinity changes upon both single- and multi-point mutations on six benchmark datasets. Moreover, we show that GeoPPI can accurately estimate the difference of binding affinities between a few recently identified SARS-CoV-2 antibodies and the receptor-binding domain (RBD) of the S protein. These results demonstrate the potential of GeoPPI as a powerful and useful computational tool in protein design and engineering. Our code and datasets are available at: https://github.com/Liuxg16/GeoPPI.
Vehicle Telematics Via Exteroceptive Sensors: A Survey

Whereas a very large number of sensors are available in the automotive field, currently just a few of them, mostly proprioceptive ones, are used in telematics, automotive insurance, and mobility safety research. In this paper, we show that exteroceptive sensors, like microphones or cameras, could replace proprioceptive ones in many fields. Our main motivation is to provide the reader with alternative ideas for the development of telematics applications when proprioceptive sensors are unusable for technological issues, privacy concerns, or lack of availability in commercial devices. We first introduce a taxonomy of sensors in telematics. Then, we review in detail all exteroceptive sensors of some interest for vehicle telematics, highlighting advantages, drawbacks, and availability in off-the-shelf devices. Successively, we present a list of notable telematics services and applications in research and industry like driving profiling or vehicular safety. For each of them, we report the most recent and important works relying on exteroceptive sensors, as long as the available datasets. We conclude showing open challenges using exteroceptive sensors both for industry and research.
Improving the segmentation of scanning probe microscope images using convolutional neural networks

A wide range of techniques can be considered for segmentation of images of nanostructured surfaces. Manually segmenting these images is time-consuming and results in a user-dependent segmentation bias, while there is currently no consensus on the best automated segmentation methods for particular techniques, image classes, and samples. Any image segmentation approach must minimise the noise in the images to ensure accurate and meaningful statistical analysis can be carried out. Here we develop protocols for the segmentation of images of 2D assemblies of gold nanoparticles formed on silicon surfaces via deposition from an organic solvent. The evaporation of the solvent drives far-from-equilibrium self-organisation of the particles, producing a wide variety of nano- and micro-structured patterns. We show that a segmentation strategy using the U-Net convolutional neural network has some benefits over traditional automated approaches and has particular potential in the processing of images of nanostructured systems.
Meta Reinforcement Learning-Based Lane Change Strategy for Autonomous Vehicles

The field of autonomous driving has seen increasing proposed use of machine learning methodologies. However, there are still challenges in applying such methods since autonomous driving involves complex and dynamic interactions with the environment. Supervised learning algorithms such as imitation learning can work in environments represented in the training data set, however, it is impractical or cost-prohibitive to collect data for all possible environments. Reinforcement learning methods can train the agent through trial and error, but it may still fail in new environments. To overcome these shortcomings, we thus propose a meta reinforcement learning (MRL) method to improve the agent's generalization capabilities to new environments. The method is applied to automated lane-changing maneuvers at different traffic congestion levels. Specifically, we train the model at light to moderate traffic conditions under a reinforcement learning framework, and then test it at heavy traffic conditions that are never encountered during training. For performance evaluation, we use both collision rate and success rate of the lane-change maneuvers to quantify the safety and effectiveness of the proposed model. A pretrained model is established as a benchmark, which uses the same network structure and training tasks as our proposed model for fair comparison. Simulation results show that the proposed method achieves an overall success rate up to 20% higher than the benchmark model when it is generalized to the new environment of heavy traffic density. The collision rate is also reduced by up to 18% compared with the benchmark model. Furthermore, the proposed model shows more stable and efficient adaptation capabilities and can achieve 100% successful rate and 0% collision rate with only a few steps of gradient update adaptation.
Graph Embedding for Combinatorial Optimization: A Survey

Graphs have been widely used to represent complex data in many applications, such as e-commerce, social networks, and bioinformatics. Efficient and effective analysis of graph data is important for graph-based applications. However, most graph analysis tasks are combinatorial optimization (CO) problems, which are NP-hard. Recent studies have focused a lot on the potential of using machine learning (ML) to solve graph-based CO problems. Using ML- based CO methods, a graph has to be represented in numerical vectors, which is known as graph embedding. In this survey, we provide a thorough overview of recent graph embedding methods that have been used to solve CO problems. Most graph embedding methods have two stages: graph preprocessing and ML model learning. This survey classifies graph embedding works from the perspective of graph preprocessing tasks and ML models. Furthermore, this survey summarizes recent graph-based CO methods that exploit graph embedding. In particular, graph embedding can be employed as part of classification techniques or can be combined with search methods to find solutions to CO problems. The survey ends with several remarks on future research directions.
Differentiation potentials of two stroma-resident tissue-specific stem cells

Objective: Several studies have demonstrated that human dental pulp from deciduous teeth and human umbilical cord stroma contains tissue-specific stem cells having varying potentials to differentiate into certain lineages. Both are multipotent, ethically available in large numbers, immunocompatible and developmentally primitive. Also, ease of isolation and high expansion potential in vitro have made them very attractive as a model system for many researchers. However, there are many controversies still existing concerning their functional roles. A comparison of their patterns to multilineage differentiation might give a clear idea for their further usage in tissue engineering.
Comparative Study of a Critical Behavior of a Coupled Spin-Electron Model on a Doubly Decorated Square Lattice in the Canonical and Grand-Canonical Ensemble

The critical behavior of a hybrid spin-electron model with localized Ising spins placed on nodal sites and mobile electrons delocalized over bonds between two nodal lattice sites is analyzed by the use of a generalized decoration-iteration transformation. Our attention is primarily concentrated on a rigorous analysis of a critical temperature in canonical and grand-canonical statistical ensemble at two particular electron concentrations, corresponding to a quarter (ρ=1) and a half (ρ = 2) filled case. It is found that the critical temperature of the investigated spin-electron system in the canonical and grand-canonical ensemble may be remarkably different and is very sensitive to the competition among the model parameters like the electron hopping amplitude (t), the Ising coupling between the localized spins (J ′), the electrostatic potential (V ) and the electron concentration (ρ). In addition, it is detected that the increasing electrostatic potential has a reduction effect upon the deviation between critical temperatures in both statistical ensembles.
Simulation-supervised deep learning for analysing organelles states and behaviour in living cells

In many real-world scientific problems, generating ground truth (GT) for supervised learning is almost impossible. The causes include limitations imposed by scientific instrument, physical phenomenon itself, or the complexity of modeling. Performing artificial intelligence (AI) tasks such as segmentation, tracking, and analytics of small sub-cellular structures such as mitochondria in microscopy videos of living cells is a prime example. The 3D blurring function of microscope, digital resolution from pixel size, optical resolution due to the character of light, noise characteristics, and complex 3D deformable shapes of mitochondria, all contribute to making this problem GT hard. Manual segmentation of 100s of mitochondria across 1000s of frames and then across many such videos is not only herculean but also physically inaccurate because of the instrument and phenomena imposed limitations. Unsupervised learning produces less than optimal results and accuracy is important if inferences relevant to therapy are to be derived. In order to solve this unsurmountable problem, we bring modeling and deep learning to a nexus. We show that accurate physics based modeling of microscopy data including all its limitations can be the solution for generating simulated training datasets for supervised learning. We show here that our simulation-supervised segmentation approach is a great enabler for studying mitochondrial states and behaviour in heart muscle cells, where mitochondria have a significant role to play in the health of the cells. We report unprecedented mean IoU score of 91% for binary segmentation (19% better than the best performing unsupervised approach) of mitochondria in actual microscopy videos of living cells. We further demonstrate the possibility of performing multi-class classification, tracking, and morphology associated analytics at the scale of individual mitochondrion.
Detection of Interstellar HC4NC and an Investigation of Isocyanopolyyne Chemistry under TMC-1 Conditions

We report an astronomical detection of HC4NC for the first time in the interstellar medium with the Green Bank Telescope toward the TMC-1 molecular cloud with a minimum significance of 10. 5σ. The total column density and excitation temperature of HC4NC are determined to be cm−2 and , respectively, using Markov chain Monte Carlo analysis. In addition to HC4NC, HCCNC is distinctly detected whereas no clear detection of HC6NC is made. We propose that the dissociative recombination of the protonated cyanopolyyne, HC5NH+, and the protonated isocyanopolyyne, HC4NCH+, are the main formation mechanisms for HC4NC while its destruction is dominated by reactions with simple ions and atomic carbon. With the proposed chemical networks, the observed abundances of HC4NC and HCCNC are reproduced satisfactorily.
Control On the Manifolds Of Mappings As a Setting For Deep Learning

We use a control-theoretic setting to model the process of training (deep learning) of Artificial Neural Networks (ANN), which are aimed at solving classification problems. A successful classifier is the network whose input-output map approximates well the classifying map defined on a finite or an infinite training set. A fruitful idea is substitution of a multi-layer ANN by a continuous-time control system, which can be seen as a neural network with infinite number of layers. Under certain conditions it can achieve high rate of approximation with presumably not so high computational cost. The problem of best approximation for this model results in optimal control problem of Bolza type for ensembles of points. The two issues to be studied are: i) possibility of a satisfactory approximation of complex classification profiles; ii) finding the values of parameters (controls) which provide the best approximation. In control-theoretic terminology it corresponds respectively to the verification of an ensemble controllability property and to the solution of an ensemble optimal control problem. In the present contribution we concentrate on the first type of problems; our main results include examples of control systems, which are approximately controllable in the groups of diffeomorphisms of $\mathbb{R}^n, \mathbb{T}^n, \mathbb{S}^2$.
Metrics for Exposing the Biases of Content-Style Disentanglement

Recent state-of-the-art semi- and un-supervised solutions for challenging computer vision tasks have used the idea of encoding image content into a spatial tensor and image appearance or "style" into a vector. These decomposed representations take advantage of equivariant properties of network design and improve performance in equivariant tasks, such as image-to-image translation. Most of these methods use the term "disentangled" for their representations and employ model design, learning objectives, and data biases to achieve good model performance. While considerable effort has been made to measure disentanglement in vector representations, currently, metrics that can characterize the degree of disentanglement between content (spatial) and style (vector) representations and the relation to task performance are lacking. In this paper, we propose metrics to measure how (un)correlated, biased, and informative the content and style representations are. In particular, we first identify key design choices and learning constraints on three popular models that employ content-style disentanglement and derive ablated versions. Then, we use our metrics to ascertain the role of each bias. Our experiments reveal a "sweet-spot" between disentanglement, task performance and latent space interpretability. The proposed metrics enable the design of better models and the selection of models that achieve the desired performance and disentanglement. Our metrics library is available at this https URL.
Molecular Characterization of Escherichia coli Isolates from Humans and Chickens in the Chungcheong Area Using MLST Analysis

Antimicrobial resistant bacteria has recently emerged and been disseminated in livestock environments because of excessive use of antimicrobial agents for the therapeutic and growth promotion purposes to food animals. In particular, there is potential for multidrug-resistant bacteria that can be transmitted from animals to mankind. In this study, we investigated the genotypes of E. coli strains isolated from humans and chickens using multi-locus sequence typing (MLST) and antimicrobial resistance patterns by disk diffusion method along with integron study involving antimicrobial resistance mechanisms. From July 2013 to July 2014, E. coli strains isolated from clinical specimens (n=44) and poultry chickens (n=34). ST131 (n=20) was most common in human-derived E. coli. ST752 (n=7) was most common in chicken-derived E. coli, with four isolates each for ST117, ST189, and ST69. Of the 44 E. coli strains isolated from humans, 25 of had a class 1 integron, as opposed to only 11 of 34 strains in the E. coli isolated from chickens. There were differences in genotypes and antimicrobial resistance patterns between the chicken-derived and the human-derived E. coli.
On Modularity in Reactive Control Architectures, with an Application to Formal Verification

Modularity is a central principle throughout the design process for cyber-physical systems. Modularity reduces complexity and increases reuse of behavior. In this article we pose and answer the following question: how can we identify independent “modules” within the structure of reactive control architectures? To this end, we propose a graph-structured control architecture we call a decision structure and show how it generalizes some reactive control architectures that are popular in Artificial Intelligence (AI) and robotics, specifically Teleo-Reactive programs (TRs), Decision Trees (DTs), Behavior Trees (BTs), and Generalised Behavior Trees (k-BTs). Inspired by the definition of a module in graph theory [16] we define modules in decision structures and show how each decision structure possesses a canonical decomposition into its modules, which can be found in polynomial time. We establish intuitive connections between our proposed modularity and modularity in structured programming. In BTs, k-BTs, and DTs the modules we propose are in a one-to-one correspondence with their subtrees. We show we can naturally characterize each of the BTs, k-BTs, DTs, and TRs by properties of their module decomposition. This allows us to recognize which decision structures are equivalent to each of these architectures in quadratic time. Following McCabe [26], we define a complexity measure called essential complexity on decision structures, which measures the degree to which they can be decomposed into simpler modules. We characterize the k-BTs as the decision structures of unit-essential complexity. Our proposed concept of modules extends to formal verification, under any verification scheme capable of verifying a decision structure. Namely, we prove that a modification to a module within a decision structure has no greater flow-on effects than a modification to an individual action within that structure. This enables verification on modules to be done locally and hierarchically, where structures can be verified and then repeatedly locally modified, with modules replaced by modules while preserving correctness. To illustrate the findings, we present an example of a solar-powered drone completing a reconnaissance-based mission using a decision structure. We use a Linear Temporal Logic-based verification scheme to verify the correctness of this structure and then show how one can repeatedly modify modules while preserving its correctness, and this can be verified by considering only those modules that have been modified.