Miracle-dz/UniFAR-FTUs huggingface dataset:
Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency

Recently, several studies (Zhou et al., 2021a; Zhang et al., 2021b; Kim et al., 2021; Zhou and Gu, 2022) have provided variance-dependent regret bounds for linear contextual bandits, which interpolates the regret for the worst-case regime and the deterministic reward regime. However, these algorithms are either computationally intractable or unable to handle unknown variance of the noise. In this paper, we present a novel solution to this open problem by proposing the first computationally efficient algorithm for linear bandits with heteroscedastic noise. Our algorithm is adaptive to the unknown variance of noise and achieves an $\tilde{O}(d \sqrt{\sum_{k = 1}^K \sigma_k^2} + d)$ regret, where $\sigma_k^2$ is the variance of the noise at the round $k$, $d$ is the dimension of the contexts and $K$ is the total number of rounds. Our results are based on an adaptive variance-aware confidence set enabled by a new Freedman-type concentration inequality for self-normalized martingales and a multi-layer structure to stratify the context vectors into different layers with different uniform upper bounds on the uncertainty. Furthermore, our approach can be extended to linear mixture Markov decision processes (MDPs) in reinforcement learning. We propose a variance-adaptive algorithm for linear mixture MDPs, which achieves a problem-dependent horizon-free regret bound that can gracefully reduce to a nearly constant regret for deterministic MDPs. Unlike existing nearly minimax optimal algorithms for linear mixture MDPs, our algorithm does not require explicit variance estimation of the transitional probabilities or the use of high-order moment estimators to attain horizon-free regret. We believe the techniques developed in this paper can have independent value for general online decision making problems.
Potential-based reward shaping for learning to play text-based adventure games

Text-based games are a popular testbed for language-based reinforcement learning (RL). In previous work, deep Q-learning is commonly used as the learning agent. Q-learning algorithms are challenging to apply to complex real-world domains due to, for example, their instability in training. Therefore, in this paper, we adapt the soft-actor-critic (SAC) algo-rithm to the text-based environment. To deal with sparse extrinsic rewards from the environment, we combine it with a potential-based reward shaping technique to provide more informative (dense) reward signals to the RL agent. We apply our method to play difﬁcult text-based games. The SAC method achieves higher scores than the Q-learning methods on many games with only half the number of training steps. This shows that it is well-suited for text-based games. Moreover, we show that the reward shaping technique helps the agent to learn the policy faster and achieve higher scores.
Minimax-Bayes Reinforcement Learning

While the Bayesian decision-theoretic framework offers an elegant solution to the problem of decision making under uncertainty, one question is how to appropriately select the prior distribution. One idea is to employ a worst-case prior. However, this is not as easy to specify in sequential decision making as in simple statistical estimation problems. This paper studies (sometimes approximate) minimax-Bayes solutions for various reinforcement learning problems to gain insights into the properties of the corresponding priors and policies. We find that while the worst-case prior depends on the setting, the corresponding minimax policies are more robust than those that assume a standard (i.e. uniform) prior.
Bayesian subtyping for multi-state brain functional connectome with application on adolescent brain cognition

Converging evidence indicates that the heterogeneity of cognitive profiles may arise through detectable alternations in brain functions. Particularly, brain functional connectivity, measured under resting and cognitive states, characterizes the unique neuronal interconnections across large-scale brain networks. Despite an unprecedented opportunity to uncover neurobiological subtypes through clustering or subtyping analyses on multi-state functional connectivity, few existing approaches are applicable here to accommodate the network topology and unique biological architecture of functional connectivity. To address this issue, we propose an innovative Bayesian nonparametric network-variate clustering analysis to uncover subgroups with homogeneous brain functional network patterns integrating different cognitive states. In light of the existing neuroscience literature, we assume there are unknown state-specific modular structures within functional connectivity and simultaneously impose selection to identify informative network features for defining subtypes within unsupervised learning. To further facilitate practical use, we develop a computationally efficient variational inference algorithm to perform posterior inference with satisfactory estimation accuracy. Extensive simulations show the superior clustering accuracy and plausible result of our method. Applying the method to the landmark Adolescent Brain Cognitive Development (ABCD) study, we successfully establish neurodevelopmental subtypes linked with impulsivity related behavior trait, and identify brain sub-network phenotypes under each state to signal neurobiological heterogeneity.
Phase relations for seismology of photospheric flux tubes

Aims. We present a mathematical framework for the seismology of photospheric flux tubes using a uniform straight cylinder as a flux-tube model. In contrast to the earlier model of Fujimura & Tsuneta (2009), we also include a non-zero gas pressure; we do not use the thin tube approximation and we use an underdense region inside the flux tube. Methods. We used the linearised ideal magnetohydrodynamic equations to describe different wave modes in the photosphere. Using the wave mode polarisations we then obtained phase relations which represent different observables. Those phase relations were used to calculate phase differences and amplitude ratios. Finally we inverted these amplitude ratios to obtain plasma parameters which are not directly observable. Results. The mathematical framework results in phase differences that can be conveniently compared with observational data to distinguish between different wave modes. Once the wave mode has been identified, the i nverted amplitude ratios can be used to either analytically or numerically estimate the magnitude of plasma parameters which are not directly observable, such as the vertical wavenumber. Artificial observations of di fferent wave modes have shown that the framework mostly succeeds in identifying the correct wave mode and in reproducing the correct plasma parameters using the inverted amplitude ratios.
Group vertex magicness of product graphs and trees

In this article, some necessary conditions of group vertex magicness of graphs with at least one pendant, group vertex magicness of product graphs, are proved. A characterization of group vertex magicness of trees of diameter up to 5. for all infinite Abelian groups with finitely many torsion elements, is also obtained.
Interpreting wealth distribution via poverty map inference using multimodal data

Poverty maps are essential tools for governments and NGOs to track socioeconomic changes and adequately allocate infrastructure and services in places in need. Sensor and online crowd-sourced data combined with machine learning methods have provided a recent breakthrough in poverty map inference. However, these methods do not capture local wealth fluctuations, and are not optimized to produce accountable results that guarantee accurate predictions to all sub-populations. Here, we propose a pipeline of machine learning models to infer the mean and standard deviation of wealth across multiple geographically clustered populated places, and illustrate their performance in Sierra Leone and Uganda. These models leverage seven independent and freely available feature sources based on satellite images, and metadata collected via online crowd-sourcing and social media. Our models show that combined metadata features are the best predictors of wealth in rural areas, outperforming image-based models, which are the best for predicting the highest wealth quintiles. Our results recover the local mean and variation of wealth, and correctly capture the positive yet non-monotonous correlation between them. We further demonstrate the capabilities and limitations of model transfer across countries and the effects of data recency and other biases. Our methodology provides open tools to build towards more transparent and interpretable models to help governments and NGOs to make informed decisions based on data availability, urbanization level, and poverty thresholds.
Exploring the Limits of Transfer Learning with Unified Model in the Cybersecurity Domain

With the increase in cybersecurity vulnerabilities of software systems, the ways to exploit them are also increasing. Besides these, malware threats, irregular network interactions, and discussions about exploits in public forums are also on the rise. To identify these threats faster, to detect potentially relevant entities from any texts, and to be aware of software vulnerabilities, automated approaches are necessary. Application of natural language processing (NLP) techniques in the Cybersecurity domain can help in achieving this. However, there are challenges such as the diverse nature of texts involved in the cybersecurity domain, the unavailability of large-scale publicly available datasets, and the significant cost of hiring subject matter experts for annotations. One of the solutions is building multi-task models that can be trained jointly with limited data. In this work, we introduce a generative multi-task model, Unified Text-to-Text Cybersecurity (UTS), trained on malware reports, phishing site URLs, programming code constructs, social media data, blogs, news articles, and public forum posts. We show UTS improves the performance of some cybersecurity datasets. We also show that with a few examples, UTS can be adapted to novel unseen tasks and the nature of data
Maximum Consensus Localization Using an Objective Function Based on Helmert's Point Error

Ego-localization is a crucial task for autonomous vehicles. On the one hand, it needs to be very accurate, and on the other hand, very robust to provide reliable pose (position and orientation) information, even in challenging environments. Finding the best ego-position is usually tied to optimizing an objective function based on the sensor measurements. The most common approach is to maximize the likelihood, which leads under the assumption of normally distributed random variables to the well-known least squares minimization, often used in conjunction with recursive estimation, e. g. using a Kalman filter. However, least squares minimization is inherently sensitive to outliers, and consequently, more robust loss functions, such as $L_{1}$ norm or Huber loss have been proposed. Arguably the most robust loss function is the outlier count, also known as maximum consensus optimization, where the outcome is independent of the outlier magnitude. In this paper, we investigate in detail the performance of maximum consensus localization based on LiDAR data. We elaborate on its shortcomings and propose a novel objective function based on Helmert's point error. In an experiment using 3001 measurement epochs, we show that the maximum consensus localization based on the introduced objective function provides superior results with respect to robustness.
A Dynamic Feedforward Control Strategy for Energy-efficient Building System Operation

The development of current building energy system operation has benefited from: 1. Informational support from the optimal design through simulation or first-principles models; 2. System load and energy prediction through machine learning (ML). Through the literature review, we note that in current control strategies and optimization algorithms, most of them rely on receiving information from real-time feedback or using only predictive signals based on ML data fitting. They do not fully utilize dynamic building information. In other words, embedding dynamic prior knowledge from building system characteristics simultaneously for system control draws less attention. In this context, we propose an engineer-friendly control strategy framework. The framework is integrated with a feedforward loop that embedded a dynamic building environment with leading and lagging system information involved: The simulation combined with system characteristic information is imported to the ML predictive algorithms. ML generates step-ahead information by rolling-window feed-in of simulation output to minimize the errors of its forecasting predecessor in a loop and achieve an overall optimal. We tested it in a case for heating system control with typical control strategies, which shows our framework owns a further energy-saving potential of 15%.
Screening of the Coulomb Potential in the Athermal Stage in Highly Excited Semiconductors

Equations for the reduced density operator are evaluated. Investigating the subdynamics of nonrelevant observables a resolvent formulation is obtained selfconsistently for the screening part of the evolution superoperator. Utilizing the diagonal representation for the time-dependent mean field term of the Hamiltonian, a selfconsistent concrete calculation of the screened potential is possible in the high density limit of the nonequilibrium. The result is an extension of the well known RPA-formula for nonequilibrium two-band systems of Fermions. An extension to the semimarkovian approximation is counted for. Ausgehend von den Evolutionsgleichungen fur den reduzierten Dichteoperator wird die Subdynamik der nichtrelevanten Observablen in einem Resolventenformalismus untersucht. Auf diese Weise erhalt man, selbstkonsistent mit der allgemeinen Theorie, einen Ausdruck fur die Abschirmfunktion im athermalen Nichtgleichgewicht in einer zeitabhangigen Theorie des gemittelten Feldes. Die verwendete stationare Naherung ist unter der Voraussetzung vieler Stose auch fur das Nichtgleichgewicht anwendbar und entspricht einer Erweiterung der RPA. Eine Verbesserung der Kurzzeitgedachtnisnaherung fur den Stosterm wird angegeben.
Incentives Can Reduce Bias in Online Reviews

Online reviews are a powerful means of propagating the reputations of products, services, and even employers. However, existing research suggests that online reviews often suffer from selection bias – people with extreme opinions are more motivated to share them than people with moderate opinions, resulting in biased distributions of reviews. Providing incentives for reviewing has the potential to reduce this selection bias, because incentives can mitigate the motivational deficit of people who hold moderate opinions. Using data from one of the leading employer review companies, Glassdoor, we show that voluntary reviews have a different distribution from incentivized reviews. The likely bias in the distribution of voluntary reviews can affect workers' choice of employers, because it changes the ranking of industries by average employee satisfaction. Because observational data from Glassdoor are not able to provide a measure of the true distribution of employer reviews, we complement our investigation with a randomized controlled experiment on MTurk. We find that when participants' decision to review their employer is voluntary, the resulting distribution of reviews differs from the distribution of forced reviews. Moreover, providing relatively high monetary rewards or a pro-social cue as incentives for reviewing reduces this bias. We conclude that while voluntary employer reviews often suffer from selection bias, incentives can significantly reduce bias and help workers make more informed employer choices.
Nucleation and coalescence effects on the density of self-induced GaN nanowires grown by molecular beam epitaxy

The evolution of the density of self-induced GaN nanowires as a function of the growth time, gallium rate, and growth temperature has been investigated by scanning and transmission electron microscopy. Nucleation and coalescence effects have been disentangled and quantified by distinguishing between single nanowires and nanowire clusters. Owing to the very specific nanowire nucleation mechanism involving a shape transition from spherical-cap-shaped islands, the nanowire density does not follow the standard island nucleation theory. Furthermore, the detrimental nanowire coalescence process can be significantly reduced by raising the growth temperature.
Targeted Screening for Spontaneous Insertion Mutations in a Lactic Acid Bacterium, Tetragenococcus halophilus

Genetic and strain improvement tools to manipulate a gene of interest are required for the nontransformable lactic acid bacterium Tetragenococcus halophilus. Here, we demonstrate that an endogenous transposable element, ISTeha4, is transposed into the host genome at an extremely high frequency. ABSTRACT Studies on the microorganisms used in food production are of interest because microbial genotypes are reflected in food qualities such as taste, flavor, and yield. However, several microbes are nonmodel organisms, and their analysis is often limited by the lack of genetic tools. Tetragenococcus halophilus, a halophilic lactic acid bacterium used in soy sauce fermentation starter culture, is one such microorganism. The lack of DNA transformation techniques for T. halophilus makes gene complementation and disruption assays difficult. Here, we report that the endogenous insertion sequence ISTeha4, belonging to the IS4 family, is translocated at an extremely high frequency in T. halophilus and causes insertional mutations at various loci. We developed a method named targeting spontaneous insertional mutations in genomes (TIMING), which combines high-frequency insertional mutations and efficient PCR screening, enabling the isolation of gene mutants of interest from a library. The method provides a reverse genetics and strain improvement tool, does not require the introduction of exogenous DNA constructs, and enables the analysis of nonmodel microorganisms lacking DNA transformation techniques. Our results highlight the important role of insertion sequences as a source of spontaneous mutagenesis and genetic diversity in bacteria. IMPORTANCE Genetic and strain improvement tools to manipulate a gene of interest are required for the nontransformable lactic acid bacterium Tetragenococcus halophilus. Here, we demonstrate that an endogenous transposable element, ISTeha4, is transposed into the host genome at an extremely high frequency. A genotype-based and non-genetically engineered screening system was constructed to isolate knockout mutants using this transposable element. The method described enables a better understanding of the genotype-phenotype relationship and serves as a tool to develop food-grade-appropriate mutants of T. halophilus.
Draft Genome Sequence of a Mixed-Serogroup W/Y Invasive Neisseria meningitidis Strain

We report the genome of a Neisseria meningitidis strain (GE-156) that was isolated in Switzerland from a patient diagnosed with bacteremia. The strain belongs to a rare mixed serogroup W/Y and sequence type 11847 (clonal complex 167), as revealed by both routine laboratory examination and genomic sequencing. ABSTRACT We report the genome of a Neisseria meningitidis strain (GE-156) that was isolated in Switzerland from a patient diagnosed with bacteremia. The strain belongs to a rare mixed serogroup W/Y and sequence type 11847 (clonal complex 167), as revealed by both routine laboratory examination and genomic sequencing.
Genome Sequence of the Endosymbiont Endozoicomonas sp. Strain GU-1 (Gammaproteobacteria), Isolated from the Staghorn Coral Acropora pulchra (Cnidaria: Scleractinia)

Endozoicomonas sp. strain GU-1 was isolated from two separate staghorn coral (Acropora pulchra) colonies collected in Guam, Micronesia. Both isolates were grown in marine broth prior to DNA extraction and Oxford Nanopore Technologies (ONT) sequencing. Genomes were approximately 6.1 Mbp in size, containing highly similar gene content and matching sets of rRNA sequences. ABSTRACT Endozoicomonas sp. strain GU-1 was isolated from two separate staghorn coral (Acropora pulchra) colonies collected in Guam, Micronesia. Both isolates were grown in marine broth prior to DNA extraction and Oxford Nanopore Technologies (ONT) sequencing. Genomes were approximately 6.1 Mbp in size, containing highly similar gene content and matching sets of rRNA sequences.
Epidemiology and Genetic Characteristics of Carbapenem-Resistant Escherichia coli in Chinese Intensive Care Unit Analyzed by Whole-Genome Sequencing: a Prospective Observational Study

CREC represents a public health threat worldwide, and its colonization precedes or occurs simultaneously with infection; once the colonization rate increases, the infection rate rises sharply. In our hospital, the colonization rate of CREC remained low, and almost all of the CREC isolates detected were ICU acquired. ABSTRACT This 4-month-long prospective observational study investigated the epidemiological characteristics, genetic composition, transmission pattern, and infection control of carbapenem-resistant Escherichia coli (CREC) colonization in patients at an intensive care unit (ICU) in China. Phenotypic confirmation testing was performed on nonduplicated isolates from patients and their environments. Whole-genome sequencing was performed for all E. coli isolates, followed by multilocus sequence typing (MLST), and antimicrobial resistance genes and single nucleotide polymorphisms (SNPs) were screened. The colonization rates of CREC were 7.29% from the patient specimens and 0.39% from the environmental specimens. Among the 214 E. coli isolates tested, 16 were carbapenem resistant, with the blaNDM-5 gene identified as the dominant carbapenemase-encoding gene. Among the low-homology sporadic strains isolated in this study, the main sequence type (ST) of carbapenem-sensitive Escherichia coli (CSEC) was ST1193, whereas the majority of CREC isolates belonged to ST1656, followed by ST131. CREC isolates were more sensitive to disinfectants than were the carbapenem-resistant Klebsiella pneumoniae (CRKP) isolates obtained in the same period, which may explain the lower separation rate. Therefore, effective interventions and active screening are beneficial to the prevention and control of CREC. IMPORTANCE CREC represents a public health threat worldwide, and its colonization precedes or occurs simultaneously with infection; once the colonization rate increases, the infection rate rises sharply. In our hospital, the colonization rate of CREC remained low, and almost all of the CREC isolates detected were ICU acquired. Contamination of the surrounding environment by CREC carrier patients shows a very limited spatiotemporal distribution. As the dominant ST of the CSEC isolates found, ST1193 CREC might be considered a strain of notable concern with potential to cause a future outbreak. ST1656 and ST131 also deserve attention, as they comprised the majority of the CREC isolates found, while blaNDM-5 gene screening should play an important role in medication guidance as the main carbapenem resistance gene identified. The disinfectant chlorhexidine, which is used commonly in the hospital, is effective for CREC rather than CRKP, possibly explaining the lower positivity rate for CREC than for CRKP.