Miracle-dz/UniFAR-FTUs huggingface dataset:
Anti-Concentrated Confidence Bonuses for Scalable Exploration

Intrinsic rewards play a central role in handling the exploration-exploitation trade-off when designing sequential decision-making algorithms, in both foundational theory and state-of-the-art deep reinforcement learning. The LinUCB algorithm, a centerpiece of the stochastic linear bandits literature, prescribes an elliptical bonus which addresses the challenge of leveraging shared information in large action spaces. This bonus scheme cannot be directly transferred to high-dimensional exploration problems, however, due to the computational cost of maintaining the inverse covariance matrix of action features. We introduce anti-concentrated conﬁdence bounds for efﬁciently approximating the elliptical bonus, using an ensemble of regressors trained to predict random noise from policy network-derived features. Using this approximation, we obtain stochastic linear bandit algorithms which obtain ˜ O ( d √ T ) regret bounds for poly( d ) ﬁxed actions. We develop a practical variant for deep reinforcement learning that is competitive with contemporary intrinsic reward heuristics on Atari benchmarks.
Feedback Linearization of Car Dynamics for Racing via Reinforcement Learning

Through the method of Learning Feedback Linearization, we seek to learn a linearizing controller to simplify the process of controlling a car to race autonomously. A soft actor-critic approach is used to learn a decoupling matrix and drift vector that effectively correct for errors in a hand-designed linearizing controller. The result is an exactly linearizing controller that can be used to enable the well-developed theory of linear systems to design path planning and tracking schemes that are easy to implement and significantly less computationally demanding. To demonstrate the method of feedback linearization, it is first used to learn a simulated model whose exact structure is known, but varied from the initial controller, so as to introduce error. We further seek to apply this method to a system that introduces even more error in the form of a gym environment specifically designed for modeling the dynamics of car racing. To do so, we posit an extension to the method of learning feedback linearization; a neural network that is trained using supervised learning to convert the output of our linearizing controller to the required input for the racing environment. Our progress towards these goals is reported and the next steps in their accomplishment are discussed.
Koszul duality in quantum field theory

In this article, we introduce basic aspects of the algebraic notion of Koszul duality for a physics audience. We then review its appearance in the physical problem of coupling QFTs to topological line defects, and illustrate the concept with some examples drawn from twists of various simple supersymmetric theories. Though much of the content of this article is well-known to experts, the presentation and examples have not, to our knowledge, appeared in the literature before. Our aim is to provide an elementary introduction for those interested in the appearance of Koszul duality in supersymmetric gauge theories with line defects and, ultimately, its generalizations to higher-dimensional defects and twisted holography.
Heritability in Morphological Robot Evolution

In the field of evolutionary robotics, choosing the correct encoding is very complicated, especially when robots evolve both behaviours and morphologies at the same time. With the objective of improving our understanding of the mapping process from encodings to functional robots, we introduce the biological notion of heritability, which captures the amount of phenotypic variation caused by genotypic variation. In our analysis we measure the heritability on the first generation of robots evolved from two different encodings, a direct encoding and an indirect encoding. In addition we investigate the interplay between heritability and phenotypic diversity through the course of an entire evolutionary process. In particular, we investigate how direct and indirect genotypes can exhibit preferences for exploration or exploitation throughout the course of evolution. We observe how an exploration or exploitation tradeoff can be more easily understood by examining patterns in heritability and phenotypic diversity. In conclusion, we show how heritability can be a useful tool to better understand the relationship between genotypes and phenotypes, especially helpful when designing more complicated systems where complex individuals and environments can adapt and influence each other.
Event-triggered Control for Nonlinear Systems with Center Manifolds

In this work, we consider the problem of event-triggered implementation of control laws designed for the local stabilization of nonlinear systems with center manifolds. We propose event-triggering conditions which are derived from a local input-to-state stability characterization of such systems. The triggering conditions ensure local ultimate boundedness of the trajectories and the existence of a uniform positive lower bound for the inter-event times. The ultimate bound can be made arbitrarily small, but by allowing for smaller inter-event times. Under certain assumptions on the controller structure, local asymptotic stability of the origin is also guaranteed. Two sets of triggering conditions are proposed, that cater to the cases where the exact center manifold and only an approximation of the center manifold is computable. The closed-loop system exhibits some desirable properties when the exact knowledge of the center manifold is employed in checking the triggering conditions. Three illustrative examples that explore different scenarios are presented and the applicability of the proposed methods is demonstrated. The third example concerns the event-triggered implementation of a position stabilizing controller for the open-loop unstable Mobile Inverted Pendulum (MIP) robot.
Pattern Division Random Access (PDRA) for M2M Communications With Massive MIMO Systems

In this work, we introduce the pattern-domain pilot design paradigm based on a “<italic>superposition of orthogonal-building-blocks</italic>” with significantly larger contention space to enhance the massive machine-type communications (mMTC) random access (RA) performance in massive multiple-input multiple-output (MIMO) systems. Specifically, the pattern-domain pilot is constructed based on the superposition of <inline-formula><tex-math notation="LaTeX">$L$</tex-math></inline-formula> cyclically-shifted Zadoff-Chu (ZC) sequences. The pattern-domain pilots exhibit zero correlation values between non-colliding patterns from the same root and low correlation values between patterns from different roots. The increased contention space, i.e., from <italic>N</italic> to <inline-formula><tex-math notation="LaTeX">$\binom{N}{L}$</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$\binom{N}{L}$</tex-math></inline-formula> denotes the number of all <italic>L</italic>-combinations of a set <italic>N</italic>, and low correlation values lead to a significantly lower pilot collision probability without compromising <italic>excessively</italic> on channel estimation performance for mMTC RA in massive MIMO systems. We present the framework and analysis of the RA success probability of the pattern-domain based scheme with massive MIMO systems. Numerical results demonstrate that the proposed pattern division random access (PDRA) scheme achieves an appreciable performance gain over the conventional one, while preserving the existing physical layer virtually unchanged. The extension of the “superposition of orthogonal-building-blocks” scheme to “superposition of quasi-orthogonal-building-blocks” is straightforward.
Axial inverse magnetic catalysis

We ﬁnd that the inverse magnetic catalysis for U (1) axial symmetry (AIMC: axial inverse magnetic catalysis) can emerge around the chiral crossover regime in the thermomagnetic QCD with 2 + 1 ﬂavors at physical point. This phenomenon can be correlated with the IMC for the chiral SU (2) L × SU (2) R symmetry (CIMC: chiral IMC). We explicitly observe the AIMC based on a Nambu-Jona-Lasinio model with 2 + 1 quark ﬂavors, where introduced anomalous magnetic moments of the quarks play the essential role to drive both the CIMC and AIMC. Our ﬁnding is shortly testable on lattices. Possible phenomenological and cosmological implications are also brieﬂy addressed.
Pick-and-Mix Information Operators for Probabilistic ODE Solvers

Probabilistic numerical solvers for ordinary differential equations compute posterior distributions over the solution of an initial value problem via Bayesian inference. In this paper, we leverage their probabilistic formulation to seamlessly include additional information as general likelihood terms. We show that second-order differential equations should be directly provided to the solver, instead of transforming the problem to first order. Additionally, by including higher-order information or physical conservation laws in the model, solutions become more accurate and more physically meaningful. Lastly, we demonstrate the utility of flexible information operators by solving differential-algebraic equations. In conclusion, the probabilistic formulation of numerical solvers offers a flexible way to incorporate various types of information, thus improving the resulting solutions.
SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark

Existing work in language grounding typically study single environments. How do we build unified models that apply across multiple environments? We propose the multi-environment Symbolic Interactive Language Grounding benchmark (SILG), which unifies a collection of diverse grounded language learning environments under a common interface. SILG consists of grid-world environments that require generalization to new dynamics, entities, and partially observed worlds (RTFM, Messenger, NetHack), as well as symbolic counterparts of visual worlds that require interpreting rich natural language with respect to complex scenes (ALFWorld, Touchdown). Together, these environments provide diverse grounding challenges in richness of observation space, action space, language specification, and plan complexity. In addition, we propose the first shared model architecture for RL on these environments, and evaluate recent advances such as egocentric local convolution, recurrent state-tracking, entity-centric attention, and pretrained LM using SILG. Our shared architecture achieves comparable performance to environment-specific architectures. Moreover, we find that many recent modelling advances do not result in significant gains on environments other than the one they were designed for. This highlights the need for a multi-environment benchmark. Finally, the best models significantly underperform humans on SILG, which suggests ample room for future work. We hope SILG enables the community to quickly identify new methodologies for language grounding that generalize to a diverse set of environments and their associated challenges.
Does Data Repair Lead to Fair Models? Curating Contextually Fair Data To Reduce Model Bias

Contextual information is a valuable cue for Deep Neural Networks (DNNs) to learn better representations and improve accuracy. However, co-occurrence bias in the training dataset may hamper a DNNmodel’s generalizabil- ity to unseen scenarios in the real world. For example, in COCO [26], many object categories have a much higher cooccurrence with men compared to women, which can bias a DNN’s prediction in favor of men. Recent works have focused on task-specific training strategies to handle bias in such scenarios, but fixing the available data is often ignored. In this paper, we propose a novel and more generic solution to address the contextual bias in the datasets by selecting a subset of the samples, which is fair in terms of the co-occurrence with various classes for a protected attribute. We introduce a data repair algorithm using the coefficient of variation( cv), which can curate fair and contextually balanced data for a protected class(es). This helps in training a fair model irrespective of the task, architecture or training methodology. Our proposed solution is simple, effective and can even be used in an active learning setting where the data labels are not present or being generated incrementally. We demonstrate the effectiveness of our algorithm for the task of object detection and multi-label image classification across different datasets. Through a series of experiments, we validate that curating contextually fair data helps make model predictions fair by balancing the true positive rate for the protected class across groups without compromising on the model’s overall performance. Code: https://github.com/sumanyumuku98/contextual-bias
Rate limits in quantum networks with lossy repeaters

The derivation of ultimate limits to communication over certain quantum repeater networks have provided extremely valuable benchmarks for assessing near-term quantum communication protocols. However, these bounds are usually derived in the limit of ideal devices and leave questions about the performance of practical implementations unanswered. To address this challenge, we quantify how the presence of loss in repeater stations affect the maximum attainable rates for quantum communication over linear repeater chains and more complex quantum networks. Extending the framework of node splitting, we model the loss introduced at the repeater stations and then prove the corresponding limits. In the linear chain scenario we show that, by increasing the number of repeater stations, the maximum rate cannot overcome a quantity which solely depends on the loss of a single station. We introduce a way of adapting the standard machinery for obtaining bounds to this realistic scenario. The difference is that whilst ultimate limits for any strategy can be derived given a fixed channel, when the repeaters introduce additional decoherence, then the effective overall channel is itself a function of the chosen repeater strategy (e.g., one-way versus two-way classical communication). Classes of repeater strategies can be analysed using additional modelling and the subsequent bounds can be interpreted as the optimal rate within that class.
Single-Mode Emission in InP Microdisks on Si Using Au Antenna

An important building block for on-chip photonic applications is a scaled emitter. Whispering gallery mode cavities based on III–Vs on Si allow for small device footprints and lasing with low thresholds. However, multimodal emission and wavelength stability over a wider range of temperature can be challenging. Here, we explore the use of Au nanorod antennae on InP whispering gallery mode lasers on Si for single-mode emission. We show that by proper choice of the antenna size and positioning, we can suppress the side modes of a cavity and achieve single-mode emission over a wide excitation range. We establish emission trends by varying the size of the antenna and show that the far-field radiation pattern differs significantly for devices with and without antenna. Furthermore, the antenna-induced single-mode emission is dominant from room temperature (300 K) down to 200 K, whereas the cavity without an antenna is multimodal and its dominant emission wavelength is highly temperature-dependent.
Is High Variance Unavoidable in RL? A Case Study in Continuous Control

Reinforcement learning (RL) experiments have notoriously high variance, and minor details can have disproportionately large effects on measured outcomes. This is problematic for creating reproducible research and also serves as an obstacle for real-world applications, where safety and predictability are paramount. In this paper, we investigate causes for this perceived instability. To allow for an in-depth analysis, we focus on a specifically popular setup with high variance -- continuous control from pixels with an actor-critic agent. In this setting, we demonstrate that variance mostly arises early in training as a result of poor"outlier"runs, but that weight initialization and initial exploration are not to blame. We show that one cause for early variance is numerical instability which leads to saturating nonlinearities. We investigate several fixes to this issue and find that one particular method is surprisingly effective and simple -- normalizing penultimate features. Addressing the learning instability allows for larger learning rates, and significantly decreases the variance of outcomes. This demonstrates that the perceived variance in RL is not necessarily inherent to the problem definition and may be addressed through simple architectural modifications.
Algorithmic amplification of politics on Twitter

Significance The role of social media in political discourse has been the topic of intense scholarly and public debate. Politicians and commentators from all sides allege that Twitter’s algorithms amplify their opponents’ voices, or silence theirs. Policy makers and researchers have thus called for increased transparency on how algorithms influence exposure to political content on the platform. Based on a massive-scale experiment involving millions of Twitter users, a fine-grained analysis of political parties in seven countries, and 6.2 million news articles shared in the United States, this study carries out the most comprehensive audit of an algorithmic recommender system and its effects on political content. Results unveil that the political right enjoys higher amplification compared to the political left. Content on Twitter’s home timeline is selected and ordered by personalization algorithms. By consistently ranking certain content higher, these algorithms may amplify some messages while reducing the visibility of others. There’s been intense public and scholarly debate about the possibility that some political groups benefit more from algorithmic amplification than others. We provide quantitative evidence from a long-running, massive-scale randomized experiment on the Twitter platform that committed a randomized control group including nearly 2 million daily active accounts to a reverse-chronological content feed free of algorithmic personalization. We present two sets of findings. First, we studied tweets by elected legislators from major political parties in seven countries. Our results reveal a remarkably consistent trend: In six out of seven countries studied, the mainstream political right enjoys higher algorithmic amplification than the mainstream political left. Consistent with this overall trend, our second set of findings studying the US media landscape revealed that algorithmic amplification favors right-leaning news sources. We further looked at whether algorithms amplify far-left and far-right political groups more than moderate ones; contrary to prevailing public belief, we did not find evidence to support this hypothesis. We hope our findings will contribute to an evidence-based debate on the role personalization algorithms play in shaping political content consumption.
CIM-PPO: Proximal Policy Optimization with Liu-Correntropy Induced Metric

As an algorithm based on deep reinforcement learning, Proximal Policy Optimization (PPO) performs well in many complex tasks and has become one of the most popular RL algorithms in recent years. According to the mechanism of penalty in surrogate objective, PPO can be divided into PPO with KL Divergence (KL-PPO) and PPO with Clip function(Clip-PPO). Clip-PPO is widely used in a variety of practical scenarios and has attracted the attention of many researchers. Therefore, many variations have also been created, making the algorithm better and better. However, as a more theoretical algorithm, KL-PPO was neglected because its performance was not as good as CliP-PPO. In this article, we analyze the asymmetry effect of KL divergence on PPO's objective function , and give the inequality that can indicate when the asymmetry will affect the efficiency of KL-PPO. Proposed PPO with Correntropy Induced Metric algorithm(CIM-PPO) that use the theory of correntropy(a symmetry metric method that was widely used in M-estimation to evaluate two distributions' difference)and applied it in PPO. Then, we designed experiments based on OpenAIgym to test the effectiveness of the new algorithm and compare it with KL-PPO and CliP-PPO.
Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees

Inverse Reinforcement Learning (IRL) is the problem of ﬁnding a reward function which describes observed/known expert behavior. The IRL setting is remarkably useful for automated control, in situations where the reward function is difﬁcult to specify manually or as a means to extract agent preference. In this work, we provide a new IRL algorithm for the continuous state space setting with unknown transition dynamics by modeling the system using a basis of orthonormal functions . Moreover, we provide a proof of correctness and formal guarantees on the sample and time complexity of our algorithm. Finally, we present synthetic experiments to corroborate our theoretical guarantees.
A spherical rearrangement proof of the stability of a Riesz-type inequality and an application to an isoperimetric type problem

We prove the stability of the ball as global minimizer of an attractive shape functional under volume constraint, by means of mass transportation arguments. The stability exponent is $1/2$ and it is sharp. Moreover, we use such stability result together with the quantitative (possibly fractional) isoperimetric inequality to prove that the ball is a global minimizer of a shape functional involving both an attractive and a repulsive term with a sufficiently large fixed volume and with a suitable (possibly fractional) perimeter penalization.
A Robotic Approach towards Quantifying Epipelagic Bound Plastic Using Deep Visual Models

The quantification of positively buoyant marine plastic debris is critical to understanding how plastic litter accumulates across the world’s oceans and is also crucial to identifying hotspots for targeted cleanup efforts. Currently, the most common method to quantify marine plastic is using manta trawls for manual sampling. However, this method is cost-intensive and requires human labor. This study removes the need for manual sampling by using an autonomous method using neural networks and computer vision models, which trained on images captured from various layers of the ocean column to perform real-time plastic quantification. The best performing model has a Mean Average Precision of 85% and an F1-Score of 0.89 while maintaining near real-time processing speeds 2 ms/img.