Miracle-dz/UniFAR-FTUs huggingface dataset:
Optimistic Curiosity Exploration and Conservative Exploitation with Linear Reward Shaping

In this work, we study the simple yet universally applicable case of reward shaping in value-based Deep Reinforcement Learning (DRL). We show that reward shifting in the form of the linear transformation is equivalent to changing the initialization of the $Q$-function in function approximation. Based on such an equivalence, we bring the key insight that a positive reward shifting leads to conservative exploitation, while a negative reward shifting leads to curiosity-driven exploration. Accordingly, conservative exploitation improves offline RL value estimation, and optimistic value estimation improves exploration for online RL. We validate our insight on a range of RL tasks and show its improvement over baselines: (1) In offline RL, the conservative exploitation leads to improved performance based on off-the-shelf algorithms; (2) In online continuous control, multiple value functions with different shifting constants can be used to tackle the exploration-exploitation dilemma for better sample efficiency; (3) In discrete control tasks, a negative reward shifting yields an improvement over the curiosity-based exploration method.
The Terminating-Random Experiments Selector: Fast High-Dimensional Variable Selection with False Discovery Rate Control

We propose the Terminating-Random Experiments (T-Rex) selector, a fast variable selection method for high-dimensional data. The T-Rex selector controls a user-defined target false discovery rate (FDR) while maximizing the number of selected variables. This is achieved by fusing the solutions of multiple early terminated random experiments. The experiments are conducted on a combination of the original predictors and multiple sets of randomly generated dummy predictors. A finite sample proof based on martingale theory for the FDR control property is provided. Numerical simulations confirm that the FDR is controlled at the target level while allowing for high power. We prove that the dummies can be sampled from any univariate probability distribution with finite expectation and variance. The computational complexity of the proposed method is linear in the number of variables. The T-Rex selector outperforms state-of-the-art methods for FDR control in numerical experiments and on a simulated genome-wide association study (GWAS), while its sequential computation time is more than two orders of magnitude lower than that of the strongest benchmark methods. The open source R package TRexSelector containing the implementation of the T-Rex selector is available on CRAN.
An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. The results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. Our benchmark is publicly available at \url{https://github.com/liaorongfan/DeepPersonality}.
A Group Public Goods Game with Position Uncertainty

We model a dynamic public good contribution game, where players are (nat-urally) formed into groups. The groups are exogenously placed in a sequence, with limited information available to players about their groups’ position in the sequence. Contribution decisions are made by players simultaneously and inde-pendently, and the groups’ total contribution is made sequentially. We try to cap-ture both inter and intra-group behaviors and analyze different situations where players observe partial history about total contributions of their predecessor groups. Given this framework, we show that even when players observe a history of defection (no contribution), a cooperative outcome is achievable. This is particularly interesting in the situation when players observe only their immediate predecessor groups’ contribution, where we observe that players play an important role in motivating others to contribute.
Loss Minimization through the Lens of Outcome Indistinguishability

We present a new perspective on loss minimization and the recent notion of Omniprediction through the lens of Outcome Indistingusihability. For a collection of losses and hypothesis class, omniprediction requires that a predictor provide a loss-minimization guarantee simultaneously for every loss in the collection compared to the best (loss-specific) hypothesis in the class. We present a generic template to learn predictors satisfying a guarantee we call Loss Outcome Indistinguishability. For a set of statistical tests--based on a collection of losses and hypothesis class--a predictor is Loss OI if it is indistinguishable (according to the tests) from Nature's true probabilities over outcomes. By design, Loss OI implies omniprediction in a direct and intuitive manner. We simplify Loss OI further, decomposing it into a calibration condition plus multiaccuracy for a class of functions derived from the loss and hypothesis classes. By careful analysis of this class, we give efficient constructions of omnipredictors for interesting classes of loss functions, including non-convex losses. This decomposition highlights the utility of a new multi-group fairness notion that we call calibrated multiaccuracy, which lies in between multiaccuracy and multicalibration. We show that calibrated multiaccuracy implies Loss OI for the important set of convex losses arising from Generalized Linear Models, without requiring full multicalibration. For such losses, we show an equivalence between our computational notion of Loss OI and a geometric notion of indistinguishability, formulated as Pythagorean theorems in the associated Bregman divergence. We give an efficient algorithm for calibrated multiaccuracy with computational complexity comparable to that of multiaccuracy. In all, calibrated multiaccuracy offers an interesting tradeoff point between efficiency and generality in the omniprediction landscape.
BICEP/Keck. XVII. Line-of-sight Distortion Analysis: Estimates of Gravitational Lensing, Anisotropic Cosmic Birefringence, Patchy Reionization, and Systematic Errors

We present estimates of line-of-sight distortion fields derived from the 95 and 150 GHz data taken by BICEP2, BICEP3, and the Keck Array up to the 2018 observing season, leading to cosmological constraints and a study of instrumental and astrophysical systematics. Cosmological constraints are derived from three of the distortion fields concerning gravitational lensing from large-scale structure, polarization rotation from magnetic fields or an axion-like field, and the screening effect of patchy reionization. We measure an amplitude of the lensing power spectrum ALϕϕ=0.95±0.20 . We constrain polarization rotation, expressed as the coupling constant of a Chern–Simons electromagnetic term g a γ ≤ 2.6 × 10−2/H I , where H I is the inflationary Hubble parameter, and an amplitude of primordial magnetic fields smoothed over 1 Mpc B 1Mpc ≤ 6.6 nG at 95 GHz. We constrain the rms of optical depth fluctuations in a simple “crinkly surface” model of patchy reionization, finding A τ < 0.19 (2σ) for the coherence scale of L c = 100. We show that all of the distortion fields of the 95 and 150 GHz polarization maps are consistent with simulations including lensed ΛCDM, dust, and noise, with no evidence for instrumental systematics. In some cases, the EB and TB quadratic estimators presented here are more sensitive than our previous map-based null tests at identifying and rejecting spurious B-modes that might arise from instrumental effects. Finally, we verify that the standard deprojection filtering in the BICEP/Keck data processing is effective at removing temperature to polarization leakage.
A Policy-Guided Imitation Approach for Offline Reinforcement Learning

Offline reinforcement learning (RL) methods can generally be categorized into two types: RL-based and Imitation-based. RL-based methods could in principle enjoy out-of-distribution generalization but suffer from erroneous off-policy evaluation. Imitation-based methods avoid off-policy evaluation but are too conservative to surpass the dataset. In this study, we propose an alternative approach, inheriting the training stability of imitation-style methods while still allowing logical out-of-distribution generalization. We decompose the conventional reward-maximizing policy in offline RL into a guide-policy and an execute-policy. During training, the guide-poicy and execute-policy are learned using only data from the dataset, in a supervised and decoupled manner. During evaluation, the guide-policy guides the execute-policy by telling where it should go so that the reward can be maximized, serving as the \textit{Prophet}. By doing so, our algorithm allows \textit{state-compositionality} from the dataset, rather than \textit{action-compositionality} conducted in prior imitation-style methods. We dumb this new approach Policy-guided Offline RL (\texttt{POR}). \texttt{POR} demonstrates the state-of-the-art performance on D4RL, a standard benchmark for offline RL. We also highlight the benefits of \texttt{POR} in terms of improving with supplementary suboptimal data and easily adapting to new tasks by only changing the guide-poicy.
Powerful Omnibus Tests of Goodness-of-Fit

This paper takes a look at omnibus tests of goodness-of-test in the context of reweighted Anderson-Darling tests and makes three fold contributions. The ﬁrst contribution is to provide a geometric understanding. It is argued that the test statistic with minimum variance can serve as a good general-purpose test. The second contribution is to propose better omnibus tests, called circularly symmetric tests and obtained by circularizing reweighted Anderson-Darling tests. A limited but arguably convincing simulation study on ﬁnite-sample performance demonstrates that the circularized tests outperform their parent methods. The third contribution is to establish new large-sample results. It is shown that like Anderson-Darling, the minimum-variance test statistic and two circularly symmetric test statistics under the null has the same distribution as that of a weighted sum of an inﬁnite number of independent squared normal random variables.
When to Update Your Model: Constrained Model-based Reinforcement Learning

Designing and analyzing model-based RL (MBRL) algorithms with guaranteed monotonic improvement has been challenging, mainly due to the interdependence between policy optimization and model learning. Existing discrepancy bounds generally ignore the impacts of model shifts, and their corresponding algorithms are prone to degrade performance by drastic model updating. In this work, we first propose a novel and general theoretical scheme for a non-decreasing performance guarantee of MBRL. Our follow-up derived bounds reveal the relationship between model shifts and performance improvement. These discoveries encourage us to formulate a constrained lower-bound optimization problem to permit the monotonicity of MBRL. A further example demonstrates that learning models from a dynamically-varying number of explorations benefit the eventual returns. Motivated by these analyses, we design a simple but effective algorithm CMLO (Constrained Model-shift Lower-bound Optimization), by introducing an event-triggered mechanism that flexibly determines when to update the model. Experiments show that CMLO surpasses other state-of-the-art methods and produces a boost when various policy optimization methods are employed.
Reflections on trusting distributed trust

Many systems today distribute trust across multiple parties such that the system provides certain security properties if a subset of the parties are honest. In the past few years, we have seen an explosion of academic and industrial cryptographic systems built on distributed trust, including secure multi-party computation applications (e.g., private analytics, secure learning, and private key recovery) and blockchains. These systems have great potential for improving security and privacy, but face a significant hurdle on the path to deployment. We initiate study of the following problem: a single organization is, by definition, a single party, and so how can a single organization build a distributed-trust system where corruptions are independent? We instead consider an alternative formulation of the problem: rather than ensuring that a distributed-trust system is set up correctly by design, what if instead, users can audit a distributed-trust deployment? We propose a framework that enables a developer to efficiently and cheaply set up any distributed-trust system in a publicly auditable way. To do this, we identify two application-independent building blocks that we can use to bootstrap arbitrary distributed-trust applications: secure hardware and an append-only log. We show how to leverage existing implementations of these building blocks to deploy distributed-trust systems, and we give recommendations for infrastructure changes that would make it easier to deploy distributed-trust systems in the future.
CUP: Critic-Guided Policy Reuse

The ability to reuse previous policies is an important aspect of human intelligence. To achieve efficient policy reuse, a Deep Reinforcement Learning (DRL) agent needs to decide when to reuse and which source policies to reuse. Previous methods solve this problem by introducing extra components to the underlying algorithm, such as hierarchical high-level policies over source policies, or estimations of source policies' value functions on the target task. However, training these components induces either optimization non-stationarity or heavy sampling cost, significantly impairing the effectiveness of transfer. To tackle this problem, we propose a novel policy reuse algorithm called Critic-gUided Policy reuse (CUP), which avoids training any extra components and efficiently reuses source policies. CUP utilizes the critic, a common component in actor-critic methods, to evaluate and choose source policies. At each state, CUP chooses the source policy that has the largest one-step improvement over the current target policy, and forms a guidance policy. The guidance policy is theoretically guaranteed to be a monotonic improvement over the current target policy. Then the target policy is regularized to imitate the guidance policy to perform efficient policy search. Empirical results demonstrate that CUP achieves efficient transfer and significantly outperforms baseline algorithms.
A lower bound for the smallest eigenvalue of a graph and an application to the associahedron graph

In this paper, we obtain a lower bound for the smallest eigenvalue of a regular graph containing many copies of a smaller ﬁxed subgraph. This generalizes a result of Aharoni, Alon, and Berger in which the subgraph is a triangle. We apply our results to obtain a lower bound on the smallest eigenvalue of the associahedron graph, and we prove that this bound gives the correct order of magnitude of this eigenvalue. We also survey what is known regarding the second-largest eigenvalue of the associahedron graph.
Analogical and inductive reasoning

The first problem discussed in this thesis is the logical problem of analogy, which, given a formal definition of analogical inference, asks under what conditions such inferences may be justified. By showing the inadequacy of approaches based on the degree of similarity between analogues, the importance of relevance between known and inferred similarities is highlighted. The need for a logical semantics for relevance motivates the definition of determinations, first-order expressions capturing the idea of relevance between predicate schemata. Determinations are shown to justify analogical inferences and single-instance generalizations non-trivially, and to express an apparently common form of knowledge hitherto overlooked in knowledge-based systems. Analogical reasoning is implemented in MRS, a logic programming system, and shown to be more efficient than simple rule-based methods for some important inference tasks. The ability to acquire and use determinations is shown strictly to increase the inferences a system can make from a given set of data. Programs are described for the inductive acquisition of determinations and their subsequent use in analogical reasoning to aid in the construction of a large knowledge base. The second problem, suggested by and subsuming the first, is to identify the ways in which existing knowledge can be used to help a system to learn from experience. A method is given for enumerating the types of knowledge, of which determinations are but one, that contribute to learning, and a general inductive inference machine is described based on these ideas. The application of a logical, knowledge-based approach to the problems of analogy and induction thus shows the need for a system to be able to detect as many logical forms of regularity as possible in order to maximize its inferential capability. The possibility that important aspects of 'common sense' are captured by complex, abstract regularities suggests further empirical research to identify this knowledge.
A Framework for Operations Research Model Use in Resilience to Fundamental Surprise Events: Observations from University Operations during COVID-19

Operations research (OR) approaches have been increasingly applied to model the resilience of a system to surprise events. In order to model a surprise event, one must have an understanding of its characteristics, which then become parameters, decisions, and/or constraints in the resulting model. This means that these models cannot (directly) handle fundamental surprise events, which are events that could not be defined before they happen. However, OR models may be adapted, improvised, or created during a fundamental surprise event, such as the COVID-19 pandemic, to help respond to it. We provide a framework for how OR models were applied by a university in response to the pandemic, thus helping to understand the role of OR models during fundamental surprise events. Our framework includes the following adaptations: adapting data, adding constraints, model switching, pulling from the modeling toolkit, and creating a new model. Each of these adaptations is formally presented, with supporting evidence gathered through interviews with modelers and users involved in the university response to the pandemic. We discuss the implications of this framework for both OR and resilience.
Modeling the Lighting in Scenes as Style for Auto White-Balance Correction

Style may refer to different concepts (e.g. painting style, hairstyle, texture, color, filter, etc.) depending on how the feature space is formed. In this work, we propose a novel idea of interpreting the lighting in the single- and multi-illuminant scenes as the concept of style. To verify this idea, we introduce an enhanced auto white-balance (AWB) method that models the lighting in single- and mixed-illuminant scenes as the style factor. Our AWB method does not require any illumination estimation step, yet contains a network learning to generate the weighting maps of the images with different WB settings. Proposed network utilizes the style information, extracted from the scene by a multi-head style extraction module. AWB correction is completed after blending these weighting maps and the scene. Experiments on single- and mixed-illuminant datasets demonstrate that our proposed method achieves promising correction results when compared to the recent works. This shows that the lighting in the scenes with multiple illuminations can be modeled by the concept of style. Source code and trained models are available on https://github.com/birdortyedi/lighting-as-style-awb-correction.
PCGen: Point Cloud Generator for LiDAR Simulation

Data is a fundamental building block for LiDAR perception systems. Unfortunately, real-world data collection and annotation is extremely costly & laborious. Recently, real data based LiDAR simulators have shown tremendous potential to complement real data, due to their scalability and high-fidelity compared to graphics engine based methods. Before simulation can be deployed in the real-world, two shortcomings need to be addressed. First, existing methods usually generate data which are more noisy and complete than the real point clouds, due to 3D reconstruction error and pure geometry-based raycasting method. Second, prior works on simulation for object detection focus solely on rigid objects, like cars, but Vulnerable Road User (VRU)s, like pedestrians, are important road participants. To tackle the first challenge, we propose First Peak Averaging (FPA) raycasting and surrogate model raydrop. FPA enables the simulation of both point cloud coordinates and sensor features, while taking into account reconstruction noise. The ray-wise surrogate raydrop model mimics the physical properties of LiDAR's laser receiver to determine whether a simulated point would be recorded by a real LiDAR. With minimal training data, the surrogate model can generalize to different geographies and scenes, closing the domain gap between raycasted and real point clouds. To tackle the simulation of deformable VRU simulation, we employ Skinned Multi-Person Linear model (SMPL) dataset to provide a pedestrian simulation baseline and compare the domain gap between CAD and reconstructed objects. Applying our pipeline to perform novel sensor synthesis, results show that object detection models trained by simulation data can achieve similar result as the real data trained model.