Miracle-dz/UniFAR-FTUs huggingface dataset:
State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewards

A common formulation of constrained reinforcement learning involves multiple rewards that must individually accumulate to given thresholds. In this class of problems, we show a simple example in which the desired optimal policy cannot be induced by any weighted linear combination of rewards. Hence, there exist constrained reinforcement learning problems for which neither regularized nor classical primal-dual methods yield optimal policies. This work addresses this shortcoming by augmenting the state with Lagrange multipliers and reinterpreting primal-dual methods as the portion of the dynamics that drives the multipliers evolution. This approach provides a systematic state augmentation procedure that is guaranteed to solve reinforcement learning problems with constraints. Thus, as we illustrate by an example, while previous methods can fail at finding optimal policies, running the dual dynamics while executing the augmented policy yields an algorithm that provably samples actions from the optimal policy.
Creolizing the Web

The evolution of language has been a hotly debated subject with contradicting hypotheses and unreliable claims. Drawing from signalling games, dynamic population mechanics, machine learning and algebraic topology, we present a method for detecting evolutionary patterns in a sociological model of language evolution. We develop a minimalistic model that provides a rigorous base for any generalized evolutionary model for language based on communication between individuals. We also discuss theoretical guarantees of this model, ranging from stability of language representations to fast convergence of language by temporal communication and language drift in an interactive setting. Further we present empirical results and their interpretations on a real world dataset from Reddit to identify communities and echo chambers for opinions, thus placing obstructions to reliable communication among communities.
SoK: A Taxonomy for Critical Analysis of Consensus Mechanisms in Consortium Blockchain

Consensus algorithms are central to blockchain technology and an emerging research area. In this paper, we begin with an overview of the different types and architectures of blockchain networks. Then, with a focus on consortium blockchains, we survey, classify, and assess their principal consensus mechanisms. Furthermore, as consensus mechanisms determine network reliability, enhance performance efficiency, and ensure system security, we conduct a critical analysis of the strengths and weaknesses of consensus algorithms using a taxonomy of three different criteria: reliability, performance, and security. We conclude with insights into current and future research challenges and opportunities in this domain.
Cwikel estimates and negative eigenvalues of Schrödinger operators on noncommutative tori

In this paper, we establish Cwikel-type estimates for noncommutative tori for any dimension n ≥ 2. We use them to derive Cwikel–Lieb–Rozenblum inequalities and Lieb–Thirring inequalities for negative eigenvalues of fractional Schrödinger operators on noncommutative tori. The latter leads to a Sobolev inequality for noncommutative tori. On the way, we establish a “borderline version” of the abstract Birman–Schwinger principle for the number of negative eigenvalues of relatively compact form perturbations of a non-negative semi-bounded operator with isolated 0-eigenvalue.
GAMA/DEVILS: constraining the cosmic star formation history from improved measurements of the 0.3–2.2  μm extragalactic background light

We present a revised measurement of the optical extragalactic background light (EBL), based on the contribution of resolved galaxies to the integrated galaxy light (IGL). The cosmic optical background radiation (COB), encodes the light generated by starformation, and provides a wealth of information about the cosmic star formation history (CSFH). We combine wide and deep galaxy number counts from the Galaxy And Mass Assembly survey (GAMA) and Deep Extragalactic VIsible Legacy Survey (DEVILS), along with the Hubble Space Telescope (HST) archive and other deep survey datasets, in 9 multi-wavelength filters to measure the COB in the range from 0.35 μm to 2.2 μm. We derive the luminosity density in each band independently and show good agreement with recent and complementary estimates of the optical-EBL from very high-energy (VHE) experiments. Our error analysis suggests that the IGL and γ-ray measurements are now fully consistent to within ∼10%, suggesting little need for any additional source of diffuse light beyond the known galaxy population. We use our revised IGL measurements to constrain the cosmic star-formation history, and place amplitude constraints on a number of recent estimates. As a consistency check, we can now demonstrate convincingly, that the CSFH, stellar mass growth, and the optical-EBL provide a fully consistent picture of galaxy evolution. We conclude that the peak of star-formation rate lies in the range 0.066 − 0.076 M⊙yr−1 Mpc−3 at a lookback time of 9.1 to 10.9 Gyrs.
Augmenting Part-of-speech Tagging with Syntactic Information for Vietnamese and Chinese

Word segmentation and part-of-speech tagging are two critical preliminary steps for downstream tasks in Vietnamese natural language processing. In reality, people tend to consider also the phrase boundary when performing word segmentation and part of speech tagging rather than solely process word by word from left to right. In this paper, we implement this idea to improve word segmentation and part of speech tagging the Vietnamese language by employing a simplified constituency parser. Our neural model for joint word segmentation and part-of-speech tagging has the architecture of the syllable-based CRF constituency parser. To reduce the complexity of parsing, we replace all constituent labels with a single label indicating for phrases. This model can be augmented with predicted word boundary and part-of-speech tags by other tools. Because Vietnamese and Chinese have some similar linguistic phenomena, we evaluated the proposed model and its augmented versions on three Vietnamese benchmark datasets and six Chinese benchmark datasets. Our experimental results show that the proposed model achieves higher performances than previous works for both languages.
Image Completion via Inference in Deep Generative Models

We consider image completion from the perspective of amortized inference in an image generative model. We leverage recent state of the art variational auto-encoder architectures that have been shown to produce photo-realistic natural images at non-trivial resolutions. Through amortized inference in such a model we can train neural artifacts that produce diverse, realistic image completions even when the vast majority of an image is missing. We demonstrate superior sample quality and diversity compared to prior art on the CIFAR-10 and FFHQ-256 datasets. We conclude by describing and demonstrating an application that requires an in-painting model with the capabilities ours exhibits: the use of Bayesian optimal experimental design to select the most informative sequence of small ﬁeld of view x-rays for chest pathology detection.
Information Directed Reward Learning for Reinforcement Learning

For many reinforcement learning (RL) applications, specifying a reward is difficult. This paper considers an RL setting where the agent obtains information about the reward only by querying an expert that can, for example, evaluate individual states or provide binary preferences over trajectories. From such expensive feedback, we aim to learn a model of the reward that allows standard RL algorithms to achieve high expected returns with as few expert queries as possible. To this end, we propose Information Directed Reward Learning (IDRL), which uses a Bayesian model of the reward and selects queries that maximize the information gain about the difference in return between plausibly optimal policies. In contrast to prior active reward learning methods designed for specific types of queries, IDRL naturally accommodates different query types. Moreover, it achieves similar or better performance with significantly fewer queries by shifting the focus from reducing the reward approximation error to improving the policy induced by the reward model. We support our findings with extensive evaluations in multiple environments and with different query types.
Characterization and recognition of handwritten digits using Julia

Automatic image and digit recognition is a computationally challenging task for image processing and pattern recognition, requiring an adequate appreciation of the syntactic and semantic importance of the image for the identification ofthe handwritten digits. Image and Pattern Recognition has been identified as one of the driving forces in the research areas because of its shifting of different types of applications, such as safety frameworks, clinical frameworks, diversion, and so on.In this study, for recognition, we implemented a hybrid neural network model that is capable of recognizing the digit of MNISTdataset and achieved a remarkable result. The proposed neural model network can extract features from the image and recognize the features in the layer by layer. To expand, it is so important for the neural network to recognize how the proposed modelcan work in each layer, how it can generate output, and so on. Besides, it also can recognize the auto-encoding system and the variational auto-encoding system of the MNIST dataset. This study will explore those issues that are discussed above, and the explanation for them, and how this phenomenon can be overcome.
DNN2LR: Automatic Feature Crossing for Credit Scoring

Credit scoring is a major application of machine learning for financial institutions to decide whether to approve or reject a credit loan. For sake of reliability, it is necessary for credit scoring models to be both accurate and globally interpretable. Simple classifiers, e.g., Logistic Regression (LR), are white-box models, but not powerful enough to model complex nonlinear interactions among features. Fortunately, automatic feature crossing is a promising way to find cross features to make simple classifiers to be more accurate without heavy handcrafted feature engineering. However, credit scoring is usually based on different aspects of users, and the data usually contains hundreds of feature fields. This makes existing automatic feature crossing methods not efficient for credit scoring. In this work, we find local piece-wise interpretations in Deep Neural Networks (DNNs) of a specific feature are usually inconsistent in different samples, which is caused by feature interactions in the hidden layers. Accordingly, we can design an automatic feature crossing method to find feature interactions in DNN, and use them as cross features in LR. We give definition of the interpretation inconsistency in DNN, based on which a novel feature crossing method for credit scoring prediction called DNN2LR is proposed. Apparently, the final model, i.e., a LR model empowered with cross features, generated by DNN2LR is a white-box model. Extensive experiments have been conducted on both public and business datasets from real-world credit scoring applications. Experimental shows that, DNN2LR can outperform the DNN model, as well as several feature crossing methods. Moreover, comparing with the state-of-the-art feature crossing methods, i.e., AutoCross, DNN2LR can accelerate the speed for feature crossing by about 10 to 40 times on datasets with large numbers of feature fields.
Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environments

Recent reinforcement learning studies extensively explore the interplay between cooperative and competitive behaviour in mixed environments. Unlike cooperative environments where agents strive towards a common goal, mixed environments are notorious for the conflicts of selfish and social interests. As a consequence, purely rational agents often struggle to achieve and maintain cooperation. A prevalent approach to induce cooperative behaviour is to assign additional rewards based on other agents' well-being. However, this approach suffers from the issue of multi-agent credit assignment, which can hinder performance. This issue is efficiently alleviated in cooperative setting with such state-of-the-art algorithms as QMIX and COMA. Still, when applied to mixed environments, these algorithms may result in unfair allocation of rewards. We propose BAROCCO, an extension of these algorithms capable to balance individual and social incentives. The mechanism behind BAROCCO is to train two distinct but interwoven components that jointly affect each agent's decisions. Our meta-algorithm is compatible with both Q-learning and Actor-Critic frameworks. We experimentally confirm the advantages over the existing methods and explore the behavioural aspects of BAROCCO in two mixed multi-agent setups.
Equal Affection or Random Selection: the Quality of Subjective Feedback from a Group Perspective

In the setting where a group of agents is asked a single subjective multi-choice question (e.g. which one do you prefer? cat or dog? ), we are interested in evaluating the quality of the collected feedback. However, the collected statistics are not sufficient to reflect how informative the feedback is since fully informative feedback (equal affection of the choices) and fully uninformative feedback (random selection) have the same uniform statistics. Here we distinguish the above two scenarios by additionally asking for respondents' predictions about others' choices. We assume that informative respondents' predictions strongly depend on their own choices while uninformative respondents' do not. With this assumption, we propose a new definition for uninformative feedback and correspondingly design a family of evaluation metrics, called f-variety, for group-level feedback which can 1) distinguish informative feedback and uninformative feedback (separation) even if their statistics are both uniform and 2) decrease as the ratio of uninformative respondents increases (monotonicity). We validate our approach both theoretically and numerically. Moreover, we conduct two real-world case studies about 1) comparisons about athletes and 2) comparisons about stand-up comedians to show the superiority of our approach.
Practitioners’ Perceptions of the Goals and Visual Explanations of Defect Prediction Models

Software defect prediction models are classifiers that are constructed from historical software data. Such software defect prediction models have been proposed to help developers optimize the limited Software Quality Assurance (SQA) resources and help managers develop SQA plans. Prior studies have different goals for their defect prediction models and use different techniques for generating visual explanations of their models. Yet, it is unclear what are the practitioners’ perceptions of (1) these defect prediction model goals, and (2) the model-agnostic techniques used to visualize these models. We conducted a qualitative survey to investigate practitioners’ perceptions of the goals of defect prediction models and the model-agnostic techniques used to generate visual explanations of defect prediction models. We found that (1) 82%-84% of the respondents perceived that the three goals of defect prediction models are useful; (2) LIME is the most preferred technique for understanding the most important characteristics that contributed to a prediction of a file, while ANOVA/VarImp is the second most preferred technique for understanding the characteristics that are associated with software defects in the past. Our findings highlight the significance of investigating how to improve the understanding of defect prediction models and their predictions. Hence, model-agnostic techniques from explainable AI domain may help practitioners to understand defect prediction models and their predictions.
Perspective: Purposeful Failure in Artificial Life and Artificial Intelligence

Complex systems fail. I argue that failures can be a blueprint characterizing living organisms and biological intelligence, a control mechanism to increase complexity in evolutionary simulations, and an alternative to classical fitness optimization. Imitating biological successes in Artificial Life and Artificial Intelligence can be misleading; imitating failures offers a path towards understanding and emulating life it in artificial systems. Failure is Knowledge, Knowledge is Power You are handed a mysterious box containing the most complex object in the universe, and must find how the object works. Where do you start? “The human brain is the most complex object in the universe” is a well worn cliche (Constable (1918)). While the claim might not be true, the human brain is definitely very complex. In neuroscience and psychology, one of the most compelling ways to understand how the brain works is to study how it fails. Brain damage, irrational decisions, sensory illusions: internal or external changes that make the brain fail are how we find how the brain succeeds. Failure is used to understand complex systems beyond neuroscience: reverse-engineering computer software, understanding animal behavior, identifying solid materials. Failure even defines Science itself. an hypothesis is considered scientific if and only if it is “falsifiable”: if it can reproducibly fail (Popper (1934)). Why default to observing failures when we don’t know what is going on? Because the success-failure boundary is full of information. Let me define “failure” in the context of this discussion. Imagine being an ant dropped somewhere on top of Fig.1-(a). What is the fastest way to map your surroundings? Rather than walking every inch of the surface, find boundaries. When you are investigating a complex system that is working as expected, you are an ant dropped on Mount Success. To find the boundary, you have to push the system into failure mode. Staying inside the success space can inform you about the robustness of the system to perturbations (at best the system recovered from the perturbation, at worst your perturbation was irrelevant), but it is not explanatory. Neither is going from failure to failure. You can only investigate causes and effects if your intervention actually changes something: the failure boundary is not just more informative, it is a different kind of information altogether. Boundaries and failures are not exactly the same. If you are observing a function of the system that does not change when it crosses the failure boundary, you will not notice the transition. If you are observing the right function, you might see the system performance on that function become better or worse. Let us call “failure points” abrupt transitions from “some performance” to 0: they are the most salient of transitions. Going back to Fig.1, the ant might not notice the transition from a gentle slope to a flat terrain, but a cliff will be noticeable. Ideally, you would want to map the entire failure boundary; in practice, you will focus on failure points. Mount Success Failure Bog
Malicious and Low Credibility URLs on Twitter during COVID-19

This study provides an in-depth analysis of a Twitter dataset around AstraZeneca COVID vaccine development released as a part of Grand Challenge, North American Social Network Conference, 2021. In brief, we show: i) the presence of malicious and low credibility information sources shared on Twitter messages in multiple languages, ii) the malicious URLs, often in shortened forms, are increasingly hosted in content delivery networks and shared cloud hosting infrastructures not only to improve reach but also to avoid being detected and blocked, iii) potential signs of coordination to promote both malicious and low credibility URLs on Twitter. We use a null model and several statistical tests to identify meaningful coordination behavior.
Performance-Optimized Components for Quantum Technologies via Additive Manufacturing

Novel quantum technologies and devices place unprecedented demands on the performance of experimental components, while their widespread deployment beyond the laboratory necessitates increased robustness and fast, affordable production. We show how the use of additive manufacturing, together with mathematical optimization techniques and innovative designs, allows the production of compact, lightweight components with greatly enhanced performance. We use such components to produce a magneto-optical trap that captures $\sim 2 \times 10^8$ rubidium atoms, employing for this purpose a compact and highly stable device for spectroscopy and optical power distribution, optimized neodymium magnet arrays for magnetic field generation, and a lightweight, additively manufactured ultra-high vacuum chamber. We show how the use of additive manufacturing enables substantial weight reduction and stability enhancement, while also illustrating the transferability of our approach to experiments and devices across the quantum technology sector and beyond.
M-subdwarf Research. II. Atmospheric Parameters and Kinematics

We applied the revised M subdwarf classification criteria discussed in Zhang et al. to Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST) DR7 and combined the result with the M subdwarf sample from Savcheva et al. to construct a new M subdwarf sample for further study. The atmospheric parameters for each object were derived from fitting to the PHOENIX grid, and the sources with available astrometry and photometry from Gaia DR2 were combined for further analysis. The relationship between the gravity and metallicity was explored according to the locus both in the color–absolute magnitude diagram and the reduced proper motion diagram. Objects that have both the largest gravity and the lowest metallicity are located away from the main-sequence cloud and may be considered as the intrinsic M subdwarfs, which can be classified as luminosity class VI. Another group of objects whose spectra show typical M subdwarf characteristics have lower gravity and relatively moderate metal deficiency and occupy part of the ordinary M dwarf region in both diagrams. The Galactic U, V, W space velocity components and their dispersion show that the local Galactic halo population sampled in the solar neighborhood is represented by objects of high gravity and an inconspicuous bimodal metallicity distribution, with a fraction of prograde orbits. The other M subdwarfs seem to belong in part to the thick disk component, with a significant fraction of thin disk, moderately metal-poor objects intricately mixed with them. However, selection effects, especially the favored anticenter direction of investigation in the LAMOST subsample, as well as contamination by multiplicity and parameter coupling, could play important roles and need to be investigated further.
Synthesis of P(St-MMA-AA) Colloids and Application in Colloidal Crystals

A series of monodispersed P(St-MMA-AA) colloids with diameters from 200 nm to 500 nm was synthesized successfully using emulsion polymerization. The corresponding colloidal crystals were fabricated and investigated. UV-Vis reflectance spectra showed that the band gaps of the obtained colloidal crystals accord well with Bragg law. The height, phase and spectrum 2D images of the obtained binary colloidal crystals were studied using different modes of atom force microscopy.