Miracle-dz/UniFAR-FTUs huggingface dataset:
Offline Learning of Counterfactual Perception as Prediction for Real-World Robotic Reinforcement Learning

We propose a method for offline learning of counterfactual predictions to address real world robotic reinforcement learning challenges. The proposed method encodes action-oriented visual observations as several "what if" questions learned offline from prior experience using reinforcement learning methods. These "what if" questions counterfactually predict how action-conditioned observation would evolve on multiple temporal scales if the agent were to stick to its current action. We show that combining these offline counterfactual predictions along with online in-situ observations (e.g. force feedback) allows efficient policy learning with only a sparse terminal (success/failure) reward. We argue that the learned predictions form an effective representation of the visual task, and guide the online exploration towards high-potential success interactions (e.g. contact-rich regions). Experiments were conducted in both simulation and real-world scenarios for evaluation. Our results demonstrate that it is practical to train a reinforcement learning agent to perform real-world fine manipulation in about half a day, without hand engineered perception systems or calibrated instrumentation. Recordings of the real robot training can be found via this https URL.
Maximizing power and velocity of an information engine

Significance Around 100 y ago, Szilard imagined how to raise a weight without doing any work, just using the information gained by “looking” at a single gas molecule bouncing inside a box. Here, we designed an engine that stores energy by raising a bead against gravity, driven purely by information about the bead position. No work is done directly on the bead; instead, all dissipation occurs in the measuring apparatus. By choosing the right size of bead and through careful design, we improved information-engine performance by more than a factor of 10 over the best previous designs. We achieve a maximum velocity of 190 μm/s and maximum power of 103 kBT/s, numbers comparable to those measured in microorganisms. Information-driven engines that rectify thermal fluctuations are a modern realization of the Maxwell-demon thought experiment. We introduce a simple design based on a heavy colloidal particle, held by an optical trap and immersed in water. Using a carefully designed feedback loop, our experimental realization of an “information ratchet” takes advantage of favorable “up” fluctuations to lift a weight against gravity, storing potential energy without doing external work. By optimizing the ratchet design for performance via a simple theory, we find that the rate of work storage and velocity of directed motion are limited only by the physical parameters of the engine: the size of the particle, stiffness of the ratchet spring, friction produced by the motion, and temperature of the surrounding medium. Notably, because performance saturates with increasing frequency of observations, the measurement process is not a limiting factor. The extracted power and velocity are at least an order of magnitude higher than in previously reported engines.
Characterising and Detecting Sponsored Influencer Posts on Instagram

Recent years have seen a new form of advertisement campaigns emerge: those involving so-called social media influencers. These influencers accept money in return for promoting products via their social media feeds. We gather a large-scale Instagram dataset covering thousands of accounts advertising products, and create a categorisation based on the number of users they reach. We then provide a detailed analysis of the types of products being advertised by these accounts, their potential reach, and the engagement they receive from their followers. Based on our findings, we train machine learning models to distinguish sponsored content from non-sponsored, and identify cases where people are generating sponsored posts without labelling them.
Docking two multirotors in midair using relative vision measurements

Modular robots have been rising in popularity for a variety of applications, and autonomous midair docking is a necessary task for real world deployment of these robots. We present a state estimator based on the extended Kalman filter for relative localization of one multirotor with respect to another using only onboard sensors, specifically an inertial measurement unit and a camera-marker pair. Acceleration and angular velocity measurements along with relative pose measurements from a camera on the first multirotor looking at a marker on the second multirotor are used to estimate the relative position and velocity of the first multirotor with respect to the second, and the absolute attitude of the first multirotor. We also present a control architecture to use these onboard state estimates to control the first multirotor at a desired setpoint with respect to the second. The performance of the estimator and control architecture are experimentally validated by successfully and repeatably performing midair docking -- a task that requires relative position precision on the order of a centimeter.
Sim-To-Real Transfer for Miniature Autonomous Car Racing

Sim-to-real, a term that describes where a model is trained in a simulator then transferred to the real world, is a technique that enables faster deep reinforcement learning (DRL) training. However, differences between the simulator and the real world often cause the model to perform poorly in the real world. Domain randomization is a way to bridge the sim-to-real gap by exposing the model to a wide range of scenarios so that it can generalize to real-world situations. However, following domain randomization to train an autonomous car racing model with DRL can lead to undesirable outcomes. Namely, a model trained with randomization tends to run slower; a higher completion rate on the testing track comes at the expense of longer lap times. This paper aims to boost the robustness of a trained race car model without compromising racing lap times. For a training track and a testing track having the same shape (and same optimal paths), but with different lighting, background, etc., we first train a model (teacher model) that overfits the training track, moving along a near optimal path. We then use this model to teach a student model the correct actions along with randomization. With our method, a model with 18.4\% completion rate on the testing track is able to help teach a student model with 52\% completion. Moreover, over an average of 50 trials, the student is able to finish a lap 0.23 seconds faster than the teacher. This 0.23 second gap is significant in tight races, with lap times of about 10 to 12 seconds.
Evaluating and Ranking Criteria for Mathematics Graduate Education Admissions through the Analytic Hierarchy Process

Graduate school admission committees consider many factors for admission into a mathematics PhD program, and aspiring applicants often wonder which factors are more important. Applicants discerning where to best dedicate their time may ask questions such as, "should I take graduate level courses or participate in research? " This paper seeks to answer such questions by constructing an ordinal ranking of admission criteria and provide insight on why some factors are more valued. Using a conjoint analysis method called Analytic Hierarchy Process from mathematician Thomas Saaty, this paper evaluates the relative influence certain predictors have on admissions into a mathematics PhD program. Additionally, this paper analyzes the difference in factor rankings between varying populations. For instance, results indicate that full professors tend to differ from their associate and assistant peers in their criteria rankings, and top 30 math PhD programs also differ in rankings when compared to programs outside the top 30. It is my hope that the ordinal rankings and its subsequent analyses will not only explain why some factors are valued more by the mathematics community but also provide some guidance to potential graduate school applicants on where to dedicate their time as undergraduates
Optimizing the Age-of-Information for Mobile Users in Adversarial and Stochastic Environments

We study a multi-user downlink scheduling problem for optimizing the freshness of information available to users roaming across multiple cells. We consider both adversarial and stochastic settings and design scheduling policies that optimize two distinct information freshness metrics, namely the average age-of-information and the peak age-of-information. We show that a natural greedy scheduling policy is competitive with the optimal offline policy in the adversarial setting. We also derive fundamental lower bounds to the competitive ratio achievable by any online policy. In the stochastic environment, we show that a Max-Weight scheduling policy that takes into account the channel statistics achieves an approximation factor of $2$ for minimizing the average age of information in two extreme mobility scenarios. We conclude the paper by establishing a large-deviation optimality result achieved by the greedy policy for minimizing the peak age of information for static users situated at a single cell.
Nonsymmetric Macdonald Superpolynomials

There are representations of the type-A Hecke algebra on spaces of polynomials in anti-commuting variables. Luque and the author [Sém. Lothar. Combin. 66 (2012), Art. B66b, 68 pages, arXiv:1106.0875] constructed nonsymmetric Macdonald polynomials taking values in arbitrary modules of the Hecke algebra. In this paper the two ideas are combined to define and study nonsymmetric Macdonald polynomials taking values in the aforementioned anti-commuting polynomials, in other words, superpolynomials. The modules, their orthogonal bases and their properties are first derived. In terms of the standard Young tableau approach to representations these modules correspond to hook tableaux. The details of the Dunkl-Luque theory and the particular application are presented. There is an inner product on the polynomials for which the Macdonald polynomials are mutually orthogonal. The squared norms for this product are determined. By using techniques of Baker and Forrester [Ann. Comb. 3 (1999), 159-170, arXiv:q-alg/9707001] symmetric Macdonald polynomials are built up from the nonsymmetric theory. Here ''symmetric'' means in the Hecke algebra sense, not in the classical group sense. There is a concise formula for the squared norm of the minimal symmetric polynomial, and some formulas for anti-symmetric polynomials. For both symmetric and anti-symmetric polynomials there is a factorization when the polynomials are evaluated at special points.
Zero-Shot Terrain Generalization for Visual Locomotion Policies

Legged robots have unparalleled mobility on unstructured terrains. However, it remains an open challenge to design locomotion controllers that can operate in a large variety of environments. In this paper, we address this challenge of automatically learning locomotion controllers that can generalize to a diverse collection of terrains often encountered in the real world. We frame this challenge as a multi-task reinforcement learning problem and define each task as a type of terrain that the robot needs to traverse. We propose an end-to-end learning approach that makes direct use of the raw exteroceptive inputs gathered from a simulated 3D LiDAR sensor, thus circumventing the need for ground-truth heightmaps or preprocessing of perception information. As a result, the learned controller demonstrates excellent zero-shot generalization capabilities and can navigate 13 different environments, including stairs, rugged land, cluttered offices, and indoor spaces with humans.
Dust Temperature of Compact Star-forming Galaxies at z ∼ 1–3 in 3D-HST/CANDELS

Recent simulation studies suggest that the compaction of star-forming galaxies (SFGs) at high redshift might be a critical process, during which the central bulge is being rapidly built, followed by quenching of the star formation. To explore dust properties of SFGs with compact morphology, we investigate the dependence of dust temperature, T dust, on their size and star formation activity, using a sample of massive SFGs with at 1 < z < 3, drawn from the 3D-HST/CANDELS database in combination with deep Herschel observations. T dust is derived via fitting the mid-to-far-infrared photometry with a mid-infrared power law and a far-infrared modified blackbody. We find that both extended and compact SFGs generally follow a similar T dust − z evolutionary track as that of the main-sequence galaxies. The compact SFGs seem to share similar dust temperature with extended SFGs. Despite the frequent occurrence of active galactic nuclei (AGNs) in compact SFGs, we do not observe any effect on dust caused by the presence of AGNs in these galaxies during the compaction. Our results disfavor different ISM properties between compact and extended SFGs, suggesting that a rapid and violent compaction process might be not necessary for the formation of compact SFGs.
Deep Neural Networks and Neuro-Fuzzy Networks for Intellectual Analysis of Economic Systems

In tis paper we consider approaches for time series forecasting based on deep neural networks and neuro-fuzzy nets. Also, we make short review of researches in forecasting based on various models of ANFIS models. Deep Learning has proven to be an effective method for making highly accurate predictions from complex data sources. Also, we propose our models of DL and Neuro-Fuzzy Networks for this task. Finally, we show possibility of using these models for data science tasks. This paper presents also an overview of approaches for incorporating rule-based methodology into deep learning neural networks.
Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences

Multi-agent reinforcement learning (MARL) has shown recent success in increasingly complex fixed-team zero-sum environments. However, the real world is not zero-sum nor does it have fixed teams; humans face numerous social dilemmas and must learn when to cooperate and when to compete. To successfully deploy agents into the human world, it may be important that they be able to understand and help in our conflicts. Unfortunately, selfish MARL agents typically fail when faced with social dilemmas. In this work, we show evidence of emergent direct reciprocity, indirect reciprocity and reputation, and team formation when training agents with randomized uncertain social preferences (RUSP), a novel environment augmentation that expands the distribution of environments agents play in. RUSP is generic and scalable; it can be applied to any multi-agent environment without changing the original underlying game dynamics or objectives. In particular, we show that with RUSP these behaviors can emerge and lead to higher social welfare equilibria in both classic abstract social dilemmas like Iterated Prisoner's Dilemma as well in more complex intertemporal environments.
Dynamical constraints on the solsticial Hadley Cell ascending edge in Earth's macroturbulent atmosphere.

How far the ascending branch of the Hadley circulation extends into the summer hemisphere is a fundamental but incompletely understood characteristic of Earth's solsticial general circulation. Here, we present a predictive, analytical theory for this ascending edge latitude based on the extent of so-called supercritical forcing. Supercriticality sets the minimal extent of a large-scale circulation based on the angular momentum and absolute vorticity distributions of the hypothetical state were the circulation absent. Despite its origins in axisymmetric theory, supercriticality constitutes a conceptually valid predictor for the ascending edge even in zonally varying, macroturbulent atmospheres like Earth's, provided certain empirical conditions are met. Numerical simulations of latitude-by-latitude radiative-convective equilibrium (RCE) under Earth's solsticial forcing show that the supercritical forcing extent aligns well with the observed climatological boreal summer ascending edge. A simple analytical approximation to the solsticial RCE temperature profile, from Lindzen and Hou (1988) but with the temperature maximum located far off-equator, accurately approximates the RCE state and thereby the supercritical forcing extent. The accuracy of the resulting analytical predictor for the solsticial ascending edge is confirmed in moist and dry idealized GCMs under solsticial forcing with varying planetary rotation rate, $\Omega$. In particular, in the small-angle limit appropriate for Earth, the simulated ascending edges exhibit the theory's predicted $\Omega^{-2/3}$ scaling.
Gaussian Compression Stream: Principle and Preliminary Results

Random projections became popular tools to process big data. In particular, when applied to Nonnegative Matrix Factorization (NMF), it was shown that structured random projections were far more efficient than classical strategies based on Gaussian compression. However, they remain costly and might not fully benefit from recent fast random projection techniques. In this paper, we thus investigate an alternative to structured ran-om projections-named Gaussian compression stream-which (i) is based on Gaussian compressions only, (ii) can benefit from the above fast techniques, and (iii) is shown to be well-suited to NMF.
Optimal synchronization control for heterogeneous multi‐agent systems: Online adaptive learning solutions

This paper presents an online adaptive learning solution to the optimal synchronization control problem of heterogeneous multi‐agent systems via a novel distributed policy iteration approach. For the leader‐follower multi‐agents, the dynamics of all the followers are heterogeneous with leader disturbance. To make the output of each follower synchronize with the leader's output, we propose a synchronization control protocol where the stability conditions for selecting the feedback gains are given. Next, with a minimization of the output tracking errors, we optimize the feedback gains for the synchronization control protocol, and the unique solutions for those feedback gains are learned via a novel distributed policy iteration approach. We prove that the proposed online adaptive learning solution will converge to the optimal control solution under some mild conditions. Finally, an illustrative numerical example is provided to show the effectiveness of the approach.
Influencing dynamics on social networks without knowledge of network microstructure

Social network-based information campaigns can be used for promoting beneficial health behaviours and mitigating polarization (e.g. regarding climate change or vaccines). Network-based intervention strategies typically rely on full knowledge of network structure. It is largely not possible or desirable to obtain population-level social network data due to availability and privacy issues. It is easier to obtain information about individuals’ attributes (e.g. age, income), which are jointly informative of an individual’s opinions and their social network position. We investigate strategies for influencing the system state in a statistical mechanics based model of opinion formation. Using synthetic and data-based examples we illustrate the advantages of implementing coarse-grained influence strategies on Ising models with modular structure in the presence of external fields. Our work provides a scalable methodology for influencing Ising systems on large graphs and the first exploration of the Ising influence problem in the presence of ambient (social) fields. By exploiting the observation that strong ambient fields can simplify control of networked dynamics, our findings open the possibility of efficiently computing and implementing public information campaigns using insights from social network theory without costly or invasive levels of data collection.
E-commerce Query-based Generation based on User Review

With the increasing number of merchandise on e-commerce platforms, users tend to refer to reviews of other shoppers to decide which product they should buy. However, with so many reviews of a product, users often have to spend lots of time browsing through reviews talking about product attributes they do not care about. We want to establish a system that can automatically summarize and answer user's product specific questions. In this study, we propose a novel seq2seq based text generation model to generate answers to user's question based on reviews posted by previous users. Given a user question and/or target sentiment polarity, we extract aspects of interest and generate an answer that summarizes previous relevant user reviews. Specifically, our model performs attention between input reviews and target aspects during encoding and is conditioned on both review rating and input context during decoding. We also incorporate a pre-trained auxiliary rating classifier to improve model performance and accelerate convergence during training. Experiments using real-world e-commerce dataset show that our model achieves improvement in performance compared to previously introduced models.
Assessment of text coherence based on the cohesion estimation

In this paper, a graph-based coherence estimation method based on the cohesion estimation is suggested. Our method uses a graph-based approach to provide a user with an understanding of the evaluation process. Moreover, it can be applied to different languages, therefore, the effectiveness of this method is examined on the set of English, Chinese, and Arabic texts.
The Impact of Text Presentation on Translator Performance

Widely used computer-aided translation (CAT) tools divide documents into segments, such as sentences, and arrange them side-by-side in a spreadsheet-like view. We present the first controlled evaluation of these design choices on translator performance, measuring speed and accuracy in three experimental text-processing tasks. We find significant evidence that sentence-by-sentence presentation enables faster text reproduction and within-sentence error identification compared to unsegmented text, and that a top-and-bottom arrangement of source and target sentences enables faster text reproduction compared to a side-by-side arrangement. For revision, on the other hand, we find that presenting unsegmented text results in the highest accuracy and time efficiency. Our findings have direct implications for best practices in designing CAT tools.
A Novel Privacy-Preserved Recommender System Framework Based On Federated Learning

Recommender System (RS) is currently an effective way to solve information overload. To meet users' next click behavior, RS needs to collect users' personal information and behavior to achieve a comprehensive and profound user preference perception. However, these centrally collected data are privacy-sensitive, and any leakage may cause severe problems to both users and service providers. This paper proposed a novel privacy-preserved recommender system framework (PPRSF), through the application of federated learning paradigm, to enable the recommendation algorithm to be trained and carry out inference without centrally collecting users' private data. The PPRSF not only able to reduces the privacy leakage risk, satisfies legal and regulatory requirements but also allows various recommendation algorithms to be applied.