Miracle-dz/UniFAR-FTUs huggingface dataset:
Descriptive and Prescriptive Visual Guidance to Improve Shared Situational Awareness in Human-Robot Teaming

In collaborative tasks involving human and robotic teammates, live communication between agents has potential to substantially improve task efficiency and fluency. Effective communication provides essential situational awareness to adapt successfully during uncertain situations and encourage informed decision-making. In contrast, poor communication can lead to incongruous mental models resulting in mistrust and failures. In this work, we first introduce characterizations of and generative algorithms for two complementary modalities of visual guidance: prescriptive guidance (visualizing recommended actions), and descriptive guidance (visualizing state space information to aid in decision-making). Robots can communicate this guidance to human teammates via augmented reality (AR) interfaces, facilitating synchronization of notions of environmental uncertainty and offering more collaborative and interpretable recommendations. We also introduce a min-entropy multi-agent collaborative planning algorithm for uncertain environments, informing the generation of these proactive visual recommendations for more informed human decision-making. We illustrate the effectiveness of our algorithm and compare these different modalities of AR-based guidance in a human subjects study involving a collaborative, partially observable search task. Finally, we synthesize our findings into actionable insights informing the use of prescriptive and descriptive visual guidance.
Optimal Local Bayesian Differential Privacy over Markov Chains

In the literature of data privacy, differential privacy is the most popular model. An algorithm is differentially private if its outputs with and without any individual's data are indistinguishable. In this paper, we focus on data generated from a Markov chain and argue that Bayesian differential privacy (BDP) offers more meaningful guarantees in this context. Our main theoretical contribution is providing a mechanism for achieving BDP when data is drawn from a binary Markov chain. We improve on the state-of-the-art BDP mechanism and show that our mechanism provides the optimal noise-privacy tradeoffs for any local mechanism up to negligible factors. We also briefly discuss a non-local mechanism which adds correlated noise. Lastly, we perform experiments on synthetic data that detail when DP is insufficient, and experiments on real data to show that our privacy guarantees are robust to underlying distributions that are not simple Markov chains.
Speeding up Deep Reinforcement Learning through Influence-Augmented Local Simulators

Learning effective policies for real-world problems is still an open challenge for the field of reinforcement learning (RL). The main limitation being the amount of data needed and the pace at which that data can be obtained. In this paper, we study how to build light-weight simulators of complicated systems that can run sufficiently fast for deep RL to be applicable. We focus on domains where agents interact with a reduced portion of a larger environment while still being affected by the global dynamics. Our method combines the use of local simulators with learned models that mimic the influence of the global system. The experiments reveal that incorporating this idea into the deep RL workflow can considerably accelerate the training process and presents several opportunities for the future
Multi-agent Covering Option Discovery through Kronecker Product of Factor Graphs

Covering option discovery has been developed to improve the exploration of reinforcement learning in single-agent scenarios with sparse reward signals, through connecting the most distant states in the embedding space provided by the Fiedler vector of the state transition graph. However, due to exponentially-large state space resulted from multi-agent scenarios, existing researches on adopting options in multi-agent scenarios still rely on single-agent option discovery and fail to directly discover the joint options that can improve the connectivity of the joint state space of agents. In this paper, we show that it is indeed possible to directly compute multi-agent options with collaborative exploratory behavior while still enjoying the ease of decomposition. Our key idea is to approximate the joint state space as a kronecker graph – the kronecker product of individual agents’ state transition graphs, based on which we can directly estimate the Fiedler vector of the joint state space using the Laplacian spectrum of individual agents’ transition graphs. This decomposition enables us to efficiently construct multi-agent joint options by encouraging agents to connect the sub-goal joint states which are corresponding to the minimum or maximum values of the estimated joint Fiedler vector. The evaluation based on multi-agent collaborative tasks shows that the proposed algorithm can successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options, in terms of both faster exploration and higher cumulative rewards.
ASM-PPO: Asynchronous and Scalable Multi-Agent PPO for Cooperative Charging

Wireless Rechargeable Sensor Networks (WRSNs) are especially promising in large-area monitoring tasks that are previously impossible to complete by traditional Wireless Sensor Networks (WSNs). Mobile Chargers (MCs) in WRSNs are to cooperatively charge battery drained sensor nodes high efficiently and with a guarantee of sensors survival. Considering the unpredictability and high dynamics of WRSNs during the charging process, Multi-Agent Reinforcement Learning (MARL) is an attractive alternative to schedule the cooperation among MCs. However, most existing MARL methods are based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDP), a general framework to describe decentralized agents making decisions at the same time step. Nevertheless, MCs in WRSNs perform charging asynchronously since the charging time of each sensor node varies. To address the problem of asynchronous behavior, we first formulate an Asynchronous Dec-POMDP (AD-POMDP). We then propose an algorithm called Asynchronous and Scalable Multi-agent Proximal Policy Optimization (ASM-PPO) that allows asynchronous learning and decision-making in AD-POMDP based on two popular multi-agent reinforcement learning methods in Dec-POMDP. Furthermore, ASM-PPO takes advantage of the translation invariance in WRSNs to avoid the huge input space dimensions caused by centralized training. The evaluation results not only indicate that our method achieves much charging efficiency and the longer lifetime of sensor nodes, but also demonstrate that ASM-PPO has advantages in terms of stability and scalability over existing methods
A Hierarchical Bayesian Process for Inverse RL in Partially-Controlled Environments

Robots learning from observations in the real world may encounter objects or agents in the environment, other than the expert giv-ing the demonstration, that cause nuisance observations. These confounding elements are typically removed in fully-controlled environments such as virtual simulations or lab settings. When complete removal is impossible the nuisance observations must be filtered out. However, identifying the sources of observations when large amounts of observations are made is difficult. To address this, we present a hierarchical Bayesian process that models both the expert’s and the confounding elements’ observations thereby explicitly modeling the diverse observations a robot may receive. We extend an existing inverse reinforcement learning algorithm originally designed to work under partial occlusion of the expert to consider the diverse and noisy observations. In a simulated robotic produce-sorting domain containing both occlusion and confounding elements, we demonstrate the model’s effectiveness. In particu-lar, our technique outperforms several other comparative methods, second only to having perfect knowledge of the subject’s trajectory.
Optimizing Multi-Agent Coordination via Hierarchical Graph Probabilistic Recursive Reasoning

Multi-agent reinforcement learning (MARL) requires coordination by some means of interaction between agents to efficiently solve tasks. Interaction graphs allow reasoning about joint actions based on the local structure of interactions, but they disregard the potential impact of an agent’s action on its neighbors’ behaviors, which could rapidly alter in dynamic settings. In this paper, we thus present a novel perspective on opponent modeling in domains with only local interactions using (level-1) Graph Probabilistic Recursive Reasoning ( GrPR2 ). Unlike previous work on recursive reasoning, each agent iteratively best-responds to other agents’ policies over all possible local interactions . Agents’ policies are approximated via a variational Bayes scheme for capturing their uncertainties, and we prove that an induced variant of Q-learning converges under self-play when there exists only one Nash equilibrium. In cooperative settings, we further devise a variational lower bound on the likelihood of each agent’s optimality. Opposed to other models, optimizing the resulting objective prevents each agent from attaining an unrealistic modelling of others, and yields an exact tabular Q-iteration method that holds convergence guarantees. Then, we deepen the recursion to level-𝑘 via Cognitive Hierarchy GrPR2 ( GrPR2-CH ), which lets each level-𝑘 player best-respond to a mixture of strictly lower levels in the hierarchy. We prove that: (1) level-3 reasoning is the optimal hierarchical level , maximizing each agent’s expected return; and (2) the weak spot of the classical CH models is that 0 -level is uniformly distributed , as it may introduce policy bias. Finally, we propose a practical actor-critic scheme, and illustrate that GrPR2-CH outperforms strong MARL baselines in the particle environment.
Autonomous Swarm Shepherding Using Curriculum-Based Reinforcement Learning

Autonomous shepherding is a bio-inspired swarm guidance approach, whereby an artificial sheepdog guides a swarm of artificial or biological agents, such as sheep, towards a goal. While the success in this guidance depends on the set of behaviours exhibited by the sheepdog, the main source of complexity for learning effective behaviours lies within the highly non-linear dynamics featured among the swarm members as well as between the swarm and the sheepdog. Attempts to apply reinforcement learning (RL) to shepherding have so far relied greatly on rule-based algorithms for calculating waypoints to guide the RL algorithm. In this paper, we propose a curriculum-based approach for RL that does not rely on any external algorithm to pre-determine waypoints for the sheep-dog. Instead, the approach uses task decomposition by formulating shepherding in terms of two sub-tasks: (1) pushing an agent from a start to a target location and (2) selecting between collecting scattered agents or driving the biggest cluster of agents to the goal. Simple-to-complex curriculum learning is used to accelerate the learning of each sub-task. For the first sub-task, the complexity is gradually increased over training time, whereas for the second sub-task a simplified environment is designed for initial learning before proceeding with the main environment. The proposed approach results in high-performance shepherding with a success rate of about 96%. While curriculum learning was found to expedite the learning of the first sub-task, it was not as efficient for the second sub-task. Our analyses highlight the need for the careful design of the curriculum to ensure that skills acquired in intermediate tasks are useful for the main tasks.
Planning, Execution, and Adaptation for Multi-Robot Systems using Probabilistic and Temporal Planning

Planning for multi-robot coordination during long horizon missions in complex environments need to consider resources, temporal constraints, and uncertainty. This could be computationally expensive and impractical for online planning and execution. We propose a decoupled framework to address this. At the high-level, we plan for multi-robot missions that require coordination amongst robots considering temporal and numeric constraints. The temporal plan is decomposed into low-level plans for individual robots. At the low-level, we perform online learning and adaptation due to unexpected probabilistic outcomes to achieve mission goals. Our framework learns over time to improve the performance by (1) updating the learned domain model to reduce model prediction errors and (2) constraining the robot’s capabilities which in turn improves goal allocation. The approach provides a solution to planning problems that require long-term robot operability. We demonstrate the performance of our approach via experiments involving a fleet of heterogeneous robots
The Long Non-Coding RNA FAM222A-AS1 Negatively Modulates MiR-Let-7f to Promote Colorectal Cancer Progression

Accumulating evidence indicates that lncRNAs are potential biomarkers and key regulators of tumor development and progression. The present study aimed to screen abnormal expression lncRNAs and investigate the mechanisms underlying the function in the progression of colorectal cancer (CRC). Potential CRC prognosis-associated dysregulated lncRNAs were screened and identified using bioinformatics analysis. Loss/gain-of-function experiments were performed to detect the biological roles of FAM222A-AS1 in CRC cell phenotypes in vitro and in vivo. The potential microRNAs that interact with FAM222A-AS1 were identified using online tools and were verified using qRT-PCR and luciferase reporter assay. The expression of FAM222A-AS1 is significantly upregulated in CRC tumor samples and cell lines. CRC patients with elevated FAM222A-AS1 expression in the tumor samples had unfavorable overall survival and disease-free survival. Silencing FAM222A-AS1 expression significantly inhibited CRC cell proliferation, migration, and invasion both in vitro and in vivo. Furthermore, FAM222A-AS1 was mainly distributed in the cytoplasm. It may directly bound to miR-let-7f and inhibit its expression and upregulate MYH9. In summary, FAM222A-AS1, as a novel oncogene in CRC, may promote the CRC progression by inhibiting miR-let-7f/MYH9 axis. The FAM222A-AS1/miR-let-7f/MYH9 signaling pathway may be a novel valuable target for inhibiting CRC.
Research on Quantitative Model of Brand Recognition Based on Sentiment Analysis of Big Data

This paper takes laptops as an example to carry out research on quantitative model of brand recognition based on sentiment analysis of big data. The basic idea is to use web crawler technology to obtain the most authentic and direct information of different laptop brands from first-line consumers from public spaces such as buyer reviews of major e-commerce platforms, including review time, text reviews, satisfaction ratings and relevant user information, etc., and then analyzes consumers’ sentimental tendencies and recognition status of the product brands. This study extracted a total of 437,815 user reviews of laptops from e-commerce platforms from January 1, 2019 to December 31, 2021, and performed data preprocessing on the obtained review data, followed by sentiment dictionary construction, attribute expansion, text quantification and algorithm evaluation. This paper analyzed the information receiving and processing hierarchy of the quantitative model of brand recognition, discussed the interactive relationship between brand recognition and consumer sentiment, discussed the brand recognition bias, style and demand in the context of big data, and performed the sentiment statistics and dimension analysis in the quantitative model of brand recognition. The study results show that the quantitative model of brand recognition based on sentiment analysis of big data can transform and map the keywords in text to word vectors in the high-dimensional semantic space by performing unsupervised machine learning on the text based on artificial neural network computer bionic metaphors; the model can accumulate each brand-related buyer review in the corresponding brand recognition dimension, so as to obtain the value of each product in each dimension of brand recognition; finally, the model will add the values of each dimension of brand recognition, that is, obtain the relevant value of the sum of each brand recognition. The results of this paper may provide a reference for further research on the quantitative model of brand recognition based on sentiment analysis of big data.
Perspective: National Football League Teams Need Chief Diversity Officers

The National Football League (NFL) and its teams, some of the world's most profitable sporting properties, face challenges with diversity, equity, and inclusion (DEI). With a history of discriminatory work environments, including a recent high-profile lawsuit, the NFL and its teams have a poor reputation on these issues. This perspective piece investigated NFL teams' utilization of organization employees dedicated to DEI. Utilizing a content analysis of publicly available data, this piece investigated DEI employees at NFL team organizations. The study analyzed the position's characteristics including the name of the role, the department in which it was housed, and the reporting structure. The study also examined the demographics and professional background of the employees in the roles. The findings conclude that NFL teams lag behind other American businesses in their adoption of Chief Diversity Officer (CDO) roles. As of 2022, only 31.25% of NFL teams had a dedicated DEI staff person. Three additional teams host diversity councils utilizing employees with other job responsibilities. The employees filling the CDO roles were majority women and majority Black. Though not the only answer to a complex problem, in order to address these challenges and move forward, the NFL teams should create Chief Diversity Officer (CDO) roles. These positions should have appropriate reporting relationships, well-crafted position responsibilities, generous resources, and qualified and experienced employees.
Stochastic Physiological Gaze-Evoked Nystagmus With Slow Centripetal Drift During Fixational Eye Movements at Small Gaze Eccentricities

Involuntary eye movement during gaze (GZ) fixation, referred to as fixational eye movement (FEM), consists of two types of components: a Brownian motion like component called drifts-tremor (DRT) and a ballistic component called microsaccade (MS) with a mean saccadic amplitude of about 0.3° and a mean inter-MS interval of about 0.5 s. During GZ fixation in healthy people in an eccentric position, typically with an eccentricity more than 30°, eyes exhibit oscillatory movements alternating between centripetal drift and centrifugal saccade with a mean saccadic amplitude of about 1° and a period in the range of 0.5–1.0 s, which has been known as the physiological gaze-evoked nystagmus (GEN). Here, we designed a simple experimental paradigm of GZ fixation on a target shifted horizontally from the front-facing position with fewer eccentricities. We found a clear tendency of centripetal DRT and centrifugal MS as in GEN, but with more stochasticity and with slower drift velocity compared to GEN, even during FEM at GZ positions with small eccentricities. Our results showed that the target shift-dependent balance between DRT and MS achieves the GZ bounded around each of the given targets. In other words, GZ relaxes slowly with the centripetal DRT toward the front-facing position during inter-MS intervals, as if there always exists a quasi-stable equilibrium posture in the front-facing position, and MS actions pull GZ intermittently back to the target position in the opposite direction to DRT.
An Integrated Systematic Analysis and the Clinical Significance of Hepcidin in Common Malignancies of the Male Genitourinary System

Tumors of the male genitourinary system are of great concern to the health of men worldwide. Although emerging experiment-based evidence indicates an association between hepcidin and such cancers, an integrated analysis is still lacking. For this reason, in this study, we determined the underlying oncogenic functions of hepcidin in common male genitourinary system tumors, including bladder urothelial carcinoma (BLCA), kidney chromophobe (KICH), kidney renal clear cell carcinoma (KIRC), kidney renal papillary cell carcinoma (KIRP), prostate adenocarcinoma (PRAD), and testicular germ cell tumors (TGCT) according to the data from The Cancer Genome Atlas. We found that hepcidin was highly expressed in kidney and testicular cancers. Meanwhile, the expression level of hepcidin was distinctly associated with the prognosis and immune cell infiltration in male patients with certain genitourinary system cancers, especially in KIRC. Elevated hepcidin levels also present as a risk factor in male genitourinary system tumors. Moreover, enrichment analyses revealed that some of the principal associated signaling pathways involving hepcidin and its related genes are identified as tumorigenesis-related. Immunofluorescence staining confirmed the conclusion of our immune infiltration analysis in KIRC tissue. In this study, for the first time, we provided evidence for the oncogenic function of hepcidin in different types of male genitourinary system tumors.
The Influence of Family Supportive Supervisor Behavior on Employee Creativity: The Mediating Roles of Psychological Capital and Positive Emotion

In an increasingly complex external environment, innovation is an important way for companies to build sustainable competitiveness. This research discusses employee creativity from the perspective of Family Supportive Supervisor Behavior (FSSB) based on conservation of resource theory, social exchange theory, psychological capital theory and emotional spillover theory. Through a series of surveys of employees in different companies and jobs, we can understand the impact of family-supporting supervisors’ behavior on their creativity. Combined with the survey data, a structural equation model (SEM) is constructed to analyze the mediating effects of psychological capital and positive emotions based on the causal mediation model. The research found that the positive influence of family-supporting supervisors’ behavior on employees’ creativity has three forms. First, supervisors improve employees’ motivation and sense of efficacy by providing various support resources. Second, supervisors can generate positive spillover effects among employees by influencing employees’ psychological state. Third, supervisors stimulate the creativity of subordinates by promoting work participation and mobility. According to the research conclusions, in order to improve the employee creativity, we should provide incentives to encourage supervisors to carry out family support behaviors, identify employee characteristics to help supervisors provide personalized support, cultivate family supportive leaders, and attach importance to emotional support and play the role of psychological capital and positive emotions.