Miracle-dz/UniFAR-FTUs huggingface dataset:
Towards Multimodal Multitask Scene Understanding Models for Indoor Mobile Agents

The perception system in personalized mobile agents requires developing indoor scene understanding models, which can understand 3D geometries, capture objectiveness, analyze human behaviors, etc. Nonetheless, this direction has not been well-explored in comparison with models for outdoor environments (e.g., the autonomous driving system that includes pedestrian prediction, car detection, traffic sign recognition, etc.). In this paper, we first discuss the main challenge: insufficient, or even no, labeled data for real-world indoor environments, and other challenges such as fusion between heterogeneous sources of information (e.g., RGB images and Lidar point clouds), modeling relationships between a diverse set of outputs (e.g., 3D object locations, depth estimation, and human poses), and computational efficiency. Then, we describe MMISM (Multi-modality input Multi-task output Indoor Scene understanding Model) to tackle the above challenges. MMISM considers RGB images as well as sparse Lidar points as inputs and 3D object detection, depth completion, human pose estimation, and semantic segmentation as output tasks. We show that MMISM performs on par or even better than single-task models; e.g., we improve the baseline 3D object detection results by 11.7% on the benchmark ARKitScenes dataset.
Structure in Theorem Proving: Analyzing and Improving the Isabelle Archive of Formal Proofs - Extended Abstract (short paper)

The Isabelle Archive of Formal Proofs has grown to a significant size in the past years. It makes up for an impressive body of research, which enables a number of statistical approaches to various aspects in theorem proving, and has not yet been utilized exhaustively. However, the growing size also poses some challenges to address: Material becomes increasingly harder to find, reusability and ease of understanding become more important. This thesis abstract summarizes my research plans on those topics and briefly touches on preliminary results, which indicate that the node in-degree of the dependency graph of the archive follows a scale-free distribution.
Assessing Digital Language Support on a Global Scale

The users of endangered languages struggle to thrive in a digitally-mediated world. We have developed an automated method for assessing how well every language recognized by ISO 639 is faring in terms of digital language support. The assessment is based on scraping the names of supported languages from the websites of 143 digital tools selected to represent a full range of ways that digital technology can support languages. The method uses Mokken scale analysis to produce an explainable model for quantifying digital language support and monitoring it on a global scale.
Efficient On-Device Session-Based Recommendation

On-device session-based recommendation systems have been achieving increasing attention on account of the low energy/resource consumption and privacy protection while providing promising recommendation performance. To fit the powerful neural session-based recommendation models in resource-constrained mobile devices, tensor-train decomposition and its variants have been widely applied to reduce memory footprint by decomposing the embedding table into smaller tensors, showing great potential in compressing recommendation models. However, these model compression techniques significantly increase the local inference time due to the complex process of generating index lists and a series of tensor multiplications to form item embeddings. The resultant on-device recommender fails to provide real-time responses and recommendations. To improve the online recommendation efficiency, we propose to learn compositional encoding-based compact item representations. Specifically, each item is represented by a compositional code that consists of several codewords, and we learn embedding vectors to represent each codeword instead of each item. Then the composition of the codeword embedding vectors from different embedding matrices (i.e., codebooks) forms the item embedding. Since the size of codebooks can be extremely small, the recommender model is thus able to fit in resource-constrained devices and save the codebooks for fast local inference. In addition, to prevent the loss of model capacity caused by compression, we propose a bidirectional self-supervised knowledge distillation framework. Extensive experimental results on two benchmark datasets demonstrate that compared with existing methods, the proposed on-device recommender not only achieves an 8× inference speedup with a large compression ratio but also shows superior recommendation performance. The code is released at https://github.com/xiaxin1998/EODRec.
Deep Unfolding of the DBFB Algorithm With Application to ROI CT Imaging With Limited Angular Density

This article presents a new method for reconstructing regions of interest (ROI) from a limited number of computed tomography (CT) measurements. Classical model-based iterative reconstruction methods lead to images with predictable features. Still, they often suffer from tedious parameterization and slow convergence. On the contrary, deep learning methods are fast, and they can reach high reconstruction quality by leveraging information from large datasets, but they lack interpretability. At the crossroads of both methods, deep unfolding networks have been recently proposed. Their design includes the physics of the imaging system and the steps of an iterative optimization algorithm. Motivated by the success of these networks for various applications, we introduce an unfolding neural network called U-RDBFB designed for ROI CT reconstruction from limited data. Few-view truncated data are effectively handled thanks to a robust non-convex data fidelity term combined with a sparsity-inducing regularization function. We unfold the Dual Block coordinate Forward-Backward (DBFB) algorithm, embedded in an iterative reweighted scheme, allowing the learning of key parameters in a supervised manner. Our experiments show an improvement over several state-of-the-art methods, including a model-based iterative scheme, a multi-scale deep learning architecture, and other deep unfolding methods.
Collaborative Decision Making Using Action Suggestions

The level of autonomy is increasing in systems spanning multiple domains, but these systems still experience failures. One way to mitigate the risk of failures is to integrate human oversight of the autonomous systems and rely on the human to take control when the autonomy fails. In this work, we formulate a method of collaborative decision making through action suggestions that improves action selection without taking control of the system. Our approach uses each suggestion efficiently by incorporating the implicit information shared through suggestions to modify the agent's belief and achieves better performance with fewer suggestions than naively following the suggested actions. We assume collaborative agents share the same objective and communicate through valid actions. By assuming the suggested action is dependent only on the state, we can incorporate the suggested action as an independent observation of the environment. The assumption of a collaborative environment enables us to use the agent's policy to estimate the distribution over action suggestions. We propose two methods that use suggested actions and demonstrate the approach through simulated experiments. The proposed methodology results in increased performance while also being robust to suboptimal suggestions.
On the Comparison between the Reliability of Units Produced by Different Production Lines

The paper discusses how to evaluate the reliability of units produced by different production lines. The procedure is based on selecting independent random samples of units produced by different production lines and then evaluating reliability functions for each group of units. The comparison between these reliability functions at a given time allows manufacturing experts to evaluate the effectiveness of production lines. A statistical methodology has been taken based on the assumption that the lifetime of units produced by each product line has a Weibull Gamma distribution. Then, real‐world data are used to illustrate the study's contribution to reliability theory applications.
Metrical almost periodicity: Levitan and Bebutov concepts

In this paper, we analyze Levitan and Bebutov metrical approximations of functions F : Λ × X → Y by trigonometric polynomials and ρ -periodic type functions, where ∅ 6 = Λ ⊆ R n , X and Y are complex Banach spaces, and ρ is a general binary relation on Y . We also analyze various classes of multi-dimensional Levitan almost periodic functions in general metric and multi-dimensional Bebutov uniformly recurrent functions in general metric. We provide several applications of our theoretical results to the abstract Volterra integro-diﬀerential equations and the partial diﬀerential equa- tions.
D-ITAGS: A Dynamic Interleaved Approach to Resilient Task Allocation, Scheduling, and Motion Planning

Complex, multi-task missions require the coordination of heterogeneous robots at multiple inter-connected levels, such as coalition formation, scheduling, and motion planning. This challenge is exacerbated by dynamic changes, such as sensor and actuator failures, communication loss, and unexpected delays. We introduce Dynamic Iterative Task Allocation Graph Search (D-ITAGS) to simultaneously address coalition formation, scheduling, and motion planning in dynamic settings involving heterogeneous teams. D-ITAGS achieves resilience via two key characteristics: i) interleaved execution, and ii) targeted repair. Interleaved execution enables an effective search for solutions at each layer while avoiding incompatibility with other layers. Targeted repair identifies and repairs parts of the existing solution impacted by a given disruption, while conserving the rest. In addition to algorithmic contributions, we derive accurate bounds on schedule suboptimality and provide insights into the inherent trade-off between time and resource optimality in these settings. Our experiments reveal that i) D-ITAGS is significantly faster than recomputation from scratch in dynamic settings, with little to no loss in solution quality, and ii) the theoretical bounds on optimality gap consistently hold in practice.
Public Wisdom Matters! Discourse-Aware Hyperbolic Fourier Co-Attention for Social-Text Classification

Social media has become the fulcrum of all forms of communication. Classifying social texts such as fake news, rumour, sarcasm, etc. has gained significant attention. The surface-level signals expressed by a social-text itself may not be adequate for such tasks; therefore, recent methods attempted to incorporate other intrinsic signals such as user behavior and the underlying graph structure. Oftentimes, the `public wisdom' expressed through the comments/replies to a social-text acts as a surrogate of crowd-sourced view and may provide us with complementary signals. State-of-the-art methods on social-text classification tend to ignore such a rich hierarchical signal. Here, we propose Hyphen, a discourse-aware hyperbolic spectral co-attention network. Hyphen is a fusion of hyperbolic graph representation learning with a novel Fourier co-attention mechanism in an attempt to generalise the social-text classification tasks by incorporating public discourse. We parse public discourse as an Abstract Meaning Representation (AMR) graph and use the powerful hyperbolic geometric representation to model graphs with hierarchical structure. Finally, we equip it with a novel Fourier co-attention mechanism to capture the correlation between the source post and public discourse. Extensive experiments on four different social-text classification tasks, namely detecting fake news, hate speech, rumour, and sarcasm, show that Hyphen generalises well, and achieves state-of-the-art results on ten benchmark datasets. We also employ a sentence-level fact-checked and annotated dataset to evaluate how Hyphen is capable of producing explanations as analogous evidence to the final prediction.
A Neural Network Approach for Stochastic Optimal Control

We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semi-linear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the non-viscous Hamilton-Jacobi-Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution and, via a modification, we show the wider applicability of our scheme.
Learning to Counter: Stochastic Feature-based Learning for Diverse Counterfactual Explanations

Interpretable machine learning seeks to understand the reasoning process of complex black-box systems that are long notorious for lack of explainability. One growing interpreting approach is through counterfactual explanations, which go beyond why a system arrives at a certain decision to further provide suggestions on what a user can do to alter the outcome. A counterfactual example must be able to counter the original prediction from the black-box classiﬁer, while also satisfying various constraints for practical applications. These constraints exist at trade-offs between one and another presenting radical challenges to existing works. To this end
Electron Reacceleration via Ion Cyclotron Waves in the Intracluster Medium

In galaxy clusters, the intracluster medium (ICM) is expected to host a diffuse, long-lived, and invisible population of “fossil” cosmic-ray electrons (CRe) with 1–100 MeV energies. These CRe, if reaccelerated by 100× in energy, can contribute synchrotron luminosity to cluster radio halos, relics, and phoenices. Reacceleration may be aided by CRe scattering upon the ion-Larmor-scale waves that spawn when ICM is compressed, dilated, or sheared. We study CRe scattering and energy gain due to ion cyclotron (IC) waves generated by continuously driven compression in 1D fully kinetic particle-in-cell simulations. We find that pitch-angle scattering of CRe by IC waves induces energy gain via magnetic pumping. In an optimal range of IC-resonant momenta, CRe may gain up to ∼10%–30% of their initial energy in one compression/dilation cycle with magnetic field amplification ∼3–6×, assuming adiabatic decompression without further scattering and averaging over initial pitch angle.
Beyond Heart Murmur Detection: Automatic Murmur Grading From Phonocardiogram

Objective: Murmurs are abnormal heart sounds, identified by experts through cardiac auscultation. The murmur grade, a quantitative measure of the murmur intensity, is strongly correlated with the patient's clinical condition. This work aims to estimate each patient's murmur grade (i.e., absent, soft, loud) from multiple auscultation location phonocardiograms (PCGs) of a large population of pediatric patients from a low-resource rural area. Methods: The Mel spectrogram representation of each PCG recording is given to an ensemble of 15 convolutional residual neural networks with channel-wise attention mechanisms to classify each PCG recording. The final murmur grade for each patient is derived based on the proposed decision rule and considering all estimated labels for available recordings. The proposed method is cross-validated on a dataset consisting of 3456 PCG recordings from 1007 patients using a stratified ten-fold cross-validation. Additionally, the method was tested on a hidden test set comprised of 1538 PCG recordings from 442 patients. Results: The overall cross-validation performances for patient-level murmur gradings are 86.3% and 81.6% in terms of the unweighted average of sensitivities and F1-scores, respectively. The sensitivities (and F1-scores) for absent, soft, and loud murmurs are 90.7% (93.6%), 75.8% (66.8%), and 92.3% (84.2%), respectively. On the test set, the algorithm achieves an unweighted average of sensitivities of 80.4% and an F1-score of 75.8%. Conclusions: This study provides a potential approach for algorithmic pre-screening in low-resource settings with relatively high expert screening costs. Significance: The proposed method represents a significant step beyond detection of murmurs, providing characterization of intensity, which may provide an enhanced classification of clinical outcomes.
Estimate of a nonflow baseline for the chiral magnetic effect in isobar collisions at RHIC

Recently, STAR reported the isobar (4496Ru + 4496Ru, 4096Zr + 4096Zr) results for the chiral magnetic effect (CME) search. The Ru+Ru to Zr+Zr ratio of the CME-sensitive observable Δγ, normalized by elliptic anisotropy (υ2), is observed to be close to the inverse multiplicity (N) ratio. In other words, the ratio of the NΔγ/υ2 observable is close to the naive background baseline of unity. However, nonflow correlations are expected to cause the baseline to deviate from unity. To further understand the isobar results, we decompose the nonflow contributions to NΔγ/υ2 (isobar ratio) into three terms and quantify each term by using the nonflow in υ2 measurement, published STAR data, and HIJING simulation. From these studies, we estimate a nonflow baseline of the isobar ratio of NΔγ/υ2 for the CME and discuss its implications.
Off-policy estimation of linear functionals: Non-asymptotic theory for semi-parametric efficiency

The problem of estimating a linear functional based on observational data is canonical in both the causal inference and bandit literatures. We analyze a broad class of two-stage procedures that first estimate the treatment effect function, and then use this quantity to estimate the linear functional. We prove non-asymptotic upper bounds on the mean-squared error of such procedures: these bounds reveal that in order to obtain non-asymptotically optimal procedures, the error in estimating the treatment effect should be minimized in a certain weighted L-norm. We analyze a two-stage procedure based on constrained regression in this weighted norm, and establish its instance-dependent optimality in finite samples via matching non-asymptotic local minimax lower bounds. These results show that the optimal non-asymptotic risk, in addition to depending on the asymptotically efficient variance, depends on the weighted norm distance between the true outcome function and its approximation by the richest function class supported by the sample size.
Multi-spacecraft observations of shocklets at an interplanetary shock.

Interplanetary (IP) shocks are fundamental building blocks of the heliosphere, and the possibility to observe them in-situ is crucial to address important aspects of energy conversion for a variety of astrophysical systems. Steepened waves known as shocklets are known to be important structures of planetary bow shocks, but they are very rarely observed related to IP shocks. We present here the first multi-spacecraft observations of shocklets observed by upstream of an unusually strong IP shock observed on November 3rd 2021 by several spacecraft at L1 and near-Earth solar wind. The same shock was detected also by radially aligned Solar Orbiter at 0.8 AU from the Sun, but no shocklets were identified from its data, introducing the possibility to study the environment in which shocklets developed. The Wind spacecraft has been used to characterise the shocklets, associated with pre-conditioning of the shock upstream by decelerating incoming plasma in the shock normal direction. Finally, using the Wind observations together with ACE and DSCOVR spacecraft at L1, as well as THEMIS B and THEMIS C in the near-Earth solar wind, the portion of interplanetary space filled with shocklets is addressed, and a lower limit for its extent is estimated to be of about 110 RE in the shock normal direction and 25 RE in the directions transverse to the shock normal. Using multiple spacecraft also reveals that for this strong IP shock, shocklets are observed for a large range of local obliquity estimates (9-64 degrees).
Analysis of nasopharyngeal microbiome patterns in Zambian infants with fatal acute febrile illness

Introduction Associative connections have previously been identified between nasopharyngeal infections and infant mortality. The nasopharyngeal microbiome may potentially influence the severity of these infections. Methods We conducted an analysis of a longitudinal prospective cohort study of 1,981 infants who underwent nasopharyngeal sampling from 1 week through 14 weeks of age at 2–3-week intervals. In all, 27 microbiome samples from 9 of the infants in the cohort who developed fatal acute febrile illness (fAFI) were analyzed in pooled comparisons with 69 samples from 10 healthy comparator infants. We completed 16S rRNA amplicon gene sequencing all infant NP samples and characterized the maturation of the infant NP microbiome among the fAFI(+) and fAFI(-) infant cohorts. Results Beta diversity measures of fAFI(-) infants were markedly higher than those of fAFI(+) infants. The fAFI(+) infant NP microbiome was marked by higher abundances of Escherichia, Pseudomonas, Leuconostoc, and Weissella, with low relative presence of Alkalibacterium, Dolosigranulum, Moraxella, and Streptococcus. Conclusions Our results suggest that nasopharyngeal microbiome dysbiosis precedes fAFI in young infants. Early dysbiosis, involving microbes such as Escherichia, may play a role in the causal pathway leading to fAFI or could be a marker of other pathogenic forces that directly lead to fAFI.