Miracle-dz/UniFAR-FTUs huggingface dataset:
Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection

Recent studies have focused on utilizing multi-modal data to develop robust models for facial Action Unit (AU) detection. However, the heterogeneity of multi-modal data poses challenges in learning effective representations. One such challenge is extracting relevant features from multiple modalities using a single feature extractor. Moreover, previous studies have not fully explored the potential of multi-modal fusion strategies. In contrast to the extensive work on late fusion, there are limited investigations on early fusion for channel information exploration. This paper presents a novel multi-modal reconstruction network, named Multimodal Channel-Mixing (MCM), as a pre-trained model to learn robust representation for facilitating multi-modal fusion. The approach follows an early fusion setup, integrating a Channel-Mixing module, where two out of five channels are randomly dropped. The dropped channels then are reconstructed from the remaining channels using masked autoencoder. This module not only reduces channel redundancy, but also facilitates multi-modal learning and reconstruction capabilities, resulting in robust feature learning. The encoder is fine-tuned on a downstream task of automatic facial action unit detection. Pre-training experiments were conducted on BP4D+, followed by fine-tuning on BP4D and DISFA to assess the effectiveness and robustness of the proposed framework. The results demonstrate that our method meets and surpasses the performance of state-of-the-art baseline methods.
A Deep Learning Approach to Analyzing Continuous-Time Systems

Scientists often use observational time series data to study complex natural processes, but regression analyses often assume simplistic dynamics. Recent advances in deep learning have yielded startling improvements to the performance of models of complex processes, but deep learning is generally not used for scientific analysis. Here we show that deep learning can be used to analyze complex processes, providing flexible function approximation while preserving interpretability. Our approach relaxes standard simplifying assumptions (e.g., linearity, stationarity, and homoscedasticity) that are implausible for many natural systems and may critically affect the interpretation of data. We evaluate our model on incremental human language processing, a domain with complex continuous dynamics. We demonstrate substantial improvements on behavioral and neuroimaging data, and we show that our model enables discovery of novel patterns in exploratory analyses, controls for diverse confounds in confirmatory analyses, and opens up research questions that are otherwise hard to study.
First-order Conditions for Optimization in the Wasserstein Space

We study first-order optimality conditions for constrained optimization in the Wasserstein space, whereby one seeks to minimize a real-valued function over the space of probability measures endowed with the Wasserstein distance. Our analysis combines recent insights on the geometry and the differential structure of the Wasserstein space with more classical calculus of variations. We show that simple rationales such as “setting the derivative to zero” and “gradients are aligned at optimality” carry over to the Wasserstein space. We deploy our tools to study and solve optimization problems in the setting of distributionally robust optimization and statistical inference. The generality of our methodology allows us to naturally deal with functionals, such as mean-variance, Kullback-Leibler divergence, and Wasserstein distance, which are traditionally difficult to study in a unified framework.
TransPOS: Transformers for Consolidating Different POS Tagset Datasets

In hope of expanding training data, researchers often want to merge two or more datasets that are created using different labeling schemes. This paper considers two datasets that label part-of-speech (POS) tags under different tagging schemes and leverage the supervised labels of one dataset to help generate labels for the other dataset. This paper further discusses the theoretical difficulties of this approach and proposes a novel supervised architecture employing Transformers to tackle the problem of consolidating two completely disjoint datasets. The results diverge from initial expectations and discourage exploration into the use of disjoint labels to consolidate datasets with different labels.
Effective low-energy models for superconducting impurity systems

We present two complementary methods to calculate the Andreev bound state energies of a single-level quantum dot connected to superconducting leads described by the superconducting impurity Anderson model. The first method, which is based on a mapping to a low-energy model, can be utilized to extract the Andreev bound state energies from finite-temperature, imaginary-time quantum Monte Carlo data without the necessity of any analytic continuation technique. The second method maps the full model on an exactly solvable superconducting atomic limit with renormalized parameters. As such, it represents a fast and reliable method for a quick scan of the parameter space. We demonstrate that after adding a simple band correction this method can provide predictions for measurable quantities, including the Josephson current, that are in a solid quantitative agreement with precise results obtained by the numerical renormalization group and quantum Monte Carlo.
Accurate 
ab initio
 modeling of solid solution strengthening in high entropy alloys

High entropy alloys (HEA) represent a class of materials with promising properties, such as high strength and ductility, radiation damage tolerance, etc. At the same time, a combinatorially large variety of compositions and a complex structure render them quite hard to study using conventional methods. In this work, we present a computationally efficient methodology based on ab initio calculations within the coherent potential approximation. To make the methodology predictive, we apply an exchange-correlation correction to the equation of state and take into account thermal effects on the magnetic state and the equilibrium volume. The approach shows good agreement with available experimental data on bulk properties of solid solutions. As a particular case, the workflow is applied to a series of iron-group HEA to investigate their solid solution strengthening within a parameter-free model based on the effective medium representation of an alloy. The results reveal intricate interactions between alloy components, which we analyze by means of a simple model of local bonding. Thanks to its computational efficiency, the methodology can be used as a basis for an adaptive learning workflow for optimal design of HEA.
Targeted Separation and Convergence with Kernel Discrepancies

Maximum mean discrepancies (MMDs) like the kernel Stein discrepancy (KSD) have grown central to a wide range of applications, including hypothesis testing, sampler selection, distribution approximation, and variational inference. In each setting, these kernel-based discrepancy measures are required to (i) separate a target P from other probability measures or even (ii) control weak convergence to P. In this article we derive new sufficient and necessary conditions to ensure (i) and (ii). For MMDs on separable metric spaces, we characterize those kernels that separate Bochner embeddable measures and introduce simple conditions for separating all measures with unbounded kernels and for controlling convergence with bounded kernels. We use these results on $\mathbb{R}^d$ to substantially broaden the known conditions for KSD separation and convergence control and to develop the first KSDs known to exactly metrize weak convergence to P. Along the way, we highlight the implications of our results for hypothesis testing, measuring and improving sample quality, and sampling with Stein variational gradient descent.
A FINITE ELEMENT METHOD FOR DIRICHLET BOUNDARY CONTROL PROBLEMS GOVERNED BY PARABOLIC PDES

Finite element approximations of Dirichlet boundary control problems governed by parabolic PDEs on convex polygonal domains are studied in this paper. The existence of a unique solution to optimal control problems is guaranteed based on very weak solution of the state equation andL 2 (0,T;L 2 ()) as control space. For the numerical discretization of the st ate equation we use standard piecewise linear and continuous finite elements for the space discretization of the state, while a dG(0) scheme is used for time discretization. The Dirichlet boundary control is realized through a space-time L 2 -projection. We consider both piecewise linear, continuous finite element approximation and variational discretization for the controls and derive a priori L 2 -error bounds for controls and states. We finally present numerical examples to support our theoretical findings.
A REVIEW OF DISPERSIVE LIMITS OF (NON)LINEAR SCHR¨ODINGER-TYPE EQUATIONS

In this review paper we present the most important mathematical properties of dispersive limits of (non)linear Schr¨odinger type equations. Different formulations are used to study these singular limits, e.g., the kinetic formulation of the linear Schr¨odinger equation based on the Wigner transform is well suited for global-in-time analysis without using WKB-(expansion) techniques, while the modified Madelung transformation reformulating Schr¨odinger equations in terms of a dispersive perturbation of a quasilinear symmetric hyperbolic system usually only gives local-in-time results due to the hyperbolic nature of the limit equations. Deterministic analogues of turbulence are also discussed. There, turbulent diffusion appears naturally in the zero dispersion limit.
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward

Advertisement video editing aims to automatically edit advertising videos into shorter videos while retaining coherent content and crucial information conveyed by advertisers. It mainly contains two stages: video segmentation and segment assemblage. The existing method performs well at video segmentation stages but suffers from the problems of dependencies on extra cumbersome models and poor performance at the segment assemblage stage. To address these problems, we propose M-SAN (Multi-modal Segment Assemblage Network) which can perform efficient and coherent segment assemblage task end-to-end. It utilizes multi-modal representation extracted from the segments and follows the Encoder-Decoder Ptr-Net framework with the Attention mechanism. Importance-coherence reward is designed for training M-SAN. We experiment on the Ads-1k dataset with 1000+ videos under rich ad scenarios collected from advertisers. To evaluate the methods, we propose a unified metric, Imp-Coh@Time, which comprehensively assesses the importance, coherence, and duration of the outputs at the same time. Experimental results show that our method achieves better performance than random selection and the previous method on the metric. Ablation experiments further verify that multi-modal representation and importance-coherence reward significantly improve the performance. Ads-1k dataset is available at: https://github.com/yunlong10/Ads-1k
An implimentation of the Differential Filter for Computing Gradient and Hessian of the Log-likelihood of Nonstationary Time Series Models

The state-space model and the Kalman ﬁlter provide us with uniﬁed and computationaly eﬃcient procedure for computing the log-likelihood of the diverse type of time series models. This paper presents an algorithm for computing the gradient and the Hessian matrix of the log-likelihood by extending the Kalman ﬁlter without resorting to the numerical diﬀerence. Diﬀerent from the previous paper [11], it is assumed that the observation noise variance R = 1. It is known that for univariate time series, by maximizing the log-likelihood of this restricted model, we can obtain the same estimates as the ones for the original state-space model. By this modiﬁcation, the algorithm for computing the gradient and the Hessian becomes somewhat complicated. However, the dimension of the parameter vector is reduce by one and thus has a signiﬁcant merit in estimating the parameter of the state-space model especially for relatively low dimentional parameter vector. Three examples of nonstationary time seirres models, i.e., trend model, statndard seasonal adjustment model and the seasonal adjustment model with AR componet are presented to exempliﬁed the speciﬁcation of structural matrices.
Application of the nnU-Net for automatic segmentation of lung lesion on CT images, and implication on radiomic models

Lesion segmentation is a crucial step of the radiomic workflow. Manual segmentation requires long execution time and is prone to variability, impairing the realisation of radiomic studies and their robustness. In this study, a deep-learning automatic segmentation method was applied on computed tomography images of non-small-cell lung cancer patients. The use of manual vs automatic segmentation in the performance of survival radiomic models was assessed, as well. A total of 899 NSCLC patients were included (2 proprietary: A and B, 1 public datasets: C). Automatic segmentation of lung lesions was performed by training a previously developed architecture, the nnU-Net, including 2D, 3D and cascade approaches. The quality of automatic segmentation was evaluated with DICE coefficient, considering manual contours as reference. The impact of automatic segmentation on the performance of a radiomic model for patient survival was explored by extracting radiomic hand-crafted and deep-learning features from manual and automatic contours of dataset A, and feeding different machine learning algorithms to classify survival above/below median. Models’ accuracies were assessed and compared. The best agreement between and (DICE=0.78±0.12) was achieved averaging predictions from 2D and 3D models, and applying a post-processing technique extracting the maximum connected component. No statistical observed hand-crafted deep The best classifier showed an accuracy between 0.65 and 0.78. the time-consuming workload the accuracy survival predictive models radiomics.
Green function for an asymptotically stable random walk in a half space

We consider an asymptotically stable multidimensional random walk S ( n ) = ( S 1 ( n ) , . ,S d ( n )). Let τ x := min { n > 0 : x 1 + S 1 ( n ) ≤ 0 } be the ﬁrst time the random walk S ( n ) leaves the upper half-space. We obtain the asymptotics of p n ( x,y ) := P ( x + S ( n ) ∈ y + ∆ ,τ x > n ) as n tends to inﬁnity, where ∆ is a ﬁxed cube. From that we obtain the local asymptotics for the Green function G ( x,y ) := P n p n ( x,y ), as | y | and/or | x | tend to inﬁnity.
Mapping and management of communication services on MP-SoC platforms

Jim: Shred it. Bernard: Shred it? Jim: No one must ever be able to find it again. Bernard: In that case, Minister, I think it’s best I file it. Yes Minister The Death List The ability to program Multi-Processor System-on-Chip (MP-SoC) platforms and verify applications are key issues in tackling the MP-SoC challenge. Using the requirements of anMP-SoC application design-flow as input we derive adequate communication services andMP-SoC communication architecture (our goals). We thus follow a system-level approach and experiment on complete MP-SoC platform emulators as well as on transaction-level simulators. The application design-flow assumes a distributed shared-memory platform and performs software-controlled scratch-padmanagement to hide thememory and communication latency. To guarantee composability of independently optimized applications it requires virtualization of communication resources (hard bounds on bandwidth and latency). It furthermore requires hardware support to perform efficient data transfers in the memory hierarchy. This PhD thesis discusses design-time and run-time virtualization of Network-onChip (NoC) communication architectures. We introduce the novel hard-guaranteed SuperGT Quality-of-Service (QoS) and the associated SuperGT NoC. We show, on a realistic example, improvements up to 8x in latency of SuperGT connections when compared to the GT QoS. The SuperGT NoC supports all three QoS classes besteffort, guaranteed-throughput (GT) and SuperGT. We provide two main contributions to the run-time management of NoCs. On the one hand we detail a central dynamic time-slot allocation algorithm for networks-on-chip and its efficient implementation on an embedded processor. On the other hand we discuss both central and distributed approaches to reactive communication control of networks-on-chip. Finally, we introduce a communication-assist to provide support for scratch-pad management in MP-SoC memory hierarchies. We quantitatively and extensively benchmark themapping of a video encoder application using our combinedMP-SoC design-flow and adequate architecture to a state-of-the-art cache-based approach. The experiments shows that, conversely to the cache-based solution, our combined design-flow/MP-SoC architecture for scratch-pad management can efficiently exploit the virtualization capabilities of a QoS-enabled network-on-chip.
Hidden and antiferromagnetic order as a rank-5 superspin in URu 2 Si 2

We propose a candidate for the hidden order in URu2Si2: a rank-5 E type spin density wave between Uranium 5f crystal field doublets breaking time reversal and lattice tetragonal symmetry in a manner consistent with recent torque measurements [R. Okazaki et al, Science 331, 439 (2011)]. We argue that coupling of this order parameter to magnetic probes can be hidden by crystal field effects, while still having significant effects on transport, thermodynamics and magnetic susceptibilities. In a simple tight-binding model for the heavy quasiparticles, we show the connection between the hidden order and antiferromagnetic phases arises since they form different components of this single rank-5 pseudo-spin vector. Using a phenomenological theory, we show the experimental pressure-temperature phase diagram can be qualitatively reproduced by tuning terms which break pseudo-spin rotational symmetry. As a test of our proposal, we predict the presence of small magnetic moments in the basal plane oriented in the [110] direction ordered at the wave-vector (0,0,1).
Rethinking Motion Deblurring Training: A Segmentation-Based Method for Simulating Non-Uniform Motion Blurred Images

—Successful training of end-to-end deep networks for real motion deblurring requires datasets of sharp/blurred image pairs that are realistic and diverse enough to achieve generaliza- tion to real blurred images. Obtaining such datasets remains a challenging task. In this paper, we ﬁrst review the limitations of existing deblurring benchmark datasets from the perspective of generalization to blurry images in the wild. Secondly, we propose an efﬁcient procedural methodology to generate sharp/blurred image pairs, based on a simple yet effective model for the formation of blurred images. This allows generating virtually unlimited realistic and diverse training pairs. We demonstrate the effectiveness of the proposed dataset by training existing deblurring architectures on the simulated pairs and evaluating them across four standard datasets of real blurred images. We observed superior generalization performance for the ultimate task of deblurring real motion-blurred photos of dynamic scenes when training with the proposed method.
Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations

Learning from Demonstration (LfD) approaches empower end-users to teach robots novel tasks via demonstrations of the desired behaviors, democratizing access to robotics. However, current LfD frameworks are not capable of fast adaptation to heterogeneous human demonstrations nor the large-scale deployment in ubiquitous robotics applications. In this paper, we propose a novel LfD framework, Fast Lifelong Adaptive Inverse Reinforcement learning (FLAIR). Our approach (1) leverages learned strategies to construct policy mixtures for fast adaptation to new demonstrations, allowing for quick end-user personalization, (2) distills common knowledge across demonstrations, achieving accurate task inference; and (3) expands its model only when needed in lifelong deployments, maintaining a concise set of prototypical strategies that can approximate all behaviors via policy mixtures. We empirically validate that FLAIR achieves adaptability (i.e., the robot adapts to heterogeneous, user-specific task preferences), efficiency (i.e., the robot achieves sample-efficient adaptation), and scalability (i.e., the model grows sublinearly with the number of demonstrations while maintaining high performance). FLAIR surpasses benchmarks across three control tasks with an average 57% improvement in policy returns and an average 78% fewer episodes required for demonstration modeling using policy mixtures. Finally, we demonstrate the success of FLAIR in a table tennis task and find users rate FLAIR as having higher task (p<.05) and personalization (p<.05) performance.
Online Submodular Coordination With Bounded Tracking Regret: Theory, Algorithm, and Applications to Multi-Robot Coordination

We enable efficient and effective coordination in unpredictable environments, i.e., in environments whose future evolution is unknown a priori and even adversarial. We are motivated by the future of autonomy that involves multiple robots coordinating in dynamic, unstructured, and adversarial environments to complete complex tasks such as target tracking, environmental mapping, and area monitoring. Such tasks are often modeled as submodular maximization coordination problems. We introduce the first submodular coordination algorithm with bounded tracking regret, i.e., with bounded suboptimality with respect to optimal time-varying actions that know the future a priori. The bound gracefully degrades with the environments' capacity to change adversarially. It also quantifies how often the robots must re-select actions to “learn” to coordinate as if they knew the future a priori. The algorithm requires the robots to select actions sequentially based on the actions selected by the previous robots in the sequence. Particularly, the algorithm generalizes the seminal Sequential Greedy algorithm by Fisher et al. to unpredictable environments, leveraging submodularity and algorithms for the problem of tracking the best expert. We validate our algorithm in simulated scenarios of target tracking.
It Takes Two: Learning to Plan for Human-Robot Cooperative Carrying

Cooperative table-carrying is a complex task due to the continuous nature of the action and state-spaces, multimodality of strategies, and the need for instantaneous adaptation to other agents. In this work, we present a method for predicting realistic motion plans for cooperative human-robot teams on the task. Using a Variational Recurrent Neural Network (VRNN) to model the variation in the trajectory of a human-robot team across time, we are able to capture the distribution over the team's future states while leveraging information from interaction history. The key to our approach is leveraging human demonstration data to generate trajectories that synergize well with humans during test time in a receding horizon fashion. Comparison between a baseline, sampling-based planner RRT (Rapidly-exploring Random Trees) and the VRNN planner in centralized planning shows that the VRNN generates motion more similar to the distribution of human-human demonstrations than the RRT. Results in a human-in-the-loop user study show that the VRNN planner outperforms decentralized RRT on task-related metrics, and is significantly more likely to be perceived as human than the RRT planner. Finally, we demonstrate the VRNN planner on a real robot paired with a human teleoperating another robot.