Miracle-dz/UniFAR-FTUs huggingface dataset:
Smoothing the Disentangled Latent Style Space for Unsupervised Image-to-Image Translation

Image-to-Image (I2I) multi-domain translation models are usually evaluated also using the quality of their semantic interpolation results. However, state-of-the-art models frequently show abrupt changes in the image appearance during interpolation, and usually perform poorly in interpolations across domains. In this paper, we propose a new training protocol based on three specific losses which help a translation network to learn a smooth and disentangled latent style space in which: 1) Both intra- and inter-domain interpolations correspond to gradual changes in the generated images and 2) The content of the source image is better preserved during the translation. Moreover, we propose a novel evaluation metric to properly measure the smoothness of latent style space of I2I translation models. The proposed method can be plugged in existing translation approaches, and our extensive experiments on different datasets show that it can significantly boost the quality of the generated images and the graduality of the interpolations.
Scene Transformer: A unified multi-task model for behavior prediction and planning

Predicting the future motion of multiple agents is necessary for planning in dynamic environments. This task is challenging for autonomous driving since agents (e.g., vehicles and pedestrians) and their associated behaviors may be diverse and inﬂuence each other. Most prior work has focused on ﬁrst predicting independent futures for each agent based on all past motion, and then planning against these independent predictions. However, planning against ﬁxed predictions can suffer from the inability to represent the future interaction possibilities between different agents, leading to sub-optimal planning. In this work, we formulate a model for predicting the behavior of all agents jointly in real-world driving environments in a uniﬁed manner. Inspired by recent language modeling approaches, we use a masking strategy as the query to our model, enabling one to invoke a single model to predict agent behavior in many ways, such as potentially conditioned on the goal or full future trajectory of the autonomous vehicle or the behavior of other agents in the environment. Our model architecture fuses heterogeneous world state in a uniﬁed Transformer architecture by employing attention across road elements, agent interactions and time steps. We evaluate our approach on autonomous driving datasets for behavior prediction, and achieve state-of-the-art performance. Our work demonstrates that formulating the problem of behavior prediction in a uniﬁed architecture with a masking strategy may allow us to have a single model that can perform multiple motion prediction and planning related tasks effectively.
Benefits, Challenges and Contributors to Success for National eHealth Systems Implementation: A Scoping Review

OBJECTIVE Our scoping review aims to assess what legal, ethical, and socio-technical factors contribute to or inhibit the success of national eHealth system implementations. In addition, our review seeks to describe the characteristics and benefits of eHealth systems. MATERIALS AND METHODS We conducted a scoping review of literature published in English between January 2000 and 2020 using a keyword search on 5 databases: PubMed, Scopus, Web of Science, IEEEXplore, and ProQuest. After removal of duplicates, abstract screening, and full-text filtering, 86 articles were included from 8276 search results. RESULTS We identified 17 stakeholder groups, 6 eHealth Systems areas, and 15 types of legal regimes and standards. In-depth textual analysis revealed challenges mainly in implementation, followed by ethico-legal and data-related aspects. Key factors influencing success include promoting trust of the system, ensuring wider acceptance among users, reconciling the system with legal requirements, and ensuring an adaptable technical platform. DISCUSSION Results revealed support for decentralized implementations because they carry less implementation and engagement challenges than centralized ones. Simultaneously, due to decentralized systems' interoperability issues, federated implementations (with a set of national standards) might be preferable. CONCLUSION This study identifies the primary socio-technical, legal, and ethical factors that challenge and contribute to the success of eHealth system implementations. This study also describes the complexities and characteristics of existing eHealth implementation programs, and suggests guidance for resolving the identified challenges.
Revisit Visual Representation in Analytics Taxonomy: A Compression Perspective

Visual analytics have played an increasingly critical role in the Internet of Things, where massive visual signals have to be compressed and fed into machines. But facing such big data and constrained bandwidth capacity, existing image/video compression methods lead to very low-quality representations, while existing feature compression techniques fail to support diversified visual analytics applications/tasks with low-bit-rate representations. In this paper, we raise and study the novel problem of supporting multiple machine vision analytics tasks with the compressed visual representation, namely, the information compression problem in analytics taxonomy. By utilizing the intrinsic transferability among different tasks, our framework successfully constructs compact and expressive representations at low bit-rates to support a diversified set of machine vision tasks, including both high-level semantic-related tasks and mid-level geometry analytic tasks. In order to impose compactness in the representations, we propose a codebook-based hyperprior, which helps map the representation into a low-dimensional manifold. As it well fits the signal structure of the deep visual feature, it facilitates more accurate entropy estimation, and results in higher compression efficiency. With the proposed framework and the codebook-based hyperprior, we further investigate the relationship of different task features owning different levels of abstraction granularity. Experimental results demonstrate that with the proposed scheme, a set of diversified tasks can be supported at a significantly lower bit-rate, compared with existing compression schemes.
Recent Advances in Epidemic Modeling: Non-Markov Stochastic Models and their Scaling Limits

In this survey paper, we review the recent advances in individual based non--Markovian epidemic models. They include epidemic models with a constant infectivity rate, varying infectivity rate or infection-age dependent infectivity, infection-age recovery rate (or equivalently, general law of infectious period), as well as varying susceptibility/immunity. We focus on the scaling limits with a large population, functional law of large numbers (FLLN) and functional central limit theorems (FCLT), while the large and moderate deviations for some Markovian epidemic models are also reviewed. In the FLLN, the limits are a set of Volterra integral equations, and in the FCLT, the limits are stochastic Volterra integral equations driven by Gaussian processes. We relate our deterministic limits to the results in the seminal papers by Kermack and McKendrick published in 1927, 1932 and 1933, where the varying infectivity and susceptibility/immunity were already considered. We also discuss some extensions, including models with heterogeneous population, spatial models and control problems, as well as open problems.
Unsupervised Domain Adaptation With Variational Approximation for Cardiac Segmentation

Unsupervised domain adaptation is useful in medical image segmentation. Particularly, when ground truths of the target images are not available, domain adaptation can train a target-specific model by utilizing the existing labeled images from other modalities. Most of the reported works mapped images of both the source and target domains into a common latent feature space, and then reduced their discrepancy either implicitly with adversarial training or explicitly by directly minimizing a discrepancy metric. In this work, we propose a new framework, where the latent features of both domains are driven towards a common and parameterized variational form, whose conditional distribution given the image is Gaussian. This is achieved by two networks based on variational auto-encoders (VAEs) and a regularization for this variational approximation. Both of the VAEs, each for one domain, contain a segmentation module, where the source segmentation is trained in a supervised manner, while the target one is trained unsupervisedly. We validated the proposed domain adaptation method using two cardiac segmentation tasks, i.e., the cross-modality (CT and MR) whole heart segmentation and the cross-sequence cardiac MR segmentation. Results show that the proposed method achieved better accuracies compared to two state-of-the-art approaches and demonstrated good potential for cardiac segmentation. Furthermore, the proposed explicit regularization was shown to be effective and efficient in narrowing down the distribution gap between domains, which is useful for unsupervised domain adaptation. The code and data have been released via https://zmiclab.github. io/projects.html.
Input Invex Neural Network

Connected decision boundaries are useful in several tasks like image segmentation, clustering, alpha-shape or defining a region in nD-space. However, the machine learning literature lacks methods for generating connected decision boundaries using neural networks. Thresholding an invex function, a generalization of a convex function, generates such decision boundaries. This paper presents two methods for constructing invex functions using neural networks. The first approach is based on constraining a neural network with Gradient Clipped-Gradient Penality (GCGP), where we clip and penalise the gradients. In contrast, the second one is based on the relationship of the invex function to the composition of invertible and convex functions. We employ connectedness as a basic interpretation method and create connected region-based classifiers. We show that multiple connected set based classifiers can approximate any classification function. In the experiments section, we use our methods for classification tasks using an ensemble of 1-vs-all models as well as using a single multiclass model on larger-scale datasets. The experiments show that connected set-based classifiers do not pose any disadvantage over ordinary neural network classifiers, but rather, enhance their interpretability. We also did an extensive study on the properties of invex function and connected sets for interpretability and network morphism with experiments on simulated and real-world data sets. Our study suggests that invex function is fundamental to understanding and applying locality and connectedness of input space which is useful for various downstream tasks.
Evolving Image Compositions for Feature Representation Learning

Convolutional neural networks for visual recognition require large amounts of training samples and usually benefit from data augmentation. This paper proposes PatchMix, a data augmentation method that creates new samples by composing patches from pairs of images in a grid-like pattern. These new samples are assigned label scores that are proportional to the number of patches borrowed from each image. We then add a set of additional losses at the patch-level to regularize and to encourage good representations at both the patch and image levels. A ResNet-50 model trained on ImageNet using PatchMix exhibits superior transfer learning capabilities across a wide array of benchmarks. Although PatchMix can rely on random pairings and random grid-like patterns for mixing, we explore evolutionary search as a guiding strategy to jointly discover optimal grid-like patterns and image pairings. For this purpose, we conceive a fitness function that bypasses the need to re-train a model to evaluate each possible choice. In this way, PatchMix outperforms a base model on CIFAR-10 (+1.91), CIFAR-100 (+5.31), Tiny Imagenet (+3.52), and ImageNet (+1.16).
A Flow-Based Neural Network for Time Domain Speech Enhancement

Speech enhancement involves the distinction of a target speech signal from an intrusive background. Although generative approaches using Variational Autoencoders or Generative Adversarial Networks (GANs) have increasingly been used in recent years, normalizing flow (NF) based systems are still scarse, despite their success in related fields. Thus, in this paper we propose a NF framework to directly model the enhancement process by density estimation of clean speech utterances conditioned on their noisy counterpart. The WaveGlow model from speech synthesis is adapted to enable direct enhancement of noisy utterances in time domain. In addition, we demonstrate that nonlinear input companding benefits the model performance by equalizing the distribution of input samples. Experimental evaluation on a publicly available dataset shows comparable results to current state-of-the-art GAN-based approaches, while surpassing the chosen baselines using objective evaluation metrics.
Beyond Tikhonov: Faster Learning with Self-Concordant Losses via Iterative Regularization

The theory of spectral filtering is a remarkable tool to understand the statistical properties of learning with kernels. For least squares, it allows to derive various regularization schemes that yield faster convergence rates of the excess risk than with Tikhonov regularization. This is typically achieved by leveraging classical assumptions called source and capacity conditions, which characterize the difficulty of the learning task. In order to understand estimators derived from other loss functions, Marteau-Ferey et al. have extended the theory of Tikhonov regularization to generalized self concordant loss functions (GSC), which contain, e.g., the logistic loss. In this paper, we go a step further and show that fast and optimal rates can be achieved for GSC by using the iterated Tikhonov regularization scheme, which is intrinsically related to the proximal point method in optimization, and overcomes the limitation of the classical Tikhonov regularization.
The role of SPICA-like missions and the Origins Space Telescope in the quest for heavily obscured AGN and synergies with Athena

Abstract In the black hole (BH)–galaxy co-evolution framework, most of the star formation (SF) and the BH accretion are expected to take place in highly obscured conditions. The large amount of gas and dust absorbs most of the UV-to-soft-X radiation and re-emits it at longer wavelengths, mostly in the IR. Thus, obscured active galactic nuclei (AGN) are very difficult to identify in optical or X-ray bands but shine bright in the IR. Moreover, X-ray background (XRB) synthesis models predict that a large fraction of the yet-unresolved XRB is due to the most obscured (Compton thick, CT: N $_{\text{H}}\ge 10^{24} \,\mathrm{cm}^{-2}$ ) of these AGN. In this work, we investigate the synergies between putative IR missions [using SPace Infrared telescope for Cosmology and Astrophysics (SPICA), proposed for European Space Agency (ESA)/M5 but withdrawn in 2020 October, and Origins Space Telescope, OST, as ‘templates’] and the X-ray mission Athena (Advanced Telescope for High ENergy Astrophysics), which should fly in early 2030s, in detecting and characterising AGN, with a particular focus on the most obscured ones. Using an XRB synthesis model, we estimated the number of AGN and the number of those which will be detected in the X-rays by Athena. For each AGN, we associated an optical-to-Far InfraRed (FIR) spectral energy distribution (SED) from observed AGN with both X-ray data and SED decomposition and used these SEDs to check if the AGN will be detected by SPICA-like or OST at IR wavelengths. We expect that, with the deepest Athena and SPICA-like (or OST) surveys, we will be able to photometrically detect in the IR more than 90% of all the AGN (down to $L_{2-10\text{keV}} \sim 10^{42}\,\mathrm{erg\ s}^{-1}$ and up to $z \sim 10$ ) predicted by XRB synthesis modeling, and we will detect at least half of them in the X-rays. The spectroscopic capabilities of the OST can provide ${\approx}51\,000$ and ${\approx}3\,400$ AGN spectra with $R= 300$ at 25–588 $\unicode[Times]{x03BC}$ m in the wide and deep surveys, respectively, the last one up to $z\approx 4$ . Athena will be extremely powerful in detecting and discerning moderate- and high-luminosity AGN, allowing us to properly select AGN even when the mid-IR torus emission is ‘hidden’ by the host galaxy contribution. We will constrain the intrinsic luminosity and the amount of obscuration for $\sim\! 20\%$ of all the AGN (and $\sim\! 50\%$ of those with $L_{2-10\text{keV}} > 3.2 \times 10^{43}\,\mathrm{erg\ s}^{-1}$ ) using the X-ray spectra provided by Athena WFI. We find that the most obscured and elusive CT-AGN will be exquisitely sampled by SPICA-like mission or OST and that Athena will allow a fine characterisation of the most luminous ones. This will provide a significant step forward in the process of placing stronger constraints on the yet-unresolved XRB and investigating the BH accretion rate evolution up to very high redshift ( $z \ge 4$ ).
Toward Affective XAI: Facial Affect Analysis for Understanding Explainable Human-AI Interactions

As machine learning approaches are increasingly used to augment human decision-making, eXplainable Artificial Intelligence (XAI) research has explored methods for communicating system behavior to humans. However, these approaches often fail to account for the affective responses of humans as they interact with explanations. Facial affect analysis, which examines human facial expressions of emotions, is one promising lens for understanding how users engage with explanations. Therefore, in this work, we aim to (1) identify which facial affect features are pronounced when people interact with XAI interfaces, and (2) develop a multitask feature embedding for linking facial affect signals with participants’ use of explanations. Our analyses and results show that the occurrence and values of facial AU1 and AU4, and Arousal are heightened when participants fail to use explanations effectively. This suggests that facial affect analysis should be incorporated into XAI to personalize explanations to individuals’ interaction styles and to adapt explanations based on the difficulty of the task performed.
Simultaneous Training of Partially Masked Neural Networks

For deploying deep learning models to lower-end devices, it is necessary to train less resource-demanding variants of state-of-the-art architectures. This does not eliminate the need for more expensive models as they have a higher performance. In order to avoid training two separate models, we show that it is possible to train neural networks in such a way that a predeﬁned ‘core’ subnetwork can be split-oﬀ from the trained full network with remarkable good performance. We extend on prior methods that focused only on core networks of smaller width, while we focus on supporting arbitrary core network architectures. Our proposed training scheme switches consecutively between optimizing only the core part of the network and the full one. The accuracy of the full model remains comparable, while the core network achieves better performance than when it is trained in isolation. In particular, we show that training a Transformer with a low-rank core gives a low-rank model with superior performance than when training the low-rank model alone. Weanalyze our training scheme theoretically, and show its convergence under assumptions that are either standard or practically justiﬁed. Moreover, we show that the developed theoretical framework allows analyzing many other partial training schemes for neural networks.
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control

Reinforcement learning is a framework for interactive decision-making with incentives sequentially revealed across time without a system dynamics model. Due to its scaling to continuous spaces, we focus on policy search where one iteratively improves a parameterized policy with stochastic policy gradient (PG) updates. In tabular Markov Decision Problems (MDPs), under persistent exploration and suitable parameterization, global optimality may be obtained. By contrast, in continuous space, the non-convexity poses a pathological challenge as evidenced by existing convergence results being mostly limited to stationarity or arbitrary local extrema. To close this gap, we step towards persistent exploration in continuous space through policy parameterizations defined by distributions of heavier tails defined by tail-index parameter α, which increases the likelihood of jumping in state space. Doing so invalidates smoothness conditions of the score function common to PG. Thus, we establish how the convergence rate to stationarity depends on the policy’s tail index α, a Hölder continuity parameter, integrability conditions, and an exploration tolerance parameter introduced here for the first time. Further, we characterize the dependence of the set of local maxima on the tail index through an exit and transition time analysis of a suitably defined Markov chain, identifying that policies associated with Lévy Processes of a heavier tail converge to wider peaks. This phenomenon yields improved stability to perturbations in supervised learning, which we corroborate also manifests in improved performance of policy search, especially when myopic and farsighted incentives are misaligned. *Equal contribution. Preprint. Under review. ar X iv :2 10 6. 08 41 4v 1 [ cs . L G ] 1 5 Ju n 20 21
Watching Too Much Television is Good: Self-Supervised Audio-Visual Representation Learning from Movies and TV Shows

The abundance and ease of utilizing sound, along with the fact that auditory clues reveal so much about what happens in the scene, make the audio-visual space a perfectly intuitive choice for self-supervised representation learning. However, the current literature suggests that training on \textit{uncurated} data yields considerably poorer representations compared to the \textit{curated} alternatives collected in supervised manner, and the gap only narrows when the volume of data significantly increases. Furthermore, the quality of learned representations is known to be heavily influenced by the size and taxonomy of the curated datasets used for self-supervised training. This begs the question of whether we are celebrating too early on catching up with supervised learning when our self-supervised efforts still rely almost exclusively on curated data. In this paper, we study the efficacy of learning from Movies and TV Shows as forms of uncurated data for audio-visual self-supervised learning. We demonstrate that a simple model based on contrastive learning, trained on a collection of movies and TV shows, not only dramatically outperforms more complex methods which are trained on orders of magnitude larger uncurated datasets, but also performs very competitively with the state-of-the-art that learns from large-scale curated data. We identify that audiovisual patterns like the appearance of the main character or prominent scenes and mise-en-sc\`ene which frequently occur through the whole duration of a movie, lead to an overabundance of easy negative instances in the contrastive learning formulation. Capitalizing on such observation, we propose a hierarchical sampling policy, which despite its simplicity, effectively improves the performance, particularly when learning from TV shows which naturally face less semantic diversity.