Miracle-dz/UniFAR-FTUs huggingface dataset:
Talking Head from Speech Audio using a Pre-trained Image Generator

We propose a novel method for generating high-resolution videos of talking-heads from speech audio and a single 'identity' image. Our method is based on a convolutional neural network model that incorporates a pre-trained StyleGAN generator. We model each frame as a point in the latent space of StyleGAN so that a video corresponds to a trajectory through the latent space. Training the network is in two stages. The first stage is to model trajectories in the latent space conditioned on speech utterances. To do this, we use an existing encoder to invert the generator, mapping from each video frame into the latent space. We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator. These displacements are relative to the back-projection into the latent space of an identity image chosen from the individuals depicted in the training dataset. In the second stage, we improve the visual quality of the generated videos by tuning the image generator on a single image or a short video of any chosen identity. We evaluate our model on standard measures (PSNR, SSIM, FID and LMD) and show that it significantly outperforms recent state-of-the-art methods on one of two commonly used datasets and gives comparable performance on the other. Finally, we report on ablation experiments that validate the components of the model. The code and videos from experiments can be found at https://mohammedalghamdi.github. io/talking-heads-acm-mm/
Shape-based Evaluation of Epidemic Forecasts

Infectious disease forecasting for ongoing epidemics has been traditionally performed, communicated, and evaluated as numerical targets – 1, 2, 3, and 4 week ahead cases, deaths, and hospitalizations. While there is great value in predicting these numerical targets to assess the burden of the disease, we argue that there is also value in communicating the future trend (description of the shape) of the epidemic – for instance, if the cases will remain flat o r a s urge i s expected. To ensure what is being communicated is useful we need to be able to evaluate how well the predicted shape matches with the ground truth shape. Instead of treating this as a classification problem ( one out of n shapes), we define a transformation of the numerical forecasts into a "shapelet"-space representation. In this representation, each dimension corresponds to the similarity of the shape with one of the shapes of interest (a shapelet). We prove that this representation satisfies the property that two shapes that one would consider similar are mapped close to each other, and vice versa. We demonstrate that our representation is able to reasonably capture the trends in COVID-19 cases and deaths time-series. With this representation, we define an evaluation measure and a measure of agreement among multiple models. We also define the shapelet-space ensemble of multiple models as the mean of their shapelet-space representations. We show that this ensemble is able to accurately predict the shape of the future trend for COVID-19 cases and trends. We also show that the agreement between models can provide a good indicator of the reliability of the forecast.
Energy-Aware JPEG Image Compression: A Multi-Objective Approach

Customer satisfaction is crucially affected by energy consumption in mobile devices. One of the most energy-consuming parts of an application is images. While different images with different quality consume different amounts of energy, there are no straightforward methods to calculate the energy consumption of an operation in a typical image. This paper, first, investigates that there is a correlation between energy consumption and image quality as well as image file size. Therefore, these two can be considered as a proxy for energy consumption. Then, we propose a multi-objective strategy to enhance image quality and reduce image file size based on the quantisation tables in JPEG image compression. To this end, we have used two general multi-objective metaheuristic approaches: scalarisation and Pareto-based. Scalarisation methods find a single optimal solution based on combining different objectives, while Pareto-based techniques aim to achieve a set of solutions. In this paper, we embed our strategy into five scalarisation algorithms, including energy-aware multi-objective genetic algorithm (EnMOGA), energy-aware multi-objective particle swarm optimisation (EnMOPSO), energy-aware multi-objective differential evolution (EnMODE), energy-aware multi-objective evolutionary strategy (EnMOES), and energy-aware multi-objective pattern search (EnMOPS). Also, two Pareto-based methods, including a non-dominated sorting genetic algorithm (NSGA-II) and a reference-point-based NSGA-II (NSGA-III) are used for the embedding scheme, and two Pareto-based algorithms, EnNSGAII and EnNSGAIII, are presented. Experimental studies show that the performance of the baseline algorithm is improved by embedding the proposed strategy into metaheuristic algorithms.
MICO: Selective Search with Mutual Information Co-training

In contrast to traditional exhaustive search, selective search first clusters documents into several groups before all the documents are searched exhaustively by a query, to limit the search executed within one group or only a few groups. Selective search is designed to reduce the latency and computation in modern large-scale search systems. In this study, we propose MICO, a Mutual Information CO-training framework for selective search with minimal supervision using the search logs. After training, MICO does not only cluster the documents, but also routes unseen queries to the relevant clusters for efficient retrieval. In our empirical experiments, MICO significantly improves the performance on multiple metrics of selective search and outperforms a number of existing competitive baselines.
Maximizing Utilization under Time-Varying Resource Requirements

Low utilization has been one of the key limiting factors for the continued scaling of today’s computing systems. One main reason for the low utilization is the temporal variation in the actual resource requirements of jobs, because reserving resources for jobs based on the peak requirements then results in idle resources for non-peak time. To increase utilization, in practice, resources are often overcommitted, so jobs may need to contend for resources at times and experience performance degradation, incurring a cost. To make use of the flexibility in resource allocation provided by overcommitment to the best advantage, it is critical to answer the following fundamental question that has not been well studied: given an acceptable budget for the cost associated with resource contention, how can we achieve the maximum utilization of the system? In this paper, we propose a job model that captures the time-varying resource requirement of each job by a Markov chain. We consider a stochastic formulation of the job dispatch problem in an infinite-server system. Our goal is to maximize system utilization by minimizing the number of active servers (servers running at least one job), subject to a budget on the cost of resource contention. Our main result is the design of a job dispatch policy that is asymptotically optimal as the job arrival rate increases. The novel technical framework that we develop reduces the policy design problem under study to that in a single-server system through policy conversions, which may be of independent interest. The framework allows us to design the asymptotically optimal policy by solving a linear program.
What drives decayless kink oscillations in active-region coronal loops on the Sun?

We study here the phenomena of decayless kink oscillations in a system of active region (AR) coronal loops. Using high resolution observations from two different instruments, namely the Extreme Ultraviolet Imager (EUI) on board Solar Orbiter and the Atmospheric Imaging Assembly (AIA) on board the Solar Dynamics Observatory, we follow these AR loops for an hour each on three consecutive days. Our results show significantly more resolved decayless waves in the higher-resolution EUI data compared with the AIA data. Furthermore, the same system of loops exhibits many of these decayless oscillations on Day-2, while on Day-3, we detect very few oscillations and on Day-1, we find none at all. Analysis of photospheric magnetic field data reveals that at most times, these loops were rooted in sunspots, where supergranular flows are generally absent. This suggests that supergranular flows, which are often invoked as drivers of decayless waves, are not necessarily driving such oscillations in our observations. Similarly, our findings also cast doubt on other possible drivers of these waves, such as a transient driver or mode conversion of longitudinal waves near the loop footpoints. In conclusion, through our analysis we find that none of the commonly suspected sources proposed to drive decayless oscillations in active region loops seems to be operating in this event and hence, the search for that elusive wave driver needs to continue.
Shapley value-based approaches to explain the robustness of classifiers in machine learning

The use of algorithm-agnostic approaches is an emerging area of research for explaining the contribution of individual features towards the predicted outcome. Whilst there is a focus on explaining the prediction itself, a little has been done on explaining the robustness of these models, that is, how each feature contributes towards achieving that robustness. In this paper, we propose the use of Shapley values to explain the contribution of each feature towards the model's robustness, measured in terms of Receiver-operating Characteristics (ROC) curve and the Area under the ROC curve (AUC). With the help of an illustrative example, we demonstrate the proposed idea of explaining the ROC curve, and visualising the uncertainties in these curves. For imbalanced datasets, the use of Precision-Recall Curve (PRC) is considered more appropriate, therefore we also demonstrate how to explain the PRCs with the help of Shapley values. The explanation of robustness can help analysts in a number of ways, for example, it can help in feature selection by identifying the irrelevant features that can be removed to reduce the computational complexity. It can also help in identifying the features having critical contributions or negative contributions towards robustness.
SPIDER: A Practical Fuzzing Framework to Uncover Stateful Performance Issues in SDN Controllers

—Performance issues in software-deﬁned network (SDN) controllers can have serious impacts on the performance and availability of networks. We speciﬁcally consider stateful performance issues, where a sequence of initial input messages drives an SDN controller into a state such that its performance degrades pathologically when processing subsequent messages. We identify key challenges in applying canonical program anal- ysis techniques: large input space of messages (e.g., stateful OpenFlow protocol), complex code base and software architecture (e.g., OSGi framework with dynamic launch), and the semantic dependencies between the internal state and external inputs. We design SPIDER, a practical fuzzing workﬂow that tackles these challenges and automatically uncovers such issues in SDN controllers. SPIDER’s design entails a careful synthesis and extension of semantic fuzzing, performance fuzzing, and static analysis, taken together with domain-speciﬁc insights to tackle these challenges. We show that our design workﬂow is robust across two controllers—ONOS and OpenDaylight—with very different internal implementations. Using SPIDER, we were able to identify and conﬁrm multiple stateful performance issues.
Regulating Effect of Biaxial Strain on Electronic, Optical and Photocatalytic Properties In Promising X2pas (X = Si, Ge and Sn) Monolayers

Photocatalytic water splitting is an effective way to obtain renewable clean energy. The challenge is to design tunable photocatalyst to meet the needs in different environments. At the same time, the oxygen and hydrogen evolution reactions (OER and HER) on the photocatalyst should be separated, which will be conducive to the separation of products. * Corresponding author E-mail: wangxiaochun@tsinghua.org. cn (Xiao-Chun Wang) 2 The electronic, optical and photocatalytic properties of Janus X2PAs (X = Si, Ge and Sn) monolayers are explored by first-principles calculation. All the strain-free X2PAs monolayers exhibit excellent photocatalytic properties with suitable band edge positions straddling the standard redox potential of water and large visible light absorption coefficients (up to 105 cm-1). Interestingly, the intrinsic internal electric field is favorable for separating photogenerated carriers to different surfaces of the monolayer. It contributes to realize the OER and HER separated on different sides of the monolayer. In particular, the energy band edge positions of X2PAs monolayers can be well adjusted by biaxial strain. Then it can effectively modulate photocatalytic reactions, suggesting X2PAs monolayers can be a piezo-photocatalytic switch between the OER, HER and fullreaction of redox for water splitting. This investigation not only highlights that the photocatalyst X2PAs monolayers with the separated OER and HER can be effectively tuned by the mechanical strain, but also provides a new strategy for designing highly adaptable and tunable piezo-photocatalysts.
Dynamical invariant formalism of shortcuts to adiabaticity

We give a pedagogical introduction to dynamical invariant formalism of shortcuts to adiabaticity. For a given operator form of the Hamiltonian with undetermined coefficients, the dynamical invariant is introduced to design the coefficients. We discuss how the method allows us to mimic adiabatic dynamics and describe a relation to the counterdiabatic formalism. The equation for the dynamical invariant takes a familiar form and is often used in various fields of physics. We introduce examples of Lax pair, quantum brachistochrone and flow equation. This article is part of the theme issue ‘Shortcuts to adiabaticity: theoretical, experimental and interdisciplinary perspectives’.
Mining and evaluation of patients' diagnostic therapeutic paths through state sequences analysis

The concept of care pathways is increasingly being used to enhance the quality of care and to optimize the use of resources for health care. Nevertheless, recommendations regarding the sequence of care are mostly based on consensus-based decisions as there is a lack of evidence on effective treatment sequences. In a real-world setting, classical statistical tools resulted to be insufficient to adequately consider a phenomenon with such high variability and has to be integrated with novel data mining techniques suitable of identifying patterns in complex data structures. Data-driven techniques can potentially support the empirical identification of effective care sequences by extracting them from data collected routinely. The purpose of this study is to perform sequence analysis to identify different patterns of treatment and to assess the most efficient in preventing adverse events. The clinical application that motivated the study of this method concerns the several problems frequently encountered in the quality of care provided in the mental health field. In particular, we analyzed administrative data provided by Regione Lombardia related to all the beneficiaries of the National Health Service with a diagnosis of schizophrenia from 2015 to 2018 resident in Lombardy, a region of northern Italy. This methodology considers the patient's therapeutic path as a conceptual unit, i.e., a sequence, composed of a succession of different states that can describe longitudinal patient's status. This kind of information, such as common patterns of care that allowed us to risk profile patients, can provide health policymakers an opportunity to plan optimum and individualized patient care by allocating appropriate resources, analyzing trends in the health status of a population, and finding the risk factors that can be leveraged to prevent the decline of mental health status at the population level.
Temporally Adjustable Longitudinal Fluid-Attenuated Inversion Recovery MRI Estimation / Synthesis for Multiple Sclerosis

Multiple Sclerosis (MS) is a chronic progressive neurological disease characterized by the development of lesions in the white matter of the brain. T2-fluid-attenuated inversion recovery (FLAIR) brain magnetic resonance imaging (MRI) provides superior visualization and characterization of MS lesions, relative to other MRI modalities. Longitudinal brain FLAIR MRI in MS, involving repetitively imaging a patient over time, provides helpful information for clinicians towards monitoring disease progression. Predicting future whole brain MRI examinations with variable time lag has only been attempted in limited applications, such as healthy aging and structural degeneration in Alzheimer's Disease. In this article, we present novel modifications to deep learning architectures for MS FLAIR image synthesis, in order to support prediction of longitudinal images in a flexible continuous way. This is achieved with learned transposed convolutions, which support modelling time as a spatially distributed array with variable temporal properties at different spatial locations. Thus, this approach can theoretically model spatially-specific time-dependent brain development, supporting the modelling of more rapid growth at appropriate physical locations, such as the site of an MS brain lesion. This approach also supports the clinician user to define how far into the future a predicted examination should target. Accurate prediction of future rounds of imaging can inform clinicians of potentially poor patient outcomes, which may be able to contribute to earlier treatment and better prognoses. Four distinct deep learning architectures have been developed. The ISBI2015 longitudinal MS dataset was used to validate and compare our proposed approaches. Results demonstrate that a modified ACGAN achieves the best performance and reduces variability in model accuracy.
Pathology Synthesis of 3D Consistent Cardiac MR Images Using 2D VAEs and GANs

We propose a method for synthesizing cardiac magnetic resonance (MR) images with plausible heart pathologies and realistic appearances for the purpose of generating labeled data for the application of supervised deep-learning (DL) training. The image synthesis consists of label deformation and label-to-image translation tasks. The former is achieved via latent space interpolation in a VAE model, while the latter is accomplished via a label-conditional GAN model. We devise three approaches for label manipulation in the latent space of the trained VAE model; i) intra-subject synthesis aiming to interpolate the intermediate slices of a subject to increase the through-plane resolution, ii) inter-subject synthesis aiming to interpolate the geometry and appearance of intermediate images between two dissimilar subjects acquired with different scanner vendors, and iii) pathology synthesis aiming to synthesize a series of pseudo-pathological synthetic subjects with characteristics of a desired heart disease. Furthermore, we propose to model the relationship between 2D slices in the latent space of the VAE prior to reconstruction for generating 3D-consistent subjects from stacking up 2D slice-by-slice generations. We demonstrate that such an approach could provide a solution to diversify and enrich an available database of cardiac MR images and to pave the way for the development of generalizable DL-based image analysis algorithms. We quantitatively evaluate the quality of the synthesized data in an augmentation scenario to achieve generalization and robustness to multi-vendor and multi-disease data for image segmentation. Our code is available at https://github.com/sinaamirrajab/CardiacPathologySynthesis
Parity-time symmetry-enhanced simultaneous magnon and photon blockade in cavity magnonic system

In this work, we consider a parity-time ( PT ) symmetric cavity magnonic system involving the magnon–photon interaction with small magnon Kerr nonlinearity. Moreover, we investigate the effect of PT -symmetry phase on both the magnon and photon blockade. We show that the PT -symmetry phase, which is achievable by properly selecting the system parameters, can relax the large Kerr nonlinearity requirement for magnon blockade. Consequently, simultaneous perfect magnon and photon blockade can be easily obtained even in the presence of a small value of magnon Kerr nonlinearity. The outstanding feature of the selected scheme is the occurrence of simultaneous perfect magnon and photon blockade with only a small value of magnon Kerr nonlinearity. While photon blockade can be easily distinguished experimentally, the experimental realization of magnon statistics and consequently magnon blockade is still a challenge. The prominent feature of the PT -symmetric cavity magnonic system can relax this challenge by following the magnon blockade criteria via the photon statistics.
The Impact of Stochastic Wavefunction Evolution in Dispersively Measured Bose-Einstein Condensates

A fundamental tenet of quantum mechanics is that measurements change a system’s wavefunction to that most consistent with the measurement outcome, even if no observer is present. Weak measurements—termed partial or non-destructive in diﬀerent settings—produce only limited information about the system, and as a result only minimally change the system’s state. Here, we theoretically and experimentally characterize quantum back-action in atomic Bose-Einstein condensates (BECs), weakly measured by the light scattered from a far-from resonant, i.e., dispersivly interacting, probe laser beam. We theoretically describe this process using a quantum trajectories approach and present a measurement model based on an ideal photodetection mechanism. We experimentally quantify the resulting wavefunction change with three observations: the change in total atom number, the contrast of a Ramsey interferometer, and the deposited energy. Further, we demonstrate control over parasitic eﬀects associated with the measurement process: probe-induced photoassociation (an intrinsic atomic process), as well as stray optical lattices (a technical consequence of weak retro-reﬂections). The observed back-action is in good agreement with our measurement model, enabling true quantum back-action limited measurements of BECs. Ultracold quantum ideal for studying the system-reservoir dynamics of large-scale many-body systems.Here, we characterize measurement back-action in Bose-Einstein condensates (BECs), weakly interacting with a far-from resonant laser beam. We develop a theoretical measurement model rooted in quantum techniques to study the interplay between the system-reservoir interaction, the scattered light, and the post-measurement system We demonstrate that both intrinsic and technical associated with the measurement eliminated, bringing the back-action into agreement with our