Miracle-dz/UniFAR-FTUs huggingface dataset:
iEmoTTS: Toward Robust Cross-Speaker Emotion Transfer and Control for Speech Synthesis Based on Disentanglement Between Prosody and Timbre

Cross-speaker emotion transfer is a common approach to generating emotional speech when speech data with emotion labels from target speakers is not available. This paper presents a novel cross-speaker emotion transfer system named iEmoTTS. The system is composed of an emotion encoder, a prosody predictor, and a timbre encoder. The emotion encoder extracts the identity of emotion type and the respective emotion intensity from the mel-spectrogram of input speech. The emotion intensity is measured by the posterior probability that the input utterance carries that emotion. Unlike many other studies which focus on disentangling speaker and style factors of speech, the iEmoTTS is designed to achieve cross-speaker emotion transfer via disentanglement between prosody and timbre. Prosody is considered the primary carrier of emotion-related speech characteristics, and timbre accounts for the essential characteristics for speaker identification. Zero-shot emotion transfer, meaning that the speech of target speakers is not seen in model training, is also realized with iEmoTTS. Extensive experiments of subjective evaluation have been carried out. The results demonstrate the effectiveness of iEmoTTS compared with other recently proposed systems of cross-speaker emotion transfer. It is shown that iEmoTTS can produce speech with designated emotion types and controllable emotion intensity. With appropriate information bottleneck capacity, iEmoTTS is able to transfer emotional information to a new speaker effectively. Audio samples are publicly available.
Three-point bounds for sphere packing

We deﬁne three-point bounds for sphere packing that reﬁne the linear programming bound, and we compute these bounds numerically using semideﬁnite programming by choosing a truncation radius for the three-point function. As a result, we obtain new upper bounds on the sphere packing density in dimensions 4 through 7 and 9 through 16. We also give a diﬀerent three-point bound for lattice packing and conjecture that this second bound is sharp in dimension 4.
Drag force on spherical particles trapped at a liquid interface

The dynamics of particles attached to an interface separating two immiscible fluids are encountered in a wide variety of applications. Here we present a combined asymptotic and numerical investigation of the fluid motion past spherical particles attached to a deformable interface undergoing uniform creeping flows in the limit of small Capillary number and small deviation of the contact angle from 90 degrees. Under the assumption of a constant three-phase contact angle, we calculate the interfacial deformation around an isolated particle and a particle pair. Applying the Lorentz reciprocal theorem to the zeroth-order approximation corresponding to spherical particles at a flat interface and the first correction in Capillary number and correction contact angle allows us to obtain explicit analytical expressions for the hydrodynamic drag in terms of the zeroth-order approximations and the correction deformations. The drag coefficients are computed as a function of the three-phase contact angle, the viscosity ratio of the two fluids, the Bond number, and the separation distance between the particles. In addition, the capillary force acting on the particles due to the interfacial deformation is calculated.
Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding

This paper studies a transferable phoneme embedding framework that aims to deal with the cross-lingual text-to-speech (TTS) problem under the few-shot setting. Transfer learning is a common approach when it comes to few-shot learning since training from scratch on few-shot training data is bound to overfit. Still, we find that the naive transfer learning approach fails to adapt to unseen languages under extremely few-shot settings, where less than 8 minutes of data is provided. We deal with the problem by proposing a framework that consists of a phoneme-based TTS model and a codebook module to project phonemes from different languages into a learned latent space. Furthermore, by utilizing phoneme-level averaged self-supervised learned features, we effectively improve the quality of synthesized speeches. Experiments show that using 4 utterances, which is about 30 seconds of data, is enough to synthesize intelligible speech when adapting to an unseen language using our framework.
Challenges and Opportunities in Multi-device Speech Processing

We review current solutions and technical challenges for automatic speech recognition, keyword spotting, device arbitration, speech enhancement, and source localization in multidevice home environments to provide context for the INTERSPEECH 2022 special session,"Challenges and opportunities for signal processing and machine learning for multiple smart devices". We also identify the datasets needed to support these research areas. Based on the review and our research experience in the multi-device domain, we conclude with an outlook on the future evolution
Modern Question Answering Datasets and Benchmarks: A Survey

Question Answering (QA) is one of the most important natural language processing (NLP) tasks. It aims using NLP technologies to generate a corresponding answer to a given question based on the massive unstructured corpus. With the development of deep learning, more and more challenging QA datasets are being proposed, and lots of new methods for solving them are also emerging. In this paper, we investigate influential QA datasets that have been released in the era of deep learning. Specifically, we begin with introducing two of the most common QA tasks - textual question answer and visual question answering - separately, covering the most representative datasets, and then give some current challenges of QA research.
On the topology of character varieties of once-punctured torus bundles

This paper presents, for the special case of once-punctured torus bundles, a natural method to study the character varieties of hyperbolic 3–manifolds that are bundles over the circle. The main strategy is to restrict characters to the ﬁbre of the bundle, and to analyse the resulting branched covering map. This allows us to extend results of Steven Boyer, Erhard Luft and Xingru Zhang. Both SL ( 2 , C ) –character varieties and PSL ( 2 , C ) –character varieties are considered. As an explicit application of these methods, we build on work of Baker and Petersen to show that there is an inﬁnite family of hyperbolic once-punctured bundles with canonical curves of PSL ( 2 , C ) –characters of unbounded genus.
Stochastic Poisson Surface Reconstruction

We introduce a statistical extension of the classic Poisson Surface Reconstruction algorithm for recovering shapes from 3D point clouds. Instead of outputting an implicit function, we represent the reconstructed shape as a modified Gaussian Process, which allows us to conduct statistical queries (e.g., the likelihood of a point in space being on the surface or inside a solid). We show that this perspective: improves PSR's integration into the online scanning process, broadens its application realm, and opens the door to other lines of research such as applying task-specific priors.
Dressing Avatars

Despite recent progress in developing animatable full-body avatars, realistic modeling of clothing - one of the core aspects of human self-expression - remains an open challenge. State-of-the-art physical simulation methods can generate realistically behaving clothing geometry at interactive rates. Modeling photorealistic appearance, however, usually requires physically-based rendering which is too expensive for interactive applications. On the other hand, data-driven deep appearance models are capable of efficiently producing realistic appearance, but struggle at synthesizing geometry of highly dynamic clothing and handling challenging body-clothing configurations. To this end, we introduce pose-driven avatars with explicit modeling of clothing that exhibit both photorealistic appearance learned from real-world data and realistic clothing dynamics. The key idea is to introduce a neural clothing appearance model that operates on top of explicit geometry: at training time we use high-fidelity tracking, whereas at animation time we rely on physically simulated geometry. Our core contribution is a physically-inspired appearance network, capable of generating photorealistic appearance with view-dependent and dynamic shadowing effects even for unseen body-clothing configurations. We conduct a thorough evaluation of our model and demonstrate diverse animation results on several subjects and different types of clothing. Unlike previous work on photorealistic full-body avatars, our approach can produce much richer dynamics and more realistic deformations even for many examples of loose clothing. We also demonstrate that our formulation naturally allows clothing to be used with avatars of different people while staying fully animatable, thus enabling, for the first time, photorealistic avatars with novel clothing.
A note on Linear Bottleneck networks and their Transition to Multilinearity

Randomly initialized wide neural networks transition to linear functions of weights as the width grows, in a ball of radius O (1) around initialization. A necessary condition for this result is that all layers of the network are wide enough, i.e., all widths tend to inﬁnity. However, the transition to linearity breaks down when this inﬁnite width assumption is violated. In this work we show that linear networks with a bottleneck layer learn bilinear functions of the weights, in a ball of radius O (1) around initialization. In general, for B − 1 bottleneck layers, the network is a degree B multilinear function of weights. Importantly, the degree only depends on the number of bottlenecks and not the total depth of the network. in each weight matrix. However, as a function of all weights together, it is a polynomial of degree equal to total number of layers. Our analysis of bottleneck networks shows that, as the width of non-bottleneck layers grows, the degree of this polynomial reduces to the number of bottleneck layers in the network plus 1, and becomes independent of the total number of layers. For wide networks without bottlenecks, this recovers the result of transition to linearity of wide neural networks. In our technical analysis, for a BNN with B − 1 bottleneck layers, we show that the spectral norm of the B + 1 st derivative of the network function with respect to parameters, scales as with 1 / √ m where m is the width of non bottleneck layers, but the spectral norm of the B th derivative is Ω(1) . As a result, when m goes to inﬁnity, the network function transitions to a B th order polynomial of the weights. We further strengthen this claim by showing this polynomial is in fact a multilinear function, where the network function is jointly linear in layer weights between consecutive bottleneck layers.
Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting

In this paper, we propose a novel end-to-end user-defined keyword spotting method that utilizes linguistically corresponding patterns between speech and text sequences. Unlike previous approaches requiring speech keyword enrollment, our method compares input queries with an enrolled text keyword sequence. To place the audio and text representations within a common latent space, we adopt an attention-based cross-modal matching approach that is trained in an end-to-end manner with monotonic matching loss and keyword classification loss. We also utilize a de-noising loss for the acoustic embedding network to improve robustness in noisy environments. Additionally, we introduce the LibriPhrase dataset, a new short-phrase dataset based on LibriSpeech for efficiently training keyword spotting models. Our proposed method achieves competitive results on various evaluation sets compared to other single-modal and cross-modal baselines.
Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehicles

Reinforcement Learning (RL) has presented an impressive performance in video games through raw pixel imaging and continuous control tasks. However, RL performs poorly with high-dimensional observations such as raw pixel images. It is generally accepted that physical state-based RL policies such as laser sensor measurements give a more sample-efficient result than learning by pixels. This work presents a new approach that extracts information from a depth map estimation to teach an RL agent to perform the mapless navigation of Unmanned Aerial Vehicle (UAV). We propose the Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning (Depth-CUPRL) that estimates the depth of images with a prioritized replay memory. We used a combination of RL and Contrastive Learning to lead with the problem of RL based on images. From the analysis of the results with Unmanned Aerial Vehicles (UAVs), it is possible to conclude that our Depth-CUPRL approach is effective for the decision-making and outperforms state-of-the-art pixel-based approaches in the mapless navigation capability.
TARGETED LEARNING IN OBSERVATIONAL STUDIES WITH MULTI-LEVEL TREATMENTS: AN EVALUATION OF ANTIPSYCHOTIC DRUG TREATMENT SAFETY FOR PATIENTS WITH SERIOUS MENTAL ILLNESSES

We investigate estimation of causal effects of multiple competing (multi-valued) treatments in the absence of randomization. Our work is motivated by an intention-to-treat study of the relative metabolic risk of assignment to one of six commonly prescribed antipsychotic drugs in a cohort of adults with serious mental illness. Doubly-robust estimators of multi-level treatment effects with observational data, such as targeted minimum loss-based estimation (TMLE), require that either the treatment model or outcome model is correctly speciﬁed to ensure consistent estimation. However, common TMLE implementations estimate treatment probabilities using multiple binomial regressions rather than a single multinomial regression. We implement a TMLE estimator that uses multinomial treatment assignment and ensemble machine learning to estimate average treatment effects. Our implementation achieves superior coverage probability relative to the binomial implementation in simulation experiments with varying treatment propensity overlap and event rates. An evaluation of the causal effects of six antipsychotic drugs on the risk of diabetes or death illustrates our approach. We ﬁnd a relative safety beneﬁt of moving from a second-generation antipyschotic thought to have more favorable metabolic risk proﬁle relative to other second-generation drugs to a less commonly prescribed ﬁrst-generation antipyschotic known for having a low rate of metabolic disturbance.
Space-Bounded Unitary Quantum Computation with Postselection

Space-bounded computation has been a central topic in classical and quantum complexity theory. In the quantum case, every elementary gate must be unitary. This restriction makes it unclear whether the power of space-bounded computation changes by allowing intermediate measurement. In the bounded error case, Fefferman and Remscrim [STOC 2021, pp.1343–1356] and Girish, Raz and Zhan [ICALP 2021, pp.73:1–73:20] recently provided the break-through results that the power does not change. This paper shows that a similar result holds for space-bounded quantum computation with postselection . Namely, it is proved possible to eliminate intermediate postselections and measurements in the space-bounded quantum computation in the bounded-error setting. Our result strengthens the recent result by Le Gall, Nishimura and Yakaryilmaz [TQC 2021, pp.10:1– 10:17] that logarithmic-space bounded-error quantum computation with intermediate postselections and measurements is equivalent in computational power to logarithmic-space unbounded-error probabilistic computation. As an application, it is shown that bounded-error space-bounded one-clean qubit computation (DQC1) with postselection is equivalent in computational power to unbounded-error space-bounded probabilistic computation, and the computational supremacy of the bounded-error space-bounded DQC1 is interpreted in complexity-theoretic terms.
Spin polarized electron beams production beyond III-V semiconductors

This paper summarizes the state of the art of photocathode based on III-V semiconductors for spin polarized electron beam production. The limitations preventing this class of material to provide the long term reliability at the highest average beam currents necessary for some of the new accelerator facilities or proposed upgrades of existing ones are illustrated. Promising alternative classes of materials are identified showing properties that can be leveraged to synthesize photocathode structures that can outperform III-V semiconductors in the production of spin polarized electron beams and support the operating conditions of advanced electron sources for new facilities.
Category-Level 6D Object Pose Estimation in the Wild: A Semi-Supervised Learning Approach and A New Dataset

6D object pose estimation is one of the fundamental problems in computer vision and robotics research. While a lot of recent efforts have been made on generalizing pose estimation to novel object instances within the same category, namely category-level 6D pose estimation, it is still restricted in constrained environments given the limited number of annotated data. In this paper, we collect Wild6D, a new unlabeled RGBD object video dataset with diverse instances and backgrounds. We utilize this data to generalize category-level 6D object pose estimation in the wild with semi-supervised learning. We propose a new model, called Rendering for Pose estimation network RePoNet, that is jointly trained using the free ground-truths with the synthetic data, and a silhouette matching objective function on the real-world data. Without using any 3D annotations on real data, our method outperforms state-of-the-art methods on the previous dataset and our Wild6D test set (with manual annotations for evaluation) by a large margin. Project page with Wild6D data: https://oasisyang.github. io/semi-pose .
Uncovering Patterns for Adverse Pregnancy Outcomes with Spatial Analysis: Evidence from Philadelphia ∗

In this case study, we analyze the risk of stillbirth and preterm birth in Philadelphia from 2010 to 2017. We exploit a rich electronic health records dataset (45,919 deliveries at hospitals within Penn Medicine), and augmented with neighborhood data from 363 census tracts of Philadelphia. We conduct a two-stage statistical analysis. In the ﬁrst stage, we introduce a Bayesian spatial logistic regression model to study the patient-speciﬁc risk of stillbirth and preterm birth. Our model accounts for heterogeneity and spatial autocorrelation between neighborhoods. We ﬁnd both patient-level characteristics (e.g. self-identiﬁed racial/ethnic group) and neighborhood-level characteristics (e.g. violent crime) are highly associated with patient-speciﬁc risk of both outcomes. In the second stage, we aggregate the estimates from our spatial model to quantify neighborhood risks of stillbirth and preterm birth. We ﬁnd that neighborhoods in West Philadelphia and North Philadelphia are at highest risk of these outcomes. Speciﬁcally, neighborhoods with higher rates of women living in poverty or on public assistance have higher risk, while neighborhoods with higher rates of women who are college-educated or in the labor force have lower risk. Our Bayesian approach provides meaningful uncertainty measures for these neighborhood risk probabilities and would be useful for public health interventions.
Submission to Generic Event Boundary Detection Challenge@CVPR 2022: Local Context Modeling and Global Boundary Decoding Approach

Generic event boundary detection (GEBD) is an important yet challenging task in video understanding, which aims at detecting the moments where humans naturally perceive event boundaries. In this paper, we present a local context modeling and global boundary decoding approach for GEBD task. Local context modeling sub-network is proposed to perceive diverse patterns of generic event boundaries, and it generates powerful video representations and reliable boundary confidence. Based on them, global boundary decoding sub-network is exploited to decode event boundaries from a global view. Our proposed method achieves 85.13% F1-score on Kinetics-GEBD testing set, which achieves a more than 22% F1-score boost compared to the baseline method. The code is available at https://github.com/JackyTown/GEBD_Challenge_CVPR2022.
LISA Gravitational Wave Sources in a Time-varying Galactic Stochastic Background

A unique challenge for data analysis with the Laser Interferometer Space Antenna (LISA) is that the noise backgrounds from instrumental noise and astrophysical sources will change significantly over both the year and the entire mission. Variations in the noise levels will be on timescales comparable to, or shorter than, the time most signals spend in the detector’s sensitive band. The variation in the amplitude of the galactic stochastic GW background from galactic binaries as the antenna pattern rotates relative to the galactic center is a particularly significant component of the noise variation. LISA’s sensitivity to different source classes will therefore vary as a function of sky location and time. The variation will impact both overall signal-to-noise and the efficiency of alerts to EM observers to search for multimessenger counterparts.
Paving the Way Toward Mobile IAB: Problems, Solutions and Challenges

Deploying access and backhaul as wireless links, a.k. a. integrated access and backhaul (IAB), is envisioned as a viable approach to enable flexible and dense networks. Even further, mobile IAB (mIAB) is a candidate solution to enhance the connectivity of multiple user equipment (UE) moving together. In this context, different of other works from the literature, the present work overviews the basis for the deployment of mIAB by presenting: 1) the current status of IAB standardization in the fifth generation (5G) new radio (NR); 2) a new taxonomy for state-of-the-art works regarding fixed IAB and mIAB; 3) an extensive performance analysis of mIAB based on simulation results; and 4) open challenges and potential future prospects of mIAB. Specifically, the proposed taxonomy classifies IAB works according to different perspectives and categorizes mIAB works according to the type of mobile node. For each type of mobile node, the main studied topics are presented. Regarding the performance evaluation, we consider an urban macro scenario where mIAB nodes are deployed in buses in order to improve the passengers’ connection. The results show that, compared to other network architectures, the deployment of mIAB nodes remarkably improves the passengers’ throughput and latency in both downlink and uplink.
Provably Efficient Reinforcement Learning for Online Adaptive Influence Maximization

Online influence maximization aims to maximize the influence spread of a content in a social network with unknown network model by selecting a few seed nodes. Recent studies followed a non-adaptive setting, where the seed nodes are selected before the start of the diffusion process and network parameters are updated when the diffusion stops. We consider an adaptive version of content-dependent online influence maximization problem where the seed nodes are sequentially activated based on real-time feedback. In this paper, we formulate the problem as an infinite-horizon discounted MDP under a linear diffusion process and present a model-based reinforcement learning solution. Our algorithm maintains a network model estimate and selects seed users adaptively, exploring the social network while improving the optimal policy optimistically. We establish $\widetilde O(\sqrt{T})$ regret bound for our algorithm. Empirical evaluations on synthetic network demonstrate the efficiency of our algorithm.