Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-perspective Information Fusion Res2Net with RandomSpecmix for Fake Speech Detection

In this paper, we propose the multi-perspective information fusion (MPIF) Res2Net with random Specmix for fake speech detection (FSD). The main purpose of this system is to improve the model's ability to learn precise forgery information for FSD task in low-quality scenarios. The task of random Specmix, a data augmentation, is to improve the generalization ability of the model and enhance the model's ability to locate discriminative information. Specmix cuts and pastes the frequency dimension information of the spectrogram in the same batch of samples without introducing other data, which helps the model to locate the really useful information. At the same time, we randomly select samples for augmentation to reduce the impact of data augmentation directly changing all the data. Once the purpose of helping the model to locate information is achieved, it is also important to reduce unnecessary information. The role of MPIF-Res2Net is to reduce redundant interference information. Deceptive information from a single perspective is always similar, so the model learning this similar information will produce redundant spoofing clues and interfere with truly discriminative information. The proposed MPIF-Res2Net fuses information from different perspectives, making the information learned by the model more diverse, thereby reducing the redundancy caused by similar information and avoiding interference with the learning of discriminative information. The results on the ASVspoof 2021 LA dataset demonstrate the effectiveness of our proposed method, achieving EER and min-tDCF of 3.29% and 0.2557, respectively.
Transfer: Cross Modality Knowledge Transfer using Adversarial Networks - A Study on Gesture Recognition

Knowledge transfer across sensing technology is a novel concept that has been recently explored in many application domains, including gesture-based human computer interaction. The main aim is to gather semantic or data driven information from a source technology to classify / recognize instances of unseen classes in the target technology. The primary challenge is the significant difference in dimensionality and distribution of feature sets between the source and the target technologies. In this paper, we propose TRANSFER, a generic framework for knowledge transfer between a source and a target technology. TRANSFER uses a language-based representation of a hand gesture, which captures a temporal combination of concepts such as handshape, location, and movement that are semantically related to the meaning of a word. By utilizing a pre-specified syntactic structure and tokenizer, TRANSFER segments a hand gesture into tokens and identifies individual components using a token recognizer. The tokenizer in this language-based recognition system abstracts the low-level technology-specific characteristics to the machine interface, enabling the design of a discriminator that learns technology-invariant features essential for recognition of gestures in both source and target technologies. We demonstrate the usage of TRANSFER for three different scenarios: a) transferring knowledge across technology by learning gesture models from video and recognizing gestures using WiFi, b) transferring knowledge from video to accelerometer, and d) transferring knowledge from accelerometer to WiFi signals.
SparseOptimizer: Sparsify Language Models through Moreau-Yosida Regularization and Accelerate through Compiler Co-design

This paper introduces SparseOptimizer, a novel deep learning optimizer that exploits Moreau-Yosida regularization to naturally induce sparsity in large language models such as BERT, ALBERT and GPT. Key to the design of SparseOptimizer is an embedded shrinkage operator, which imparts sparsity directly within the optimization process. This operator, backed by a sound theoretical framework, includes an analytical solution, thereby reinforcing the optimizer's robustness and efficacy. Crucially, SparseOptimizer's plug-and-play functionality eradicates the need for code modifications, making it a universally adaptable tool for a wide array of large language models. Empirical evaluations on benchmark datasets such as GLUE, RACE, SQuAD1, and SQuAD2 confirm that SparseBERT and SparseALBERT, when sparsified using SparseOptimizer, achieve performance comparable to their dense counterparts, BERT and ALBERT, while significantly reducing their parameter count. Further, this work proposes an innovative optimizer-compiler co-design strategy, demonstrating the potential of inference acceleration (\textbf{3.37x}, \textbf{6.30x}, and \textbf{7.15x} in comparison with Pytorch, TensorFlow, and LLVM generic compile, respectively) in SparseBERT when paired with an appropriately designed compiler. This study represents a significant step forward in the evolution of efficient, scalable, and high-performing large language models, setting a precedent for future exploration and optimization in this domain. The SparseOptimizer code and SparseALBERT model will be publicly available upon paper acceptance.
Towards Language-Based Modulation of Assistive Robots through Multimodal Models

In the field of Geriatronics, enabling effective and transparent communication between humans and robots is crucial for enhancing the acceptance and performance of assistive robots. Our early-stage research project investigates the potential of language-based modulation as a means to improve human-robot interaction. We propose to explore real-time modulation during task execution, leveraging language cues, visual references, and multimodal inputs. By developing transparent and interpretable methods, we aim to enable robots to adapt and respond to language commands, enhancing their usability and flexibility. Through the exchange of insights and knowledge at the workshop, we seek to gather valuable feedback to advance our research and contribute to the development of interactive robotic systems for Geriatronics and beyond.
Validating a virtual human and automated feedback system for training doctor-patient communication skills

Effective communication between a clinician and their patient is critical for delivering healthcare maximizing outcomes. Unfortunately, traditional communication training approaches that use human standardized patients and expert coaches are difficult to scale. Here, we present the development and validation of a scalable, easily accessible, digital tool known as the Standardized Online Patient for Health Interaction Education (SOPHIE) for practicing and receiving feedback on doctor-patient communication skills. SOPHIE was validated by conducting an experiment with 30 medical professionals. We found that participants who underwent SOPHIE condition performed significantly better than the control in overall communication, aggregate scores, empowering the patient, and showing empathy (p < 0.05 in all cases). The results presented in this paper provide early evidence of scalable and accessible virtual humans, chatbot technologies, and automated feedback generation being transformative for medical communication training by supplementing existing resources.
Revisiting Tropical Polynomial Division: Theory, Algorithms and Application to Neural Networks

Tropical geometry has recently found several applications in the analysis of neural networks with piecewise linear activation functions. This paper presents a new look at the problem of tropical polynomial division and its application to the simplification of neural networks. We analyze tropical polynomials with real coefficients, extending earlier ideas and methods developed for polynomials with integer coefficients. We first prove the existence of a unique quotient-remainder pair and characterize the quotient in terms of the convex bi-conjugate of a related function. Interestingly, the quotient of tropical polynomials with integer coefficients does not necessarily have integer coefficients. Furthermore, we develop a relationship of tropical polynomial division with the computation of the convex hull of unions of convex polyhedra and use it to derive an exact algorithm for tropical polynomial division. An approximate algorithm is also presented, based on an alternation between data partition and linear programming. We also develop special techniques to divide composite polynomials, described as sums or maxima of simpler ones. Finally, we present some numerical results to illustrate the efficiency of the algorithms proposed, using the MNIST handwritten digit and CIFAR-10 datasets.
Observing parity-time symmetry breaking in a Josephson parametric amplifier

A coupled two-mode system with balanced gain and loss is a paradigmatic example of an open quantum system that can exhibit real spectra despite being described by a non-Hermitian Hamiltonian. We utilize a degenerate parametric amplifier operating in three-wave mixing mode to realize such a system of balanced gain and loss between the two quadrature modes of the amplifier. By examining the time-domain response of the amplifier, we observe a characteristic transition from real-to-imaginary energy eigenvalues associated with the Parity-Time-symmetry-breaking transition.
FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction

Universal Information Extraction (UIE) has been introduced as a unified framework for various Information Extraction (IE) tasks and has achieved widespread success. Despite this, UIE models have limitations. For example, they rely heavily on span boundaries in the data during training, which does not reflect the reality of span annotation challenges. Slight adjustments to positions can also meet requirements. Additionally, UIE models lack attention to the limited span length feature in IE. To address these deficiencies, we propose the Fuzzy Span Universal Information Extraction (FSUIE) framework. Specifically, our contribution consists of two concepts: fuzzy span loss and fuzzy span attention. Our experimental results on a series of main IE tasks show significant improvement compared to the baseline, especially in terms of fast convergence and strong performance with small amounts of data and training epochs. These results demonstrate the effectiveness and generalization of FSUIE in different tasks, settings, and scenarios.
Detect Depression from Social Networks with Sentiment Knowledge Sharing

Social network plays an important role in propagating people's viewpoints, emotions, thoughts, and fears. Notably, following lockdown periods during the COVID-19 pandemic, the issue of depression has garnered increasing attention, with a significant portion of individuals resorting to social networks as an outlet for expressing emotions. Using deep learning techniques to discern potential signs of depression from social network messages facilitates the early identification of mental health conditions. Current efforts in detecting depression through social networks typically rely solely on analyzing the textual content, overlooking other potential information. In this work, we conduct a thorough investigation that unveils a strong correlation between depression and negative emotional states. The integration of such associations as external knowledge can provide valuable insights for detecting depression. Accordingly, we propose a multi-task training framework, DeSK, which utilizes shared sentiment knowledge to enhance the efficacy of depression detection. Experiments conducted on both Chinese and English datasets demonstrate the cross-lingual effectiveness of DeSK.
Towards predicting Pedestrian Evacuation Time and Density from Floorplans using a Vision Transformer

Conventional pedestrian simulators are inevitable tools in the design process of a building, as they enable project engineers to prevent overcrowding situations and plan escape routes for evacuation. However, simulation runtime and the multiple cumbersome steps in generating simulation results are potential bottlenecks during the building design process. Data-driven approaches have demonstrated their capability to outperform conventional methods in speed while delivering similar or even better results across many disciplines. In this work, we present a deep learning-based approach based on a Vision Transformer to predict density heatmaps over time and total evacuation time from a given floorplan. Specifically, due to limited availability of public datasets, we implement a parametric data generation pipeline including a conventional simulator. This enables us to build a large synthetic dataset that we use to train our architecture. Furthermore, we seamlessly integrate our model into a BIM-authoring tool to generate simulation results instantly and automatically.
Bayesian Interrupted Time Series for evaluating policy change on mental well-being: an application to England's welfare reform

Factors contributing to social inequalities are also associated with negative mental health outcomes leading to disparities in mental well-being. We propose a Bayesian hierarchical model which can evaluate the impact of policies on population well-being, accounting for spatial/temporal dependencies. Building on an interrupted time series framework, our approach can evaluate how different profiles of individuals are affected in different ways, whilst accounting for their uncertainty. We apply the framework to assess the impact of the United Kingdoms welfare reform, which took place throughout the 2010s, on mental well-being using data from the UK Household Longitudinal Study. The additional depth of knowledge is essential for effective evaluation of current policy and implementation of future policy.
Solving Bilevel Programs Based on Lower-Level Mond-Weir Duality

This paper focuses on developing effective algorithms for solving a bilevel program. The most popular approach is to replace the lower-level problem with its Karush-Kuhn-Tucker conditions to generate a mathematical program with complementarity constraints (MPCC). However, MPCC does not satisfy the Mangasarian-Fromovitz constraint qualification (MFCQ) at any feasible point. In this paper, inspired by a recent work using the lower-level Wolfe duality (WDP), we apply the lower-level Mond-Weir duality to present a new reformulation, called MDP, for bilevel program. It is shown that, under mild assumptions, they are equivalent in globally or locally optimal sense. An example is given to show that, different from MPCC, MDP may satisfy the MFCQ at its feasible points. Relations among MDP, WDP, and MPCC are investigated. On this basis, we extend the MDP reformulation to present another new reformulation (called eMDP), which has similar properties to MDP. Furthermore, to compare two new reformulations with the MPCC and WDP approaches, we design a procedure to generate 150 tested problems randomly and comprehensive numerical experiments show that MDP has quite evident advantages over MPCC and WDP in terms of feasibility to the original bilevel programs, success efficiency, and average CPU time, whereas eMDP is far superior to all other three reformulations. History: Accepted by Pascal Van Hentenryck, Area Editor for Computational Modeling: Methods & Analysis. Funding: This work was supported by the National Natural Science Foundation of China [Grants 12071280 and 11901380]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0108 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0108 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github. io/ .
Proof of the bounded conformal conjecture

Given any asymptotically flat 3-manifold $(M,g)$ with smooth, non-empty, compact boundary $\Sigma$, the conformal conjecture states that for every $\delta>0$, there exists a metric $g' = u^4 g$, with $u$ a harmonic function, such that the area of outermost minimal area enclosure $\tilde{\Sigma}_{g'}$ of $\Sigma$ with respect to $g'$ is less than $\delta$. Recently, the conjecture was used to prove the Riemannian Penrose inequality for black holes with zero horizon area, and was proven to be true under the assumption of existence of only a finite number of minimal area enclosures of boundary $\Sigma$, and boundedness of harmonic function $u$. We prove the conjecture assuming only the boundedness of $u$.
Automating Steady and Unsteady Adjoints: Efficiently Utilizing Implicit and Algorithmic Differentiation

Algorithmic differentiation (AD) has become increasingly capable and straightforward to use. However, AD is inefficient when applied directly to solvers, a feature of most engineering analyses. We can leverage implicit differentiation to define a general AD rule, making adjoints automatic. Furthermore, we can leverage the structure of differential equations to automate unsteady adjoints in a memory efficient way. We also derive a technique to speed up explicit differential equation solvers, which have no iterative solver to exploit. All of these techniques are demonstrated on problems of various sizes, showing order of magnitude speed-ups with minimal code changes. Thus, we can enable users to easily compute accurate derivatives across complex analyses with internal solvers, or in other words, automate adjoints using a combination of AD and implicit differentiation.
Dynamical phase transition of light in time-varying nonlinear dispersive media

We demonstrate the existence of a prethermal dynamical phase transition (DPT) for fluctuating optical beams propagating in nonlinear dispersive media. The DPT can be probed by suddenly changing in time the dispersion and nonlinearity parameters of the medium (thus realizing a"temporal interface"), a procedure that emulates a quench in a massive $\varphi^4$ model. Above a critical value of the quench identifying the transition, the fluctuating beam after the temporal interface is characterized by a correlation length that diverges algebraically at the transition. Below the critical quench, the beam exhibits an algebraic relaxation and a self-similar scaling. Our analysis also reveals a dimensional cross-over of the critical exponent, a characteristic feature of the optical DPT.
Rebricking frames and bases

In 1949, Denis Gabor introduced the ``complex signal'' (nowadays called ``analytic signal'') by combining a real function $f$ with its Hilbert transform $Hf$ to a complex function $f+ iHf$. His aim was to extract phase information, an idea that has inspired techniques as the monogenic signal and the complex dual tree wavelet transform. In this manuscript, we consider two questions: When do two real-valued bases or frames $\{f_{n} : n\in\mathbb{N}\}$ and $\{g_{n} : n\in\mathbb{N}\}$ form a complex basis or frame of the form $\{f_{n} + i g_{n}: n\in\mathbb{N}\}$? And for which bounded linear operators $A$ forms $\{f_{n} + i A f_{n} : n\in\mathbb{N}\}$ a complex-valued orthonormal basis, Riesz basis or frame, when $\{f_{n} : n\in\mathbb{N}\}$ is a real-valued orthonormal basis, Riesz basis or frame? We call this approach \emph{rebricking}. It is well-known that the analytic signals don't span the complex vector space $L^{2}(\mathbb{R}; \mathbb{C})$, hence $H$ is not a rebricking operator. We give a full characterization of rebricking operators for bases, in particular orthonormal and Riesz bases, Parseval frames, and frames in general. We also examine the special case of finite dimensional vector spaces and show that we can use any real, invertible matrix for rebricking if we allow for permutations in the imaginary part.
Gathering Galaxy Distances in Abundance with Roman Wide-Area Data

The extragalactic distance scale is fundamental to our understanding of astrophysics and cosmology. In recent years, the surface brightness fluctuation (SBF) method, applied in the near-IR, has proven especially powerful for measuring galaxy distances, first with HST and now with a new JWST program to calibrate the method directly from the tip of the red giant branch (TRGB). So far, however, the distances from space have been gathered slowly, one or two at a time. With the Roman Space Telescope, we have the opportunity to measure uniformly high-quality SBF distances to thousands of galaxies out to hundreds of Mpc. The impact of these data on cosmology and galaxy studies depends on the specifics of the survey, including the filter selection, exposure depth, and (especially) the sky coverage. While the baseline HLWAS survey in four filters plus the grism would yield useful data, the impact would be limited by the relatively small area. A more optimal approach would concentrate on the most efficient passband (F146), adopt an exposure time sufficient to measure good quality distances well out into the Hubble flow, and then maximize the sky coverage within the total time constraints. Grism observations over the same area can provide the needed information on redshifts and spectral energy distributions for compact sources, while colors for larger objects can be obtained from lower resolution surveys. The proposed plan will enable accurate determination of the physical properties of thousands of nearby galaxies, an independent measure of the Hubble constant $H_0$ with negligible statistical error, and competitive constraints on $S_8{\,=\,}\sigma_8(\Omega_m/0.3)^{0.5}$. The resulting data set will be a phenomenal resource for a wide range of studies in astrophysics and cosmology.
Detector-Free Structure from Motion

We propose a new structure-from-motion framework to recover accurate camera poses and point clouds from unordered images. Traditional SfM systems typically rely on the successful detection of repeatable keypoints across multiple views as the first step, which is difficult for texture-poor scenes, and poor keypoint detection may break down the whole SfM system. We propose a new detector-free SfM framework to draw benefits from the recent success of detector-free matchers to avoid the early determination of keypoints, while solving the multi-view inconsistency issue of detector-free matchers. Specifically, our framework first reconstructs a coarse SfM model from quantized detector-free matches. Then, it refines the model by a novel iterative refinement pipeline, which iterates between an attention-based multi-view matching module to refine feature tracks and a geometry refinement module to improve the reconstruction accuracy. Experiments demonstrate that the proposed framework outperforms existing detector-based SfM systems on common benchmark datasets. We also collect a texture-poor SfM dataset to demonstrate the capability of our framework to reconstruct texture-poor scenes. Based on this framework, we take $\textit{first place}$ in Image Matching Challenge 2023.
Symphonize 3D Semantic Scene Completion with Contextual Instance Queries

`3D Semantic Scene Completion (SSC) has emerged as a nascent and pivotal undertaking in autonomous driving, aiming to predict voxel occupancy within volumetric scenes. However, prevailing methodologies primarily focus on voxel-wise feature aggregation, while neglecting instance semantics and scene context. In this paper, we present a novel paradigm termed Symphonies (Scene-from-Insts), that delves into the integration of instance queries to orchestrate 2D-to-3D reconstruction and 3D scene modeling. Leveraging our proposed Serial Instance-Propagated Attentions, Symphonies dynamically encodes instance-centric semantics, facilitating intricate interactions between image-based and volumetric domains. Simultaneously, Symphonies enables holistic scene comprehension by capturing context through the efficient fusion of instance queries, alleviating geometric ambiguity such as occlusion and perspective errors through contextual scene reasoning. Experimental results demonstrate that Symphonies achieves state-of-the-art performance on challenging benchmarks SemanticKITTI and SSCBench-KITTI-360, yielding remarkable mIoU scores of 15.04 and 18.58, respectively. These results showcase the paradigm's promising advancements. The code is available at https://github.com/hustvl/Symphonies.