Miracle-dz/UniFAR-FTUs huggingface dataset:
Large-scale unsupervised audio pre-training for video-to-speech synthesis

Video-to-speech synthesis is the task of reconstructing the speech signal from a silent video of a speaker. Most established approaches to date involve a two-step process, whereby an intermediate representation from the video, such as a spectrogram, is extracted first and then passed to a vocoder to produce the raw audio. Some recent work has focused on end-to-end synthesis, whereby the generation of raw audio and any intermediate representations is performed jointly. All such approaches involve training on data from almost exclusively audio-visual datasets, i.e. every audio sample has a corresponding video sample. This precludes the use of abundant audio-only datasets which may not have a corresponding visual modality (e.g. audiobooks, radio podcasts, speech recognition datasets etc.), as well as audio-only architectures that have been developed by the audio machine learning community over the years. In this paper we propose to train encoder-decoder models on more than 3,500 hours of audio data at 24kHz, and then use the pre-trained decoders to initialize the audio decoders for the video-to-speech synthesis task. The pre-training step uses audio samples only and does not require labels or corresponding samples from other modalities (visual, text). We demonstrate that this pre-training step improves the reconstructed speech and that it is an unexplored way to improve the quality of the generator in a cross-modal task while only requiring samples from one of the modalities. We conduct experiments using both raw audio and mel spectrograms as target outputs and benchmark our models with existing work.
Mesoscale mode coupling theory for the weakly asymmetric simple exclusion process

The asymmetric simple exclusion process and its analysis by mode coupling theory (MCT) is reviewed. To treat the weakly asymmetric case at large space scale $x\varepsilon^{-1}$, %(corresponding to small Fourier momentum at scale $p\varepsilon$), large time scale $t \varepsilon^{-\chi}$ and weak hopping bias $b \varepsilon^{\kappa}$ in the limit $\varepsilon \to 0$ we develop a mesoscale MCT that allows for studying the crossover at $\kappa=1/2$ and $\chi=2$ from Kardar-Parisi-Zhang (KPZ) to Edwards-Wilkinson (EW) universality. The dynamical structure function is shown to satisfy for all $\kappa$ an integral equation that is independent of the microscopic model parameters and has a solution that yields a scale-invariant function with the KPZ dynamical exponent $z=3/2$ at scale $\chi=3/2+\kappa$ for $0\leq\kappa<1/2$ and for $\chi=2$ the exact Gaussian EW solution with $z=2$ for $\kappa>1/2$. At the crossover point it is a function of both scaling variables which converges at macroscopic scale to the conventional MCT approximation of KPZ universality for $\kappa<1/2$. This fluctuation pattern confirms long-standing conjectures for $\kappa \leq 1/2$ and is in agreement with mathematically rigorous results for $\kappa>1/2$ despite the numerous uncontrolled approximations on which MCT is based.
Efficient High-Resolution Template Matching with Vector Quantized Nearest Neighbour Fields

Template matching is a fundamental problem in computer vision with applications in fields including object detection, image registration, and object tracking. Current methods rely on nearest-neighbour (NN) matching, where the query feature space is converted to NN space by representing each query pixel with its NN in the template. NN-based methods have been shown to perform better in occlusions, appearance changes, and non-rigid transformations; however, they scale poorly with high-resolution data and high feature dimensions. We present an NN-based method which efficiently reduces the NN computations and introduces filtering in the NN fields (NNFs). A vector quantization step is introduced before the NN calculation to represent the template with $k$ features, and the filter response over the NNFs is used to compare the template and query distributions over the features. We show that state-of-the-art performance is achieved in low-resolution data, and our method outperforms previous methods at higher resolution.
MOVESe: MOVablE and Moving LiDAR Scene Segmentation with Improved Navigation in Seg-label free settings

Accurate detection of movable and moving objects in LiDAR is of vital importance for navigation. Most existing works focus on extracting and removing moving objects during navigation. Movable objects like pedestrians, parked vehicles, etc. although static may move in the future. This leads to erroneous navigation and accidents. In such cases, it becomes necessary to detect potentially movable objects. To this end, we present a learning-based approach that segments movable and moving objects by generating static parts of scenes that are otherwise occluded. Our model performs superior to existing baselines on static Li-DAR reconstructions using 3 datasets including a challenging sparse industrial dataset. We achieve this without the assistance of any segmentation labels because such labels might not always be available for less popular yet important settings like industrial environments. The non-movable static parts of the scene generated by our model are of vital importance for downstream navigation for SLAM. The movable objects detected by our model can be fed to a down-stream 3D detector for aiding navigation. Though we do not use segmentation, we evaluate our method against navigation baselines that use it to remove dynamic objects for SLAM. Through extensive experiments on several datasets, we showcase that our model surpasses these baselines on navigation.
Can Competition Outperform Collaboration? The Role of Misbehaving Agents

We investigate a novel approach to resilient distributed optimization with quadratic costs in a multiagent system prone to unexpected events that make some agents misbehave. In contrast to commonly adopted filtering strategies, we draw inspiration from phenomena modeled through the Friedkin–Johnsen dynamics and argue that adding competition to the mix can improve resilience in the presence of misbehaving agents. Our intuition is corroborated by analytical and numerical results showing that 1) there exists a nontrivial tradeoff between full collaboration and full competition and 2) our competition-based approach can outperform state-of-the-art algorithms based on weighted mean subsequence reduced. We also study the impact of communication topology and connectivity on resilience, pointing out insights into robust network design.
Hidden connectivity structures control collective network dynamics

Many observables of brain dynamics appear to be optimized for computation. Which connectivity structures underlie this fine-tuning? We propose that many of these structures are naturally encoded in the space that more directly relates to network dynamics - the space of the connectivity eigenmodes. We develop a mathematical theory to impose eigenmode structures on connectivity, systematically characterizing their effect on network dynamics. We find the density of nearly-critical eigenvalues to be a particularly fundamental structure. It flexibly controls the power-law scaling of dynamical observables, in analogy with the system's spatial dimension in classical critical phenomena. This mechanism provides control over observables which are found to be fine-tuned in brain networks, but remained so far unexplained by traditionally studied structures, such as connectivity motifs. Specifically, the slope of the principal component spectrum of neural activity can be fine-tuned, as observed in primary visual cortex of mice. Furthermore, a novel transition between high and low dimensional activity allows for a wide and flexible tuning of dimensionality, as observed throughout cortex. The here discovered structures thus largely complement motifs. In fact, they are of a different, collective nature: they are not reflected by any local motif configuration. This result shows that many functionally relevant structures can remain hidden within the apparent randomness of highly heterogeneous cortical circuits. Our methods enable revealing these structures and investigate their effect on network dynamics.
Reversible Quantization Index Modulation for Static Deep Neural Network Watermarking

Static deep neural network (DNN) watermarking techniques typically employ irreversible methods to embed watermarks into the DNN model weights. However, this approach causes permanent damage to the watermarked model and fails to meet the requirements of integrity authentication. Reversible data hiding (RDH) methods offer a potential solution, but existing approaches suffer from weaknesses in terms of usability, capacity, and fidelity, hindering their practical adoption. In this paper, we propose a novel RDH-based static DNN watermarking scheme using quantization index modulation (QIM). Our scheme incorporates a novel approach based on a one-dimensional quantizer for watermark embedding. Furthermore, we design two schemes to address the challenges of integrity protection and legitimate authentication for DNNs. Through simulation results on training loss and classification accuracy, we demonstrate the feasibility and effectiveness of our proposed schemes, highlighting their superior adaptability compared to existing methods.
IKK promotes naïve T cell survival by repressing RIPK1-dependent apoptosis and activating NF-κB

The inhibitor of κB kinase (IKK) complex regulates the activation of the nuclear factor κB (NF-κB) family of transcription factors. In addition, IKK represses extrinsic cell death pathways dependent on receptor-interacting serine/threonine-protein kinase 1 (RIPK1) by directly phosphorylating this kinase. Here, we showed that peripheral naïve T cells in mice required the continued expression of IKK1 and IKK2 for their survival; however, the loss of these cells was only partially prevented when extrinsic cell death pathways were blocked by either deleting Casp8 (which encodes the apoptosis-inducing caspase 8) or inhibiting the kinase activity of RIPK1. Inducible deletion of Rela (which encodes the NF-κB p65 subunit) in mature CD4+ T cells also resulted in loss of naïve CD4+ T cells and in reduced abundance of the interleukin-7 receptor (IL-7R) encoded by the NF-κB target Il7r, revealing an additional reliance upon NF-κB for the long-term survival of mature T cells. Together, these data indicate that the IKK-dependent survival of naïve CD4+ T cells depends on both repression of extrinsic cell death pathways and activation of an NF-κB–dependent survival program. Description Naïve mouse T cell homeostasis requires both IKK-mediated inhibition of death and activation of survival pathways. Editor’s summary Mouse thymocyte survival depends on the suppression of cell death by the inhibitor of κB (IKK) complex, which phosphorylates and inactivates the cell death–promoting kinase RIPK1. Using lineage-specific, conditional knockout mice, Carty et al. investigated the role of IKK in the survival of mature naïve T cells after they leave the thymus. In addition to inhibiting RIPK1-dependent cell death, IKK also activated NF-κB–dependent, pro-survival signaling, which depended on the cytokine receptor IL-7R. Together, these data indicate how cell death and NF-κB signaling pathways are modulated during cellular differentiation. –John F. Foley
Characteristic features of statistical models and machine learning methods derived from pest and disease monitoring datasets

While many studies have used traditional statistical methods when analysing monitoring data to predict future population dynamics of crop pests and diseases, increasing studies have used machine learning methods. The characteristic features of these methods have not been fully elucidated and arranged. We compared the prediction performance between two statistical and seven machine learning methods using 203 monitoring datasets recorded over several decades on four major crops in Japan and meteorological and geographical information as the explanatory variables. The decision tree and random forest of machine learning were found to be most efficient, while regression models of statistical and machine learning methods were relatively inferior. The best two methods were better for biased and scarce data, while the statistical Bayesian model was better for larger dataset sizes. Therefore, researchers should consider data characteristics when selecting the most appropriate method.
How to be a good partner and father? The role of adult males in pair bond maintenance and parental care in Javan gibbons

In pair-living species, female and male pairs may maintain stable social bonds by adjusting spatial and social associations. Nevertheless, each sex invests differently to maintain the pair bond, and the investment can depend on the presence of paternal care or ‘male services. ’ While most species live in pairs, the sex responsible for pair bond maintenance in gibbons is still controversial. We investigated pair bond maintenance and parental care in three pairs of wild Javan gibbons in Gunung Halimun-Salak National Park, Indonesia, for over 21 months. We found that Javan gibbon fathers groomed their offspring more than adult females, especially as offspring got older. While both parents increased playing time with offspring when offspring became older and more independent, fathers played with offspring 20 times more than mothers on average. Grooming within Javan gibbon pairs was male-biased, suggesting that pair bond maintenance was heavily the job of males. However, offspring age as a proxy for paternal care did not affect the pair bond maintenance. Our study highlights that adult male Javan gibbons may have an important role in pair bond maintenance and the care of juveniles.
Controlling continuous bioreactor via nonlinear feedback: modelling and simulations approach

The aim of this work is to present a class of nonlinear controller with an exponential-type feedback in order to regulate the sulfate mass concentration via the input ﬂow in a continuous stirred tank bioreactor of an anaerobic sulfate-reducing process. The corresponding kinetic terms in the bioreactor’s modeling are modeled by unstructured modeling approach, which was experimentally corroborated. A sketch of proof of the closed-loop stability of the considered system is done under the framework of Lyapunov theory. Numerical experiments are conducted to show the performance of the proposed methodology in comparison with a well-tuned sigmoid controller.
Adaptive assistive robotics: a framework for triadic collaboration between humans and robots

Robots and other assistive technologies have a huge potential to help society in domains ranging from factory work to healthcare. However, safe and effective control of robotic agents in these environments is complex, especially when it involves close interactions and multiple actors. We propose an effective framework for optimizing the behaviour of robots and complementary assistive technologies in systems comprising a mix of human and technological agents with numerous high-level goals. The framework uses a combination of detailed biomechanical modelling and weighted multi-objective optimization to allow for the fine tuning of robot behaviours depending on the specification of the task at hand. We illustrate our framework via two case studies across assisted living and rehabilitation scenarios, and conduct simulations and experiments of triadic collaboration in practice. Our results indicate a marked benefit to the triadic approach, showing the potential to improve outcome measures for human agents in robot-assisted tasks.
Reduced-order variational mode decomposition to reveal transient and non-stationary dynamics in fluid flows

Abstract A novel data-driven modal analysis method, reduced-order variational mode decomposition (RVMD), is proposed, inspired by the Hilbert–Huang transform and variational mode decomposition (VMD), to resolve transient or statistically non-stationary flow dynamics. First, the form of RVMD modes (referred to as an ‘elementary low-order dynamic process’, ELD) is constructed by combining low-order representation and the idea of intrinsic mode function, which enables the computed modes to characterize the non-stationary properties of space–time fluid flows. Then, the RVMD algorithm is designed based on VMD to achieve a low-redundant adaptive extraction of ELDs in flow data, with the modes computed by solving an elaborate optimization problem. Further, a combination of RVMD and Hilbert spectral analysis leads to a modal-based time-frequency analysis framework in the Hilbert view, providing a potentially powerful tool to discover, quantify and analyse the transient and non-stationary dynamics in complex flow problems. To provide a comprehensive evaluation, the computational cost and parameter dependence of RVMD are discussed, as well as the relations between RVMD and some classic modal decomposition methods. Finally, the virtues and utility of RVMD and the modal-based time-frequency analysis framework are well demonstrated via two canonical problems: the transient cylinder wake and the planar supersonic screeching jet.
A network for acute bilirubin encephalopathy classification based upon attention mechanism and 3D convolution kernels

Neonatal hyperbilirubinemia is a disease of bilirubin metabolism disorder, which is a common in newborns. Without timely medical attention, neonatal hyperbilirubinemia may develop into acute bilirubin encephalopathy, resulting in serious long-term neurological deficits. Magnetic resonance imaging, as a non-invasive imaging technique, is widely used in the diagnosis of acute bilirubin encephalopathy in newborns. However, the T1-weighted images of magnetic resonance imaging of newborns with normal myelin development and newborns with acute bilirubin encephalopathy have similar high signal intensity, making it difficult to make a clinical diagnosis based on the conventional radiological reading. As an important computer-aided diagnosis method, deep convolutional neural network has been widely used to improve the work efficiency of radiologists. In this paper, a convolutional neural network based on classification network for acute bilirubin encephalopathy is proposed. It contains a feature fusion section and a fairly deep Resnet classification network. Experimental results show that the performance of the proposal is better than those of deep learning models in discussion.
Turbo code optimization based on terahertz communication

In order to achieve reliable terahertz communication at ultra-high data rates, this paper proposes Turbo coding with S-random interleaver combining circular propagation constraint and circular shift, firstly, the overall structure is optimized mainly by using parallel loops for multi-core computation and reserving memory in advance, secondly, the coding structure is optimized, then S-random interleaver with circular propagation constraint and circular shift is used, and finally, the decoding The structure is optimized. Through simulation experiments, it can be concluded that the proposed Turbo coding method improves the coding operation rate and reduces the BER and transmission delay.
Medical image fusion based on multi-scale transform and sparse representation

Multi-modality image fusion is an effective technique to fuse the complementary information from multi-modality images into a fused image. In this paper, a new fusion method is proposed based on multi-scale transform and sparse representation. In the sparse representation-based method, the key point of it is to construct a good dictionary. In this method, a set of adaptive sub-dictionaries are constructed instead of a single dictionary. First, the source images are decomposed by nonsubsampled contourlet transform (NSCT) to obtain the low-pass coefficients and high-pass coefficients. Second, the sparse representation-based method is applied to fuse the low-pass coefficients. In this processing, low-pass images are divided into classified patches which are based on the patch oriented gradient. A set of sub-dictionaries are constructed from the classified patches. For the high-pass coefficients, we choose the "max-absolute" rule to fuse them. Third, combining the fused low-pass coefficients and the fused high-pass coefficients we can obtain the fused coefficients. In the last, the fused image is reconstructed by using inverse NSCT. Experimental results have shown the effectiveness of the proposed method.
Medical waste detection base on improved YOLOv5-s

Aiming at the characteristics of high detection accuracy and fast detection speed in the task of medical waste detection, this paper proposes an improved YOLOv5-s object detection model. Firstly, the Convolutional Block Attention Module (CBAM) is added to the original YOLOv5-s model backbone network to enhance the attention of the network model to medical waste; then the CIoU loss is used to replace the IoU loss to enhance the positioning accuracy and accelerate the convergence speed of the network model. This paper uses a variety of data enhancement methods to expand the experimental data set, a large number of experiments show that the mean average precision of this improved model reaches 90.73%, compared to the original YOLOv5-s model improved by 2.66%, detection speed reaches 115.7frame/s, the detection effect is better than the current mainstream object detection model.
A network deception defense mechanism based on virtual topology generation

To address the problem that traditional network defense techniques are difficult to cope with scanning attacks launched by internal attackers, this paper proposes a Virtual Network Topology Deception Defense mechanism(VNTDD)based on the idea of deception defense. In order to solve the problem of lack of randomness and security of virtual network topology, we divide the virtual network topology into three categories: forwarding nodes, real nodes and virtual nodes, and analyze their deployment locations and numbers respectively to generate a random virtual network topology, and enables the deployment of virtual network topology on the underlying real network through traffic control mechanisms. Finally, through experimental analysis, the VNTDD deception defense mechanism proposed in this paper can effectively prolong the scanning process of the intranet by internal attackers.
End-to-end mesh reconstruction from partial point cloud based on continuous implicit function

The existing mesh reconstruction methods rely on the complete point cloud input. However, the point cloud collected in the actual environment is always the partial point cloud of an object/scene. The direct mesh reconstruction from the partial point cloud is a challenging problem but there is currently no end-to-end method for mesh reconstruction from partial point clouds. To solve this challenge, this paper proposes an end-to-end mesh reconstruction from the partial point cloud method based on continuous implicit function. Specifically, the continuous implicit function of a complete point cloud is learned by combining the local and global features of a partial point cloud. Then we sample a point cloud from the continuous implicit function and reconstruct the mesh by leveraging the deformation network. A loss function based on the point cloud normal vector is proposed to further optimize mesh reconstruction. Experiments on the ShapeNet-55 dataset show that the loss of our method in three different incomplete degrees of the point cloud is reduced by 35%, 38%, and 38% respectively compared with NMF, with an average reduction of 38%.