Miracle-dz/UniFAR-FTUs huggingface dataset:
V2Meow: Meowing to the Visual Beat via Video-to-Music Generation

Video-to-music generation demands both a temporally localized high-quality listening experience and globally aligned video-acoustic signatures. While recent music generation models excel at the former through advanced audio codecs, the exploration of video-acoustic signatures has been confined to specific visual scenarios. In contrast, our research confronts the challenge of learning globally aligned signatures between video and music directly from paired music and videos, without explicitly modeling domain-specific rhythmic or semantic relationships. We propose V2Meow, a video-to-music generation system capable of producing high-quality music audio for a diverse range of video input types using a multi-stage autoregressive model. Trained on 5k hours of music audio clips paired with video frames mined from in-the-wild music videos, V2Meow is competitive with previous domain-specific models when evaluated in a zero-shot manner. It synthesizes high-fidelity music audio waveforms solely by conditioning on pre-trained general-purpose visual features extracted from video frames, with optional style control via text prompts. Through both qualitative and quantitative evaluations, we demonstrate that our model outperforms various existing music generation systems in terms of visual-audio correspondence and audio quality. Music samples are available at tinyurl.com/v2meow.
Meta-Learners for Few-Shot Weakly-Supervised Medical Image Segmentation

Most uses of Meta-Learning in visual recognition are very often applied to image classification, with a relative lack of works in other tasks {such} as segmentation and detection. We propose a generic Meta-Learning framework for few-shot weakly-supervised segmentation in medical imaging domains. We conduct a comparative analysis of meta-learners from distinct paradigms adapted to few-shot image segmentation in different sparsely annotated radiological tasks. The imaging modalities include 2D chest, mammographic and dental X-rays, as well as 2D slices of volumetric tomography and resonance images. Our experiments consider a total of 9 meta-learners, 4 backbones and multiple target organ segmentation tasks. We explore small-data scenarios in radiology with varying weak annotation styles and densities. Our analysis shows that metric-based meta-learning approaches achieve better segmentation results in tasks with smaller domain shifts in comparison to the meta-training datasets, while some gradient- and fusion-based meta-learners are more generalizable to larger domain shifts.
Biophysical Model for Signal-Embedded Droplet Soaking into 2D Cell Culture

Using agar plates hosting a 2D cell population stimulated with signaling molecules is crucial for experiments such as gene regulation and drug discovery in a wide range of biological studies. In this paper, a biophysical model is proposed that incorporates droplet soaking, diffusion of molecules within agar, cell growth over an agar surface, and absorption of signaling molecules by cells. The proposed model describes the channel response and provides valuable insights for designing experiments more efficiently and accurately. The molecule release rate due to droplet soaking into agar, which is characterized and modeled as the source term for the diffusion model, is derived. Furthermore, cell growth is considered over the surface, which dictates the dynamics of signaling molecule reactions and leads to a variable boundary condition. As a case study, genetically-modified E. coli bacteria are spread over the surface of agar and Isopropyl-beta-D thiogalactopyranoside (IPTG) is considered as a signaling molecule. IPTG droplets are dropped onto the bacteria-covered agar surface. The parameters for the IPTG molecule release rate as a diffusion source into the agar are estimated from this experiment. Then, a particle-based simulator is used to obtain the spatio-temporal profile of the signaling molecules received by the surface bacteria. The results indicate that the number of molecules reacting with or absorbed by bacteria at different locations on the surface could be widely different, which highlights the importance of taking this variation into account for biological inferences.
What are neutron stars made of? Gravitational waves may reveal the answer

Neutron stars are one of the most mysterious wonders in the Universe. Their extreme densities hint at new and exotic physics at work within. Gravitational waves could be the key to unlocking their secrets. In particular, a first detection of gravitational waves from rapidly-spinning, deformed neutron stars could yield new insights into the physics of matter at extreme densities and under strong gravity. Once a first detection is made, a critical challenge will be to robustly extract physically interesting information from the detected signals. In this essay, we describe initial research towards answering this challenge, and thereby unleashing the full power of gravitational waves as an engine for the discovery of new physics.
A Generic Approach to Integrating Time into Spatial-Temporal Forecasting via Conditional Neural Fields

Self-awareness is the key capability of autonomous systems, e.g., autonomous driving network, which relies on highly efficient time series forecasting algorithm to enable the system to reason about the future state of the environment, as well as its effect on the system behavior as time progresses. Recently, a large number of forecasting algorithms using either convolutional neural networks or graph neural networks have been developed to exploit the complex temporal and spatial dependencies present in the time series. While these solutions have shown significant advantages over statistical approaches, one open question is to effectively incorporate the global information which represents the seasonality patterns via the time component of time series into the forecasting models to improve their accuracy. This paper presents a general approach to integrating the time component into forecasting models. The main idea is to employ conditional neural fields to represent the auxiliary features extracted from the time component to obtain the global information, which will be effectively combined with the local information extracted from autoregressive neural networks through a layer-wise gated fusion module. Extensive experiments on road traffic and cellular network traffic datasets prove the effectiveness of the proposed approach.
Reverse Ordering Techniques for Attention-Based Channel Prediction

Channel state information (CSI) is crucial for enhancing the performance of wireless systems by allowing to adjust the transmission strategies based on the current channel conditions. However, obtaining precise CSI is difficult because of the fast-changing channel conditions caused by multi-path fading. An inaccurate CSI hinders the performance of various adaptive wireless systems, highlighting the need for channel prediction techniques to effectively mitigate the drawbacks of outdated CSI. Conventional methods typically depend on assumptions regarding user velocity or require knowledge of the Doppler frequency. In contrast to existing approaches, we aim for a more robust and practical solution by training neural networks without making any assumptions about user velocity, relying solely on noisy channel observations during training. Specifically, we adapt both the sequence-to-sequence with attention (Seq2Seq-attn) and transformer models for channel prediction. Additionally, a new technique called reverse positional encoding is introduced in the transformer model to improve the robustness of the model against varying sequence lengths. Similarly, the encoder outputs of the Seq2Seq-attn model are reversed prior to the application of attention mechanisms. By means of simulations, we show that these proposed techniques enable the models to effectively capture relationships within sequences of channel snapshots without increasing the complexity. Importantly, this capability remains robust across varying sequence lengths, representing a substantial improvement over existing methodologies.
A semiderivative approach for shape optimization problems constrained by variational inequalities

Shape optimization problems constrained by variational inequalities (VI) are non-smooth and non-convex optimization problems. The non-smoothness arises due to the variational inequality constraint, which makes it challenging to derive optimality conditions. Besides the non-smoothness there are complementary aspects due to the VIs as well as distributed, non-linear, non-convex and infinite-dimensional aspects due to the shapes which complicate to set up an optimality system and, thus, to develop efficient solution algorithms. In this paper, we consider the Hadamard semiderivative in order to formulate optimality conditions. In this context, we set up a Hadamard shape semiderivative approach and demonstrate its advantages over an proposed approach based on regulatizations.
When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks

Though majority vote among annotators is typically used for ground truth labels in natural language processing, annotator disagreement in tasks such as hate speech detection may reflect differences in opinion across groups, not noise. Thus, a crucial problem in hate speech detection is determining whether a statement is offensive to the demographic group that it targets, when that group may constitute a small fraction of the annotator pool. We construct a model that predicts individual annotator ratings on potentially offensive text and combines this information with the predicted target group of the text to model the opinions of target group members. We show gains across a range of metrics, including raising performance over the baseline by 22% at predicting individual annotators' ratings and by 33% at predicting variance among annotators, which provides a metric for model uncertainty downstream. We find that annotator ratings can be predicted using their demographic information and opinions on online content, without the need to track identifying annotator IDs that link each annotator to their ratings. We also find that use of non-invasive survey questions on annotators' online experiences helps to maximize privacy and minimize unnecessary collection of demographic information when predicting annotators' opinions.
Word Grounded Graph Convolutional Network

Graph Convolutional Networks (GCNs) have shown strong performance in learning text representations for various tasks such as text classification, due to its expressive power in modeling graph structure data (e.g., a literature citation network). Most existing GCNs are limited to deal with documents included in a pre-defined graph, i.e., it cannot be generalized to out-of-graph documents. To address this issue, we propose to transform the document graph into a word graph, to decouple data samples (i.e., documents in training and test sets) and a GCN model by using a document-independent graph. Such word-level GCN could therefore naturally inference out-of-graph documents in an inductive way. The proposed Word-level Graph (WGraph) can not only implicitly learning word presentation with commonly-used word co-occurrences in corpora, but also incorporate extra global semantic dependency derived from inter-document relationships (e.g., literature citations). An inductive Word-grounded Graph Convolutional Network (WGCN) is proposed to learn word and document representations based on WGraph in a supervised manner. Experiments on text classification with and without citation networks evidence that the proposed WGCN model outperforms existing methods in terms of effectiveness and efficiency.
Surface Simplification using Intrinsic Error Metrics

This paper describes a method for fast simplification of surface meshes. Whereas past methods focus on visual appearance, our goal is to solve equations on the surface. Hence, rather than approximate the extrinsic geometry, we construct a coarse intrinsic triangulation of the input domain. In the spirit of the quadric error metric (QEM), we perform greedy decimation while agglomerating global information about approximation error. In lieu of extrinsic quadrics, however, we store intrinsic tangent vectors that track how far curvature "drifts" during simplification. This process also yields a bijective map between the fine and coarse mesh, and prolongation operators for both scalar- and vector-valued data. Moreover, we obtain hard guarantees on element quality via intrinsic retriangulation---a feature unique to the intrinsic setting. The overall payoff is a "black box" approach to geometry processing, which decouples mesh resolution from the size of matrices used to solve equations. We show how our method benefits several fundamental tasks, including geometric multigrid, all-pairs geodesic distance, mean curvature flow, geodesic Voronoi diagrams, and the discrete exponential map.
Halo Mass-observable Proxy Scaling Relations and Their Dependencies on Galaxy and Group Properties

Based on the DECaLS shear catalog, we study the scaling relations between halo mass (M h) and various proxies for Sloan Digital Sky Survey central galaxies, including stellar mass (M *), stellar velocity dispersion (σ *), abundance-matching halo mass (M AM), and satellite velocity dispersion (σ s), and their dependencies on galaxy and group properties. In general, these proxies all have strong positive correlations with M h, consistent with previous studies. We find that the M h–M * and M h–σ * relations depend strongly on group richness (N sat), while the M h–M AM and M h–σ s relations are independent of it. Moreover, the dependence on the star formation rate (SFR) is rather weak in the M h–σ * and M h–σ s relations, but very prominent in the other two. σ s is thus the best proxy among them, and its scaling relation is in good agreement with hydrodynamical simulations. However, estimating σ s accurately for individual groups/clusters is challenging because of interlopers and the requirement for sufficient satellites. We construct new proxies by combining M *, σ *, and M AM, and find that the proxy with 30% contribution from M AM and 70% from σ * can minimize the dependence on N sat and the SFR. We obtain the M h–supermassive black hole (SMBH) mass relation via the SMBH scaling relation and find indications for rapid and linear growth phases for the SMBH. We also find that correlations among M h, M *, and σ * change with M *, indicating that different processes drive the growth of galaxies and SMBHs at different stages.
REMaQE - Reverse Engineering Math Equations from Executables

Cybersecurity attacks against industrial control systems and cyber-physical systems can cause catastrophic real-world damage by infecting device binaries with malware. Mitigating such attacks can benefit from reverse engineering tools that recover sufficient semantic knowledge in terms of mathematical operations in the code. Conventional reverse engineering tools can decompile binaries to low-level code, but offer little semantic insight. This paper proposes REMaQE, an automated framework for reverse engineering of math equations from binary executables. REMaQE uses symbolic execution for dynamic analysis of the binary to extract the relevant semantic knowledge of the implemented algorithms. REMaQE provides an automatic parameter analysis pass which also leverages symbolic execution to identify input, output, and constant parameters of the implemented math equations. REMaQE automatically handles parameters accessed via registers, the stack, global memory, or pointers, and supports reverse engineering of object-oriented implementations such as C++ classes. REMaQE uses an algebraic simplification method which allows it to scale to complex conditional equations with ease. These features make REMaQE stand out over existing reverse engineering approaches for math equations. On a dataset of randomly generated math equations compiled to binaries from C and Simulink implementations, REMaQE accurately recovers a semantically matching equation for 97.53% of the models. For complex equations with more operations, accuracy stays consistently over 94%. REMaQE executes in 0.25 seconds on average and in 1.3 seconds for more complex equations. This real-time execution speed enables a smooth integration in an interactive mathematics-oriented reverse engineering workflow.
Demand Privacy in Hotplug Caching Systems

Coded caching, introduced by Maddah-Ali and Niesen (MAN), is a model where a server broadcasts multicast packets to users with a local cache that is leveraged so as to reduce the peak network communication load. The original MAN model does not consider missing demands (i.e., some users may not request a file) or privacy issues (i.e., decoding the multicast packets may expose the users’ demands). The former issue was captured by the hotplug model with offline users, where the server starts sending multicast packets after having received a certain number of file requests. The latter issue was addressed by devoting part of the cache to store privacy keys to help users decode their requested file while remaining completely ignorant about the demands of the remaining users. This paper investigates the problem of private demands against colluding users in the hotplug model with offline users. Two achievable schemes are proposed based on Maximum Distance Separable (MDS) codes. They achieve lower subpacketization, and lower load in the small memory regime compared to baseline schemes that trivially include demand privacy or offline users in known schemes.
Detection and Classification of Pole-Like Landmarks for Domain-Invariant 3D Point Cloud Map Matching

Pole landmarks have a lot of potential as landmarks for accurate and reliable visual self-localization in 3D point cloud-based visual self-localization due to their long-term stability under seasonal and weather changes. In the context of pole landmark-based self-localization, the goal of this study is to investigate the application of recently developed deep learning models for pole classification. In particular, there are two main modules in the proposed scheme: pole class matching and matching of pole maps. In the previous module, a local pole map is created and its configuration is compared to a global pole map that has already been computed. To strike a balance between accuracy and efficiency in computation, an effective RANSAC map matching is used. In the last option post class matching module, the nearby and worldwide shafts matched by the RANSAC map-matching are additionally thought about through shaft trait class. Using self-supervised k-means clustering, a predefined set of pseudo pole classes is learned for this purpose. Tests utilizing openly accessible NCLT dataset showed that the post like milestone arrangement strategy improvedly affects the visual self-confinement framework contrasted and the gauge technique.
Cost-efficient Crowdsourcing for Span-based Sequence Labeling: Worker Selection and Data Augmentation

This paper introduces a novel worker selection algorithm, enhancing annotation quality and reducing costs in challenging span-based sequence labeling tasks in Natural Language Processing (NLP). Unlike previous studies targeting simpler tasks, this study contends with the complexities of label interdependencies in sequence labeling tasks. The proposed algorithm utilizes a Combinatorial Multi-Armed Bandit (CMAB) approach for worker selection. The challenge of dealing with imbalanced and small-scale datasets, which hinders offline simulation of worker selection, is tackled using an innovative data augmentation method termed shifting, expanding, and shrinking (SES). The SES method is designed specifically for sequence labeling tasks. Rigorous testing on CoNLL 2003 NER and Chinese OEI datasets showcased the algorithm's efficiency, with an increase in F1 score up to 100.04% of the expert-only baseline, alongside cost savings up to 65.97%. The paper also encompasses a dataset-independent test emulating annotation evaluation through a Bernoulli distribution, which still led to an impressive 97.56% F1 score of the expert baseline and 59.88% cost savings. This research addresses and overcomes numerous obstacles in worker selection for complex NLP tasks.
Convergence of Alternating Gradient Descent for Matrix Factorization

We consider alternating gradient descent (AGD) with fixed step size applied to the asymmetric matrix factorization objective. We show that, for a rank-$r$ matrix $\mathbf{A} \in \mathbb{R}^{m \times n}$, $T = C (\frac{\sigma_1(\mathbf{A})}{\sigma_r(\mathbf{A})})^2 \log(1/\epsilon)$ iterations of alternating gradient descent suffice to reach an $\epsilon$-optimal factorization $\| \mathbf{A} - \mathbf{X} \mathbf{Y}^{T} \|^2 \leq \epsilon \| \mathbf{A}\|^2$ with high probability starting from an atypical random initialization. The factors have rank $d \geq r$ so that $\mathbf{X}_{T}\in\mathbb{R}^{m \times d}$ and $\mathbf{Y}_{T} \in\mathbb{R}^{n \times d}$, and mild overparameterization suffices for the constant $C$ in the iteration complexity $T$ to be an absolute constant. Experiments suggest that our proposed initialization is not merely of theoretical benefit, but rather significantly improves the convergence rate of gradient descent in practice. Our proof is conceptually simple: a uniform Polyak-\L{}ojasiewicz (PL) inequality and uniform Lipschitz smoothness constant are guaranteed for a sufficient number of iterations, starting from our random initialization. Our proof method should be useful for extending and simplifying convergence analyses for a broader class of nonconvex low-rank factorization problems.
HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level

Link Prediction on Hyper-relational Knowledge Graphs (HKG) is a worthwhile endeavor. HKG consists of hyper-relational facts (H-Facts), composed of a main triple and several auxiliary attribute-value qualifiers, which can effectively represent factually comprehensive information. The internal structure of HKG can be represented as a hypergraph-based representation globally and a semantic sequence-based representation locally. However, existing research seldom simultaneously models the graphical and sequential structure of HKGs, limiting HKGs’ representation. To overcome this limitation, we propose a novel Hierarchical Attention model for HKG Embedding (HAHE), including global-level and local-level attention. The global-level attention can model the graphical structure of HKG using hypergraph dual-attention layers, while the local-level attention can learn the sequential structure inside H-Facts via heterogeneous self-attention layers. Experiment results indicate that HAHE achieves state-of-the-art performance in link prediction tasks on HKG standard datasets. In addition, HAHE addresses the issue of HKG multi-position prediction for the first time, increasing the applicability of the HKG link prediction task. Our code is publicly available.