Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimating time of arrival of vehicle fleets with GCN based traffic prediction

This paper presents an effective framework for estimating time of arrival of vehicles (buses) in an Intelligent Transit Management System (ITMS) having sparse position updates. Our contributions towards this is firstly in implementing a constrained optimization based road linestring segmenting framework ensuring ideal segment lengths and segments with sufficient density of vehicle position measurements which will result in valid statistics for scenarios involving sparse position measurements. Over this we propose a comprehensive approach for predicting traffic delays and estimated time of vehicle arrival addressing both the spatial and temporal dependencies of traffic. The traffic delay model is built on top of the T-GCN architecture on which we optimally augment an adjacency matrix which models a complexly connected road network considering the degree of influence between road segments, enabling the traffic delay model to look beyond physical road connectivity in predicting traffic delays and therefore producing better estimates of arrival times to points along the designated route of the vehicles.
Schwarzschild black hole surrounded by a cavity and phase transition in the non-commutative gauge theory of gravity

In this work, we investigate the phase transition of the Schwarzschild black hole (SBH) inside an isothermal spherical cavity in the context of the non-commutative (NC) gauge theory of gravity, by using the Seiberg-Witten (SW) map and the star product. Firstly, we compute the NC correction to the Hawking temperature and derive the logarithmic correction to the entropy, then we derive the local temperature and local energy of NC SBH in isothermal cavity. Our results show that the non-commutativity removes the commutative divergence behavior of temperature, and prevents the SBH from the complete evaporation, which leads to a remnant black hole, and this geometry has predicted a minimal length in the order of Planck scale $\Theta\sim l_{planck}$. Therefore, the thermodynamic stability and phase transition is studied by analyzing the behavior of the local heat capacity and the Helmholtz free energy in the NC spacetime, where the results show that, the NC SBH has a two second-order phase transition and one first-order phase transition, with two Hawking-Page phase transition in the NC gauge theory.
AudioLog: LLMs-Powered Long Audio Logging with Hybrid Token-Semantic Contrastive Learning

Previous studies in automated audio captioning have faced difficulties in accurately capturing the complete temporal details of acoustic scenes and events within long audio sequences. This paper presents AudioLog, a large language models (LLMs)-powered audio logging system with hybrid token-semantic contrastive learning. Specifically, we propose to fine-tune the pre-trained hierarchical token-semantic audio Transformer by incorporating contrastive learning between hybrid acoustic representations. We then leverage LLMs to generate audio logs that summarize textual descriptions of the acoustic environment. Finally, we evaluate the AudioLog system on two datasets with both scene and event annotations. Experiments show that the proposed system achieves exceptional performance in acoustic scene classification and sound event detection, surpassing existing methods in the field. Further analysis of the prompts to LLMs demonstrates that AudioLog can effectively summarize long audio sequences. To the best of our knowledge, this approach is the first attempt to leverage LLMs for summarizing long audio sequences.
Eliminating Quantization Errors in Classification-Based Sound Source Localization

Sound Source Localization (SSL) involves estimating the Direction of Arrival (DOA) of sound sources. Since the DOA estimation output space is continuous, regression might be more suitable for DOA, offering higher precision. However, in practice, classification often outperforms regression, exhibiting greater robustness to interference. Conversely, classification's drawback is inherent quantization error. Within the classification paradigm, the DOA output space is discretized into intervals, each treated as a class. These classes show strong inter-class correlations, being inherently ordered, with higher similarity as intervals grow closer. Nevertheless, this has not been fully exploited. To address this, we propose an Unbiased Label Distribution (ULD) to eliminate quantization error in training targets. Furthermore, we tailor two loss functions for the soft label family: Negative Log Absolute Error (NLAE) and Mean Squared Error without activation (MSE(wo)). Finally, we introduce Weighted Adjacent Decoding (WAD) to overcome quantization error during model prediction decoding. Experimental results demonstrate our approach surpasses classification quantization limits, achieving state-of-the-art performance. Our code and supplementary materials are available at https://github.com/linfeng-feng/ULD.
Quantifying Impairment and Disease Severity Using AI Models Trained on Healthy Subjects

Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a novel framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment ($\rho = 0.644$, 95% C.I [0.585,0.696]).
RFTrans: Leveraging Refractive Flow of Transparent Objects for Surface Normal Estimation and Manipulation

Transparent objects are widely used in our daily lives, making it important to teach robots to interact with them. However, it's not easy because the reflective and refractive effects can make depth cameras fail to give accurate geometry measurements. To solve this problem, this paper introduces RFTrans, an RGB-D-based method for surface normal estimation and manipulation of transparent objects. By leveraging refractive flow as an intermediate representation, the proposed method circumvents the drawbacks of directly predicting the geometry (e.g. surface normal) from images and helps bridge the sim-to-real gap. It integrates the RFNet, which predicts refractive flow, object mask, and boundaries, followed by the F2Net, which estimates surface normal from the refractive flow. To make manipulation possible, a global optimization module will take in the predictions, refine the raw depth, and construct the point cloud with normal. An off-the-shelf analytical grasp planning algorithm is followed to generate the grasp poses. We build a synthetic dataset with physically plausible ray-tracing rendering techniques to train the networks. Results show that the proposed method trained on the synthetic dataset can consistently outperform the baseline method in both synthetic and real-world benchmarks by a large margin. Finally, a real-world robot grasping task witnesses an 83% success rate, proving that refractive flow can help enable direct sim-to-real transfer.
BundleMoCap: Efficient, Robust and Smooth Motion Capture from Sparse Multiview Videos

Capturing smooth motions from videos using markerless techniques typically involves complex processes such as temporal constraints, multiple stages with data-driven regression and optimization, and bundle solving over temporal windows. These processes can be inefficient and require tuning multiple objectives across stages. In contrast, BundleMoCap introduces a novel and efficient approach to this problem. It solves the motion capture task in a single stage, eliminating the need for temporal smoothness objectives while still delivering smooth motions. BundleMoCap outperforms the state-of-the-art without increasing complexity. The key concept behind BundleMoCap is manifold interpolation between latent keyframes. By relying on a local manifold smoothness assumption, we can efficiently solve a bundle of frames using a single code. Additionally, the method can be implemented as a sliding window optimization and requires only the first frame to be properly initialized, reducing the overall computational burden. BundleMoCap’s strength lies in its ability to achieve high-quality motion capture results with simplicity and efficiency.
Pursing the Sparse Limitation of Spiking Deep Learning Structures

Spiking Neural Networks (SNNs), a novel brain-inspired algorithm, are garnering increased attention for their superior computation and energy efficiency over traditional artificial neural networks (ANNs). To facilitate deployment on memory-constrained devices, numerous studies have explored SNN pruning. However, these efforts are hindered by challenges such as scalability challenges in more complex architectures and accuracy degradation. Amidst these challenges, the Lottery Ticket Hypothesis (LTH) emerges as a promising pruning strategy. It posits that within dense neural networks, there exist winning tickets or subnetworks that are sparser but do not compromise performance. To explore a more structure-sparse and energy-saving model, we investigate the unique synergy of SNNs with LTH and design two novel spiking winning tickets to push the boundaries of sparsity within SNNs. Furthermore, we introduce an innovative algorithm capable of simultaneously identifying both weight and patch-level winning tickets, enabling the achievement of sparser structures without compromising on the final model's performance. Through comprehensive experiments on both RGB-based and event-based datasets, we demonstrate that our spiking lottery ticket achieves comparable or superior performance even when the model structure is extremely sparse.
CSMeD: Bridging the Dataset Gap in Automated Citation Screening for Systematic Literature Reviews

Systematic literature reviews (SLRs) play an essential role in summarising, synthesising and validating scientific evidence. In recent years, there has been a growing interest in using machine learning techniques to automate the identification of relevant studies for SLRs. However, the lack of standardised evaluation datasets makes comparing the performance of such automated literature screening systems difficult. In this paper, we analyse the citation screening evaluation datasets, revealing that many of the available datasets are either too small, suffer from data leakage or have limited applicability to systems treating automated literature screening as a classification task, as opposed to, for example, a retrieval or question-answering task. To address these challenges, we introduce CSMeD, a meta-dataset consolidating nine publicly released collections, providing unified access to 325 SLRs from the fields of medicine and computer science. CSMeD serves as a comprehensive resource for training and evaluating the performance of automated citation screening models. Additionally, we introduce CSMeD-FT, a new dataset designed explicitly for evaluating the full text publication screening task. To demonstrate the utility of CSMeD, we conduct experiments and establish baselines on new datasets.
On $q$-Order Statistics

Building on the notion of $q$-integral introduced by Thomae in 1869, we introduce $q$-order statistics (that, is $q$-analogues of the classical order statistics, for $0<q<1$) which arise from dependent and not identically distributed $q$-continuous random variables and study their distributional properties. We study the $q$-distribution functions and the $q$-density functions of the relative $q$-ordered random variables. We focus on $q$-ordered variables arising from dependent and not identically $q$-uniformly distributed random variables and we derive their $q$-distributions, including $q$-power law, $q$-beta and $q$-Dirichlet distributions.
Context-Dependent Memory in Situated Visualization

Situated visualization presents data alongside their source context (physical referent). While environmental factors influence memory recall (known as Context-Dependent Memory or CDM), how physical context affects cognition in real-world tasks such as working with visualizations in situated contexts is unclear. This study explores the design space of information memorability in situated visualization through the lens of CDM. We investigate the presence of physical referents for creating contextual cues in desktop and Virtual Reality (VR) environments. Across three studies (n=144), we observe a trend suggesting a CDM effect due to contextual referent is more apparent in VR. Overall, we did not find statistically significant evidence of a CDM effect due to the presence of a referent. However, we did find a significant CDM effect for lighting conditions. This suggests that representing the entire environment, rather than the physical objects alone, may be necessary to provide sufficiently strong contextual memory cues.
Noise in Relation Classification Dataset TACRED: Characterization and Reduction

The overarching objective of this paper is two-fold. First, to explore model-based approaches to characterize the primary cause of the noise. in the RE dataset TACRED Second, to identify the potentially noisy instances. Towards the first objective, we analyze predictions and performance of state-of-the-art (SOTA) models to identify the root cause of noise in the dataset. Our analysis of TACRED shows that the majority of the noise in the dataset originates from the instances labeled as no-relation which are negative examples. For the second objective, we explore two nearest-neighbor-based strategies to automatically identify potentially noisy examples for elimination and reannotation. Our first strategy, referred to as Intrinsic Strategy (IS), is based on the assumption that positive examples are clean. Thus, we have used false-negative predictions to identify noisy negative examples. Whereas, our second approach, referred to as Extrinsic Strategy, is based on using a clean subset of the dataset to identify potentially noisy negative examples. Finally, we retrained the SOTA models on the eliminated and reannotated dataset. Our empirical results based on two SOTA models trained on TACRED-E following the IS show an average 4% F1-score improvement, whereas reannotation (TACRED-R) does not improve the original results. However, following ES, SOTA models show the average F1-score improvement of 3.8% and 4.4% when trained on respective eliminated (TACRED-EN) and reannotated (TACRED-RN) datasets respectively. We further extended the ES for cleaning positive examples as well, which resulted in an average performance improvement of 5.8% and 5.6% for the eliminated (TACRED-ENP) and reannotated (TACRED-RNP) datasets respectively.
nach0: Multimodal Natural and Chemical Languages Foundation Model

Large Language Models (LLMs) have substantially driven scientific progress in various domains, and many papers have demonstrated their ability to tackle complex problems with creative solutions. Our paper introduces a new foundation model, nach0, capable of solving various chemical and biological tasks: biomedical question answering, named entity recognition, molecular generation, molecular synthesis, attributes prediction, and others. nach0 is a multi-domain and multi-task encoder-decoder LLM pre-trained on unlabeled text from scientific literature, patents, and molecule strings to incorporate a range of chemical and linguistic knowledge. We employed instruction tuning, where specific task-related instructions are utilized to fine-tune nach0 for the final set of tasks. To train nach0 effectively, we leverage the NeMo framework, enabling efficient parallel optimization of both base and large model versions. Extensive experiments demonstrate that our model outperforms state-of-the-art baselines on single-domain and cross-domain tasks. Furthermore, it can generate high-quality outputs in molecular and textual formats, showcasing its effectiveness in multi-domain setups.
Hybrid III-V/Silicon photonic circuits embedding generation and routing of entangled photon pairs

The demand for integrated photonic chips combining the generation and manipulation of quantum states of light is steadily increasing, driven by the need for compact and scalable platforms for quantum information technologies. While photonic circuits with diverse functionalities are being developed in different single material platforms, it has become crucial to realize hybrid photonic circuits that harness the advantages of multiple materials while mitigating their respective weaknesses, resulting in enhanced capabilities. Here, we demonstrate a hybrid III-V/Silicon quantum photonic device combining the strong second-order nonlinearity and compliance with electrical pumping of the III-V semiconductor platform with the high maturity and CMOS compatibility of the silicon photonic platform. Our device embeds the spontaneous parametric down-conversion (SPDC) of photon pairs into an AlGaAs source and their subsequent routing to a silicon-on-insulator circuitry, within an evanescent coupling scheme managing both polarization states. This enables the on-chip generation of broadband telecom photons by type 0 and type 2 SPDC from the hybrid device, at room temperature and with internal pair generation rates exceeding $10^5$ $s^{-1}$ for both types, while the pump beam is strongly rejected. Two-photon interference with 92% visibility (and up to 99% upon 5 nm spectral filtering) proves the high energy-time entanglement quality characterizing the produced quantum state, thereby enabling a wide range of quantum information applications on-chip, within an hybrid architecture merging the assets of two mature and highly complementary platforms in view of out-of-the-lab deployment of quantum technologies.
Echocardiogram Foundation Model - Application 1: Estimating Ejection Fraction

Cardiovascular diseases stand as the primary global cause of mortality. Among the various imaging techniques available for visualising the heart and evaluating its function, echocardiograms emerge as the preferred choice due to their safety and low cost. Quantifying cardiac function based on echocardiograms is very laborious, time-consuming and subject to high interoperator variability. In this work, we introduce EchoAI, an echocardiogram foundation model, that is trained using self-supervised learning (SSL) on 1.5 million echocardiograms. We evaluate our approach by fine-tuning EchoAI to estimate the ejection fraction achieving a mean absolute percentage error of 9.40%. This level of accuracy aligns with the performance of expert sonographers.
Measure of Dependence for Financial Time-Series

Assessing the predictive power of both data and models holds paramount significance in time-series machine learning applications. Yet, preparing time series data accurately and employing an appropriate measure for predictive power seems to be a non-trivial task. This work involves reviewing and establishing the groundwork for a comprehensive analysis of shaping time-series data and evaluating various measures of dependence. Lastly, we present a method, framework, and a concrete example for selecting and evaluating a suitable measure of dependence.
Few-Shot Classification & Segmentation Using Large Language Models Agent

The task of few-shot image classification and segmentation (FS-CS) requires the classification and segmentation of target objects in a query image, given only a few examples of the target classes. We introduce a method that utilises large language models (LLM) as an agent to address the FS-CS problem in a training-free manner. By making the LLM the task planner and off-the-shelf vision models the tools, the proposed method is capable of classifying and segmenting target objects using only image-level labels. Specifically, chain-of-thought prompting and in-context learning guide the LLM to observe support images like human; vision models such as Segment Anything Model (SAM) and GPT-4Vision assist LLM understand spatial and semantic information at the same time. Ultimately, the LLM uses its summarizing and reasoning capabilities to classify and segment the query image. The proposed method's modular framework makes it easily extendable. Our approach achieves state-of-the-art performance on the Pascal-5i dataset.
How AI-driven Digital Twins Can Empower Mobile Networks

The growing complexity of next-generation networks exacerbates the modeling and algorithmic flaws of conventional network optimization methodology. In this paper, we propose a mobile network digital twin (MNDT) architecture for 6G networks. To address the modeling and algorithmic shortcomings, the MNDT uses a simulation-optimization structure. The feedback from the network simulation engine, which serves as validation for the optimizer's decision outcomes, is used explicitly to train artificial intelligence (AI) empowered optimizers iteratively. In practice, we develop a network digital twin prototype system leveraging data-driven technology to accurately model the behaviors of mobile network elements (e.g., mobile users and base stations), wireless environments, and network performance. An AI-powered network optimizer has been developed based on the deployed MNDT prototype system for providing reliable and optimized network configurations. The results of the experiments demonstrate that the proposed MNDT infrastructure can provide practical network optimization solutions while adapting to the more complex environment.
Neural Network Pruning by Gradient Descent

The rapid increase in the parameters of deep learning models has led to significant costs, challenging computational efficiency and model interpretability. In this paper, we introduce a novel and straightforward neural network pruning framework that incorporates the Gumbel-Softmax technique. This framework enables the simultaneous optimization of a network's weights and topology in an end-to-end process using stochastic gradient descent. Empirical results demonstrate its exceptional compression capability, maintaining high accuracy on the MNIST dataset with only 0.15\% of the original network parameters. Moreover, our framework enhances neural network interpretability, not only by allowing easy extraction of feature importance directly from the pruned network but also by enabling visualization of feature symmetry and the pathways of information propagation from features to outcomes. Although the pruning strategy is learned through deep learning, it is surprisingly intuitive and understandable, focusing on selecting key representative features and exploiting data patterns to achieve extreme sparse pruning. We believe our method opens a promising new avenue for deep learning pruning and the creation of interpretable machine learning systems.