Miracle-dz/UniFAR-FTUs huggingface dataset:
Multi-Modal Hypergraph Diffusion Network with Dual Prior for Alzheimer Classification

The automatic early diagnosis of prodromal stages of Alzheimer's disease is of great relevance for patient treatment to improve quality of life. We address this problem as a multi-modal classification task. Multi-modal data provides richer and complementary information. However, existing techniques only consider either lower order relations between the data and single/multi-modal imaging data. In this work, we introduce a novel semi-supervised hypergraph learning framework for Alzheimer's disease diagnosis. Our framework allows for higher-order relations among multi-modal imaging and non-imaging data whilst requiring a tiny labelled set. Firstly, we introduce a dual embedding strategy for constructing a robust hypergraph that preserves the data semantics. We achieve this by enforcing perturbation invariance at the image and graph levels using a contrastive based mechanism. Secondly, we present a dynamically adjusted hypergraph diffusion model, via a semi-explicit flow, to improve the predictive uncertainty. We demonstrate, through our experiments, that our framework is able to outperform current techniques for Alzheimer's disease diagnosis.
Comparison of Multi-Criteria Decision Support Methods for Integrated Rehabilitation Prioritization

The decisions taken in rehabilitation planning for the urban water networks will have a long lasting impact on the functionality and quality of future services provided by urban infrastructure. These decisions can be assisted by different approaches ranging from linear depreciation for estimating the economic value of the network over using a deterioration model to assess the probability of failure or the technical service life to sophisticated multi-criteria decision support systems. Subsequently, the aim of this paper is to compare five available multi-criteria decision-making (MCDM) methods (ELECTRE, AHP, WSM, TOPSIS, and PROMETHEE) for the application in an integrated rehabilitation management scheme for a real world case study and analyze them with respect to their suitability to be used in integrated asset management of water systems. The results of the different methods are not equal. This occurs because the chosen score scales, weights and the resulting distributions of the scores within the criteria do not have the same impact on all the methods. Independently of the method used, the decision maker must be familiar with its strengths but also weaknesses. Therefore, in some cases, it would be rational to use one of the simplest methods. However, to check for consistency and increase the reliability of the results, the application of several methods is encouraged.
Greedier is Better: Selecting Multiple Neighbors per Iteration for Sparse Subspace Clustering

Sparse subspace clustering (SSC) using greedy-based neighbor selection, such as orthogonal matching pursuit (OMP), has been known as a popular computationally-efficient alternative to the popular L1-minimization based methods. This paper proposes a new SSC scheme using generalized OMP (GOMP), a soup-up of OMP whereby multiple neighbors are identified per iteration, along with a new stopping rule requiring nothing more than a knowledge of the ambient signal dimension. Compared to conventional OMP, which identifies one neighbor per iteration, the proposed GOMP method involves fewer iterations, thereby enjoying lower algorithmic complexity; advantageously, the proposed stopping rule is free from off-line estimation of subspace dimension and noise power. Under the semi-random model, analytic performance guarantees, in terms of neighbor recovery rates, are established to justify the advantage of the proposed GOMP. The results show that, with a high probability, GOMP (i) is halted by the proposed stopping rule, and (ii) can retrieve more true neighbors than OMP, consequently yielding higher final data clustering accuracy. Computer simulations using both synthetic data and real human face data are provided to validate our analytic study and evidence the effectiveness of the proposed approach.
An Empirical Study of Remote Sensing Pretraining

Deep learning has largely reshaped remote sensing (RS) research for aerial image understanding and made a great success. Nevertheless, most of the existing deep models are initialized with the ImageNet pretrained weights. Since natural images inevitably present a large domain gap relative to aerial images, probably limiting the finetuning performance on downstream aerial scene tasks. This issue motivates us to conduct an empirical study of remote sensing pretraining (RSP) on aerial images. To this end, we train different networks from scratch with the help of the largest RS scene recognition dataset up to now — MillionAID, to obtain a series of RS pretrained backbones, including both convolutional neural networks (CNN) and vision transformers such as Swin and ViTAE, which have shown promising performance on computer vision tasks. Then, we investigate the impact of RSP on representative downstream tasks including scene recognition, semantic segmentation, object detection, and change detection using these CNN and vision transformer backbones. Empirical study shows that RSP can help deliver distinctive performances in scene recognition tasks and in perceiving RS related semantics such as “Bridge” and “Airplane”. We also find that, although RSP mitigates the data discrepancies of traditional ImageNet pretraining on RS images, it may still suffer from task discrepancies, where downstream tasks require different representations from scene recognition tasks. These findings call for further research efforts on both large-scale pretraining datasets and effective pretraining methods. The codes and pretrained models will be released at https://github.com/ViTAE-Transformer/ViTAE-Transformer-Remote-Sensing.
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound

We introduce an audiovisual method for long-range text-to-video retrieval. Unlike previous approaches designed for short video retrieval (e.g., 5-15 seconds in duration), our approach aims to retrieve minute-long videos that capture complex human actions. One challenge of standard video-only approaches is the large computational cost associated with processing hundreds of densely extracted frames from such long videos. To address this issue, we propose to replace parts of the video with compact audio cues that succinctly summarize dynamic audio events and are cheap to process. Our method, named ECLIPSE (Efficient CLIP with Sound Encoding), adapts the popular CLIP model to an audiovisual video setting, by adding a unified audiovisual transformer block that captures complementary cues from the video and audio streams. In addition to being 2.92x faster and 2.34x memory-efficient than long-range video-only approaches, our method also achieves better text-to-video retrieval accuracy on several diverse long-range video datasets such as ActivityNet, QVHighlights, YouCook2, DiDeMo and Charades.
KNN-Diffusion: Image Generation via Large-Scale Retrieval

Recent text-to-image models have achieved impressive results. However, since they require large-scale datasets of text-image pairs, it is impractical to train them on new domains where data is scarce or not labeled. In this work, we propose using large-scale retrieval methods, in particular, efficient k-Nearest-Neighbors (kNN), which offers novel capabilities: (1) training a substantially small and efficient text-to-image diffusion model without any text, (2) generating out-of-distribution images by simply swapping the retrieval database at inference time, and (3) performing text-driven local semantic manipulations while preserving object identity. To demonstrate the robustness of our method, we apply our kNN approach on two state-of-the-art diffusion backbones, and show results on several different datasets. As evaluated by human studies and automatic metrics, our method achieves state-of-the-art results compared to existing approaches that train text-to-image generation models using images only (without paired text data)
Implicit Motion-Compensated Network for Unsupervised Video Object Segmentation

Unsupervised video object segmentation (UVOS) aims at automatically separating the primary foreground object(s) from the background in a video sequence. Existing UVOS methods either lack robustness when there are visually similar surroundings (appearance-based) or suffer from deterioration in the quality of their predictions because of dynamic background and inaccurate flow (flow-based). To overcome the limitations, we propose an implicit motion-compensated network (IMCNet) combining complementary cues (i.e., appearance and motion) with aligned motion information from the adjacent frames to the current frame at the feature level without estimating optical flows. The proposed IMCNet consists of an affinity computing module (ACM), an attention propagation module (APM), and a motion compensation module (MCM). The light-weight ACM extracts commonality between neighboring input frames based on appearance features. The APM then transmits global correlation in a top-down manner. Through coarse-to-fine iterative inspiring, the APM will refine object regions from multiple resolutions so as to efficiently avoid losing details. Finally, the MCM aligns motion information from temporally adjacent frames to the current frame which achieves implicit motion compensation at the feature level. We perform extensive experiments on $\textit {DAVIS}_{\textit {16}}$ and $\textit {YouTube-Objects}$ . Our network achieves favorable performance while running at a faster speed compared to the state-of-the-art methods. Our code is available at https://github.com/xilin1991/IMCNet.
PVI-DSO: Leveraging Planar Regularities for Direct Sparse Visual-Inertial Odometry

The monocular visual-inertial odometry (VIO) based on the direct method can leverage all available pixels in the image to simultaneously estimate the camera motion and reconstruct the denser map of the scene in real-time. However, the direct method is sensitive to photometric changes, which can be compensated by introducing geometric information in the environment. In this article, we propose a monocular direct sparse VIO, which exploits the planar regularities (PVI-DSO). Our system detects the planar regularities from the 3-D mesh built on the estimated map points. To improve the pose estimation accuracy with the geometric information, a tightly coupled coplanar constraint expression is used to express photometric error in the direct method. In addition, to improve the optimization efficiency, we elaborately derive the analytical Jacobian of the linearization form for the coplanar constraint. Finally, the inertial measurement error, coplanar point photometric error, noncoplanar photometric error, and prior error are added into the optimizer, which simultaneously improves the pose estimation accuracy and mesh itself. We verified the performance of the whole system on the simulation and real-world datasets. Extensive experiments have demonstrated that our system outperforms the state-of-the-art counterparts.
Adaptive Fine-Grained Predicates Learning for Scene Graph Generation

The performance of current Scene Graph Generation (SGG) models is severely hampered by hard-to-distinguish predicates, e.g., “woman-on/standing on/walking on-beach”. As general SGG models tend to predict head predicates and re-balancing strategies prefer tail categories, none of them can appropriately handle hard-to-distinguish predicates. To tackle this issue, inspired by fine-grained image classification, which focuses on differentiating hard-to-distinguish objects, we propose an Adaptive Fine-Grained Predicates Learning (FGPL-A) which aims at differentiating hard-to-distinguish predicates for SGG. First, we introduce an Adaptive Predicate Lattice (PL-A) to figure out hard-to-distinguish predicates, which adaptively explores predicate correlations in keeping with model's dynamic learning pace. Practically, PL-A is initialized from SGG dataset, and gets refined by exploring model's predictions of current mini-batch. Utilizing PL-A, we propose an Adaptive Category Discriminating Loss (CDL-A) and an Adaptive Entity Discriminating Loss (EDL-A), which progressively regularize model's discriminating process with fine-grained supervision concerning model's dynamic learning status, ensuring balanced and efficient learning process. Extensive experimental results show that our proposed model-agnostic strategy significantly boosts performance of benchmark models on VG-SGG and GQA-SGG datasets by up to 175% and 76% on Mean Recall@100, achieving new state-of-the-art performance. Moreover, experiments on Sentence-to-Graph Retrieval and Image Captioning tasks further demonstrate practicability of our method.
Stateful Greybox Fuzzing

Many protocol implementations are reactive systems, where the protocol process is in continuous interaction with other processes and the environment. If a bug can be exposed only in a certain state, a fuzzer needs to provide a specific sequence of events as inputs that would take protocol into this state before the bug is manifested. We call these bugs as"stateful"bugs. Usually, when we are testing a protocol implementation, we do not have a detailed formal specification of the protocol to rely upon. Without knowledge of the protocol, it is inherently difficult for a fuzzer to discover such stateful bugs. A key challenge then is to cover the state space without an explicit specification of the protocol. In this work, we posit that manual annotations for state identification can be avoided for stateful protocol fuzzing. Specifically, we rely on a programmatic intuition that the state variables used in protocol implementations often appear in enum type variables whose values (the state names) come from named constants. In our analysis of the Top-50 most widely used open-source protocol implementations, we found that every implementation uses state variables that are assigned named constants (with easy to comprehend names such as INIT, READY) to represent the current state. In this work, we propose to automatically identify such state variables and track the sequence of values assigned to them during fuzzing to produce a"map"of the explored state space. Our experiments confirm that our stateful fuzzer discovers stateful bugs twice as fast as the baseline greybox fuzzer that we extended. Starting from the initial state, our fuzzer exercises one order of magnitude more state/transition sequences and covers code two times faster than the baseline fuzzer. Several zero-day bugs in prominent protocol implementations were found by our fuzzer, and 8 CVEs have been assigned.
PAGP: A physics-assisted Gaussian process framework with active learning for forward and inverse problems of partial differential equations

In this work, a Gaussian process regression(GPR) model incorporated with given physical information in partial di ﬀ erential equations(PDEs) is developed: physics-assisted Gaussian processes(PAGP). The targets of this model can be divided into two types of problem: ﬁnding solutions or discovering unknown coe ﬃ cients of given PDEs with initial and boundary conditions. We introduce three di ﬀ erent models: continuous time, discrete time and hybrid models. The given physical information is integrated into Gaussian process model through our designed GP loss functions. Three types of loss function are provided in this paper based on two di ﬀ erent approaches to train the standard GP model. The ﬁrst part of the paper introduces the continuous time model which treats temporal domain the same as spatial domain. The unknown coe ﬃ cients in given PDEs can be jointly learned with GP hyper-parameters by minimizing the designed loss function. In the discrete time models, we ﬁrst choose a time discretization scheme to discretize the temporal domain. Then the PAGP model is applied at each time step together with the scheme to approximate PDE solutions at given test points of ﬁnal time. To discover unknown coe ﬃ cients in this setting, observations at two speciﬁc time are needed and a mixed mean square error function is constructed to obtain the optimal coe ﬃ cients. In the last part, a novel hybrid model combining the continuous and discrete time models is presented. It merges the ﬂexibility of continuous time model and the accuracy of the discrete time model. The performance of choosing di ﬀ erent models with di ﬀ erent GP loss functions is also discussed. The e ﬀ ectiveness of the proposed PAGP methods is illustrated in our numerical section.
Analysis and design of lean direct injection fuel nozzles by eddy resolved turbulence simulation

xiv CHAPTER 1. OVERVIEW . 1 1.1 First hypothesis . 1 1.2 Second hypothesis . 1 1.3 Criteria review . 2 CHAPTER 2. REVIEW OF LITERATURE . 3 2.1 Pollutant NOx . 3 2.2 Early MLDI research . 4 2.3 Review of combustion test results for UTAS MLDI arrays and injectors . 7 2.4 Improvement opportunities for future MLDI arrays . 10 CHAPTER 3. METHODS AND PROCEDURES . 11 3.
Online Feedback Droop Scheduling in Distribution Grids for Frequency and Local Voltage Control

This paper presents a novel framework for collective control of Distributed Energy Resources (DERs) in active Distribution Networks (DNs). The proposed approach unifies the commonly employed local (i.e., decentralized) voltage and frequency droop control schemes into a transfer matrix relating frequency and voltage magnitude measurements to active and reactive power injection adjustments. Furthermore, the transfer matrices of individual DER units are adaptively tuned in real-time via slow communication links using an online gain scheduling approach, with the objective to enable frequency support provision to the transmission system and ensure that the DN voltages are kept within the allowable limits. A global asymptomatic stability condition of the analyzed droop-controlled DN is analytically established. The considered gain scheduling problem is solved by leveraging an online primal-dual gradient-based method and a suitable linearized power flow model. Additional ancillary service providers can be trivially incorporated into the proposed framework in a plug-and-play fashion. Numerical simulations of the 37-bus IEEE test system confirm the validity of the approach and demonstrate numerous advantages of the proposed scheme over the state-of-the-art.
Semi-DRDNet: Semi-supervised Detail-recovery Image Deraining Network via Unpaired Contrastive Learning

—The intricacy of rainy image contents often leads cutting-edge deraining models to image degradation including remnant rain, wrongly-removed details, and distorted appearance. Such degradation is further exacerbated when applying the models trained on synthetic data to real-world rainy images. We raise an intriguing question – if leveraging both accessible unpaired clean/rainy yet real-world images and additional detail repair guidance, can improve the generalization ability of a deraining model? To answer it, we propose a semi-supervised detail-recovery image deraining network (termed as Semi-DRDNet). Semi-DRDNet consists of three branches: 1) for removing rain streaks without remnants, we present a squeeze-and-excitation (SE)-based rain residual network; 2) for encouraging the lost details to return, we construct a structure detail context aggregation (SDCAB)-based detail repair network; to our knowledge, this is the ﬁrst time; and 3) for bridging the domain gap, we develop a novel contrastive regularization network to learn from unpaired positive (clean) and negative (rainy) yet real-world images. As a semi-supervised learning paradigm, Semi-DRDNet operates smoothly on both synthetic and real-world rainy data in terms of deraining robustness and detail accuracy. Comparisons on four datasets show clear visual and numerical improvements of our Semi-DRDNet over thirteen state-of-the-arts. Code is available at https://github.com/Dengsgithub/DRD-Net.
Towards Better Understanding of User Satisfaction in Open-Domain Conversational Search

With the increasing popularity of conversational search, how to evaluate the performance of conversational search systems has become an important question in the IR community. Existing works on conversational search evaluation can mainly be categorized into two streams: (1) constructing metrics based on semantic similarity (e.g. BLUE, METEOR and BERTScore), or (2) directly evaluating the response ranking performance of the system using traditional search methods (e.g. nDCG, RBP and nERR). However, these methods either ignore the information need of the user or ignore the mixed-initiative property of conversational search. This raises the question of how to accurately model user satisfaction in conversational search scenarios. Since explicitly asking users to provide satisfaction feedback is difficult, traditional IR studies often rely on the Cranfield paradigm (i.e., third-party annotation) and user behavior modeling to estimate user satisfaction in search. However, the feasibility and effectiveness of these two approaches have not been fully explored in conversational search. In this paper, we dive into the evaluation of conversational search from the perspective of user satisfaction. We build a novel conversational search experimental platform and construct a Chinese open-domain conversational search behavior dataset containing rich annotations and search behavior data. We also collect third-party satisfaction annotation at the session-level and turn-level, to investigate the feasibility of the Cranfield paradigm in the conversational search scenario. Experimental results show both some consistency and considerable differences between the user satisfaction annotations and third-party annotations. We also propose dialog continuation or ending behavior models (DCEBM) to capture session-level user satisfaction based on turn-level information.
Influence of spin-orbit coupling on chemical bonding

The inﬂuence of spin-orbit interaction on chemical bonds in elemental solids and homonuclear dimers is analyzed by means of density-functional-theory calculations. Employing highly precise all-electron full-potential methodology, our results represent benchmark quality. Comparison of the scalar- and fully-relativistic approaches for elemental solids shows that the spin-orbit interaction may contract or expand the volume of the considered material. The largest variation of the volume is obtained for Au, Tl, I, Bi, Po and Hg, exhibiting changes between 1.0–7.6%. Using the tight-binding model, we show for diatomic molecules that the nature of this eﬀect lies in the angular rearrangement of bonding and antibonding orbitals introduced by spin-orbit coupling. Such an angular rearrangement appears in partially ﬁlled p - or d -orbitals in heavy elements. Finally, we discuss the impact of the relativistic eﬀects on the chemical bonding in single-layer iodides and transition metal dichalcogenides.