Miracle-dz/UniFAR-FTUs huggingface dataset:
SGRAM: Improving Scene Graph Parsing via Abstract Meaning Representation

Scene graph is structured semantic representation that can be modeled as a form of graph from images and texts. Image-based scene graph generation research has been actively conducted until recently, whereas text-based scene graph generation research has not. In this paper, we focus on the problem of scene graph parsing from textual description of a visual scene. The core idea is to use abstract meaning representation (AMR) instead of the dependency parsing mainly used in previous studies. AMR is a graph-based semantic formalism of natural language which abstracts concepts of words in a sentence contrary to the dependency parsing which considers dependency relationships on all words in a sentence. To this end, we design a simple yet effective two-stage scene graph parsing framework utilizing abstract meaning representation, SGRAM (Scene GRaph parsing via Abstract Meaning representation): 1) transforming a textual description of an image into an AMR graph (Text-to-AMR) and 2) encoding the AMR graph into a Transformer-based language model to generate a scene graph (AMR-to-SG). Experimental results show the scene graphs generated by our framework outperforms the dependency parsing-based model by 11.61\% and the previous state-of-the-art model using a pre-trained Transformer language model by 3.78\%. Furthermore, we apply SGRAM to image retrieval task which is one of downstream tasks for scene graph, and confirm the effectiveness of scene graphs generated by our framework.
ICMRec: Item Cluster-Wise Multi-Objective Optimization for Unbiased Recommendation

—Recommender system based on historical user-item interactions is of vital importance for web-based services. However, the observed data used to train the recommender model suffers from severe bias issues (e.g., exposure bias, popularity bias). Practically, the item frequency distribution of the dataset is a highly skewed power-law distribution. Interactions of a small fraction of head (popular) items account for almost the whole training data. The normal training paradigm from such biased data tends to repetitively generate recommendations from the head items, which further exacerbates the biases and affects the exploration of potentially interesting items from the niche (long-tail) set. In this work, distinct from existing methods, we innovatively explore the central theme of unbiased recommendation from an item cluster-wise multi-objective optimization perspective. Through an empirical study, we ﬁnd that head items are highly likely to be recommended because the gradients coming from head items dominate the overall gradient update process, which further affects the optimization of niche items . Aiming to balance the learning on various item clusters that differ in popularity during the training process, we characterize the recommendation task as an item cluster-wise multi-objective optimization problem. To this end, we propose a model-agnostic framework namely I tem C luster-Wise M ulti-Objective Rec ommendation (ICMRec) for unbiased recommendation. In detail, we deﬁne our item cluster-wise optimization target that the recommender model should balance all item clusters that differ in popularity. Thus we set the model learning on each item cluster as a unique optimization objective. To achieve this goal, we ﬁrst explore items’ popularity levels from a novel causal reasoning perspective. Then, we devise popularity discrepancy-based bisecting clustering to separate the discriminated item clusters. Next, we adaptively ﬁnd the overall harmonious gradient direction for multiple item cluster-wise optimization objectives from a Pareto-efﬁcient solver. Finally, in the prediction stage, we perform counterfactual inference to further eliminate the impact of user conformity. We conduct experiments on three public datasets, instantiating
Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptors

Humans intuitively solve tasks in versatile ways, varying their behavior in terms of trajectory-based planning and for individual steps. Thus, they can easily generalize and adapt to new and changing environments. Current Imitation Learning algorithms often only consider unimodal expert demonstrations and act in a state-action-based setting, making it difficult for them to imitate human behavior in case of versatile demonstrations. Instead, we combine a mixture of movement primitives with a distribution matching objective to learn versatile behaviors that match the expert's behavior and versatility. To facilitate generalization to novel task configurations, we do not directly match the agent's and expert's trajectory distributions but rather work with concise geometric descriptors which generalize well to unseen task configurations. We empirically validate our method on various robot tasks using versatile human demonstrations and compare to imitation learning algorithms in a state-action setting as well as a trajectory-based setting. We find that the geometric descriptors greatly help in generalizing to new task configurations and that combining them with our distribution-matching objective is crucial for representing and reproducing versatile behavior.
UNCERTAINTY QUANTIFICATION IN POSITRON EMISSION PARTICLE TRACKING

Positron Emission Particle Tracking (PEPT) is an imaging method for the visualiza- tion of ﬂuid motion, capable of reconstructing three-dimensional trajectories of small tracer particles suspended in nearly any medium, including ﬂuids that are opaque or contained within opaque vessels. The particles are labeled radioactively, and their positions are reconstructed from the detection of pairs of back-to-back photons emitted by positron annihilation. Current reconstruction algorithms are heuristic and typically based on minimizing the distance between the particles and the so-called lines of response (LoRs) joining the detection points, while accounting for spurious LoRs generated by scattering. Here we develop a probabilistic framework for the Bayesian inference and uncertainty quantiﬁcation of particle positions from PEPT data. We formulate a likelihood by describing the emission of photons and their noisy detection as a Poisson process in the space of LoRs. We derive formulas for the corresponding Poisson rate in the case of cylindrical detectors, accounting for both undetected and scattered photons. We illustrate the formulation by quantifying the uncertainty in the reconstruction of the position of a single particle on a circular path from data generated by state- of-the-art Monte Carlo simulations. The results show how the observation time ∆ t can be chosen optimally to balance the need for a large number of LoRs with the requirement of small particle displacement imposed by the assumption that the particle is static over ∆ t . We further show how this assumption can be relaxed by inferring jointly the position and velocity of the particle, with clear beneﬁts for the accuracy of the reconstruction.
Toward the Gravity Dual of Heterotic Small Instantons

The question of what happens when the heterotic SO(32) instanton becomes small was answered sometime back by Witten. The heterotic theory develops an enhanced Sp(2k) gauge symmetry for k small instantons, besides the allowed SO(32) gauge symmetry. An interesting question now is to ask what happens when we take the large k limit. In this paper we argue that in some special cases, where Gauss' law allows the large k limit, the dynamics of the large k small instantons can be captured by a dual gravitational description. For the cases that we elaborate in this paper, the gravity duals are non-Kaehler manifolds although in general they could be nongeometric. These small instantons are heterotic five-branes and the duality allows us to study the strongly coupled field theories on these five-branes. We review and elaborate on some of the recent observations pointing towards this duality and argue that in certain cases the gauge-gravity duality may be understood as small instanton transitions under which the instantons smoothen out and consequently lose the Sp(2k) gauge symmetry. This may explain how branes disappear on the dual side and are replaced by fluxes. We analyze the torsion classes before and after the transitions and discuss more » briefly how the Atiyah-Drinfeld-Hitchin-Manin sigma model and related vector bundles could be studied for these scenarios. « less
Risk-Sensitive Markov Decision Processes with Long-Run CVaR Criterion

CVaR (Conditional Value at Risk) is a risk metric widely used in finance. However, dynamically optimizing CVaR is difficult since it is not a standard Markov decision process (MDP) and the principle of dynamic programming fails. In this paper, we study the infinite-horizon discrete-time MDP with a long-run CVaR criterion, from the view of sensitivity-based optimization. By introducing a pseudo CVaR metric, we derive a CVaR difference formula which quantifies the difference of long-run CVaR under any two policies. The optimality of deterministic policies is derived. We obtain a so-called Bellman local optimality equation for CVaR, which is a necessary and sufficient condition for local optimal policies and only necessary for global optimal policies. A CVaR derivative formula is also derived for providing more sensitivity information. Then we develop a policy iteration type algorithm to efficiently optimize CVaR, which is shown to converge to local optima in the mixed policy space. We further discuss some extensions including the mean-CVaR optimization and the maximization of CVaR. Finally, we conduct numerical experiments relating to portfolio management to demonstrate the main results. Our work may shed light on dynamically optimizing CVaR from a sensitivity viewpoint.
Second-Order Coding Rate of Quasi-Static Rayleigh-Product MIMO Channels

—With the development of innovative applications that require high reliability and low latency, ultra-reliable and low latency communications become critical for wireless networks. In this paper, the second-order coding rate of the coherent quasi-static Rayleigh-product MIMO channel is investigated. We consider the coding rate within O ( 1 √ Mn ) of the capacity, where M and n denote the number of transmit antennas and the blocklength, respectively, and derive the closed-form upper and lower bounds for the optimal average error probability. This analysis is achieved by setting up a central limit theorem (CLT) for the mutual information density (MID) with the assumption that the block-length, the number of the scatterers, and the number of the antennas go to inﬁnity with the same pace. To obtain more physical insights, the high and low SNR approximations for the upper and lower bounds are also given. One interesting observation is that rank-deﬁciency degrades the performance of MIMO systems with FBL and the fundamental limits of the Rayleigh-product channel approaches those of the single Rayleigh case when the number of scatterers approaches inﬁnity. Finally, the ﬁtness of the CLT and the gap between the derived bounds and the performance of practical LDPC coding are illustrated by simulations.
PAR: Political Actor Representation Learning with Social Context and Expert Knowledge

Modeling the ideological perspectives of political actors is an essential task in computational political science with applications in many downstream tasks. Existing approaches are generally limited to textual data and voting records, while they neglect the rich social context and valuable expert knowledge for holistic ideological analysis. In this paper, we propose PAR, a Political Actor Representation learning framework that jointly leverages social context and expert knowledge. Specifically, we retrieve and extract factual statements about legislators to leverage social context information. We then construct a heterogeneous information network to incorporate social context and use relational graph neural networks to learn legislator representations. Finally, we train PAR with three objectives to align representation learning with expert knowledge, model ideological stance consistency, and simulate the echo chamber phenomenon. Extensive experiments demonstrate that PAR is better at augmenting political text understanding and successfully advances the state-of-the-art in political perspective detection and roll call vote prediction. Further analysis proves that PAR learns representations that reflect the political reality and provide new insights into political behavior.
Compressibility regularizes the 𝜇(I)-rheology for dense granular flows

The $\unicode[STIX]{x1D707}(I)$ -rheology was recently proposed as a potential candidate to model the incompressible flow of frictional grains in the dense inertial regime. However, this rheology was shown to be ill-posed in the mathematical sense for a large range of parameters, notably in the low and large inertial number limits (Barker et al., J. Fluid Mech., vol. 779, 2015, pp. 794–818). In this rapid communication, we extend the stability analysis of Barker et al. (J. Fluid Mech., vol. 779, 2015, pp. 794–818) to compressible flows. We show that compressibility regularizes the equations, making the problem well-posed for all parameters, with the condition that sufficient dissipation be associated with volume changes. In addition to the usual Coulomb shear friction coefficient $\unicode[STIX]{x1D707}$ , we introduce a bulk friction coefficient $\unicode[STIX]{x1D707}_{b}$ , associated with volume changes and show that the problem is well-posed if $\unicode[STIX]{x1D707}_{b}>1-7\unicode[STIX]{x1D707}/6$ . Moreover, we show that the ill-posed domain defined by Barker et al. (J. Fluid Mech., vol. 779, 2015, pp. 794–818) transforms into a domain where the flow is unstable but remains well-posed when compressibility is taken into account. These results suggest the importance of taking into account dynamic compressibility for the modelling of dense granular flows and open new perspectives to investigate the emission and propagation of acoustic waves inside these flows.
Learning Texture Transformer Network for Light Field Super-Resolution

Contrary to the traditional camera, the light field (LF) camera captures light rays approaching its surface, preserving the angular information of the incident light rays on the camera’s sensor. Light field acquisition can be done in a variety of ways, including micro-lens arrays (MLAs) [1], coded masks [2], and camera array [5]. Among these different implementations, MLA based LF cameras offer a cost-effective approach. However, there lies a spatio-angular tradeoff in this design, since a single sensor is shared to capture both spatial and angular information. To overcome this tradeoff, recently, some learning-based methods have been proposed [3, 6, 7]. In [7], a residual convolutional network is utilized to achieve high spatial resolution. Whereas, in [3] each sub-aperture image is super resolved individually and then to maintain parallax structure, a regularization network was appended. In [6], a texture transformer network is proposed that transfers high-quality texture from the reference image for target image generation. Motivated by this design, we propose Light-field Refinement: Since each view of the LF is super-resolved using TTSR independently, the resulting LF may not follow the LF structure. We take this problem as an opportunity to further enhance the quality of the LF by imposing the LF constraint. Similar research is done in [3] to remove the artifacts during the view synthesis process. We utilize the LF refinement step (known as LF blending module) to improve the overall quality. Furthermore, EPI loss function is incorporated to enforce the LF prior.
End-to-End Learning to Index and Search in Large Output Spaces

Extreme multi-label classification (XMC) is a popular framework for solving many real-world problems that require accurate prediction from a very large number of potential output choices. A popular approach for dealing with the large label space is to arrange the labels into a shallow tree-based index and then learn an ML model to efficiently search this index via beam search. Existing methods initialize the tree index by clustering the label space into a few mutually exclusive clusters based on pre-defined features and keep it fixed throughout the training procedure. This approach results in a sub-optimal indexing structure over the label space and limits the search performance to the quality of choices made during the initialization of the index. In this paper, we propose a novel method ELIAS which relaxes the tree-based index to a specialized weighted graph-based index which is learned end-to-end with the final task objective. More specifically, ELIAS models the discrete cluster-to-label assignments in the existing tree-based index as soft learnable parameters that are learned jointly with the rest of the ML model. ELIAS achieves state-of-the-art performance on several large-scale extreme classification benchmarks with millions of labels. In particular, ELIAS can be up to 2.5% better at precision@1 and up to 4% better at recall@100 than existing XMC methods. A PyTorch implementation of ELIAS along with other resources is available at https://github.com/nilesh2797/ELIAS.
Statistical, Robustness, and Computational Guarantees for Sliced Wasserstein Distances

Sliced Wasserstein distances preserve properties of classic Wasserstein distances while being more scalable for computation and estimation in high dimensions. The goal of this work is to quantify this scalability from three key aspects: (i) empirical convergence rates; (ii) robustness to data contamination; and (iii) efficient computational methods. For empirical convergence, we derive fast rates with explicit dependence of constants on dimension, subject to log-concavity of the population distributions. For robustness, we characterize minimax optimal, dimension-free robust estimation risks, and show an equivalence between robust sliced 1-Wasserstein estimation and robust mean estimation. This enables lifting statistical and algorithmic guarantees available for the latter to the sliced 1-Wasserstein setting. Moving on to computational aspects, we analyze the Monte Carlo estimator for the average-sliced distance, demonstrating that larger dimension can result in faster convergence of the numerical integration error. For the max-sliced distance, we focus on a subgradient-based local optimization algorithm that is frequently used in practice, albeit without formal guarantees, and establish an $O(\epsilon^{-4})$ computational complexity bound for it. Our theory is validated by numerical experiments, which altogether provide a comprehensive quantitative account of the scalability question.
Heterogeneous Feature Distillation Network for SAR Image Semantic Segmentation

Semantic segmentation for SAR (Synthetic Aperture Radar) images has attracted increasing attention in the remote sensing community recently, due to SAR's all-time and all-weather imaging capability. However, SAR images are generally more difficult to be segmented than their EO (Electro-Optical) counterparts, since speckle noises and layovers are inevitably involved in SAR images. To address this problem, we investigate how to introduce EO features to assist the training of a SAR-segmentation model, and propose a heterogeneous feature distillation network for segmenting SAR images, called HFD-Net, where a SAR-segmentation student model gains knowledge from a pre-trained EO-segmentation teacher model. In the proposed HFD-Net, both the student and teacher models employ an identical architecture but different parameter configurations, and a heterogeneous feature distillation model is explored for transferring latent EO features from the teacher model to the student model and then enhancing the ability of the student model for SAR image segmentation. In addition, a heterogeneous feature alignment module is explored to aggregate multi-scale features for segmentation in each of the student model and teacher model. Extensive experimental results on two public datasets demonstrate that the proposed HFD-Net outperforms seven state-of-the-art SAR image semantic segmentation methods.
A Secure Federated Data-Driven Evolutionary Multi-Objective Optimization Algorithm

Data-driven evolutionary algorithms usually aim to exploit the information behind a limited amount of data to perform optimization, which have proved to be successful in solving many complex real-world optimization problems. However, most data-driven evolutionary algorithms are centralized, causing privacy and security concerns. Existing federated Bayesian optimization algorithms and data-driven evolutionary algorithms mainly protect the raw data on each client. To address this issue, this article proposes a secure federated data-driven evolutionary multi-objective optimization algorithm to protect both the raw data and the newly infilled solutions obtained by optimizing the acquisition function conducted on the server. We select the query points on a randomly selected client at each round of surrogate update by calculating the acquisition function values of the unobserved points on this client, thereby reducing the risk of leaking the information about the solution to be sampled. In addition, since the predicted objective values of each client may contain sensitive information, we mask the objective values with Diffie-Hellman-based noise, and then send only the masked objective values of other clients to the selected client via the server. Since the calculation of the acquisition function also requires both the predicted objective value and the uncertainty of the prediction, the predicted mean objective and uncertainty are normalized to reduce the influence of noise. Experimental results on a set of widely used multi-objective optimization benchmarks show that the proposed algorithm can protect privacy and enhance security with only negligible sacrifice in the performance of federated data-driven evolutionary optimization.
QuAnt: Quantum Annealing with Learnt Couplings

Modern quantum annealers can find high-quality solutions to combinatorial optimisation objectives given as quadratic unconstrained binary optimisation (QUBO) problems. Unfortunately, obtaining suitable QUBO forms in computer vision remains challenging and currently requires problem-specific analytical derivations. Moreover, such explicit formulations impose tangible constraints on solution encodings. In stark contrast to prior work, this paper proposes to learn QUBO forms from data through gradient backpropagation instead of deriving them. As a result, the solution encodings can be chosen flexibly and compactly. Furthermore, our methodology is general and virtually independent of the specifics of the target problem type. We demonstrate the advantages of learnt QUBOs on the diverse problem types of graph matching, 2D point cloud alignment and 3D rotation estimation. Our results are competitive with the previous quantum state of the art while requiring much fewer logical and physical qubits, enabling our method to scale to larger problems. The code and the new dataset will be open-sourced.