Miracle-dz/UniFAR-FTUs huggingface dataset:
TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval

Most existing methods in vision-language retrieval match two modalities by either comparing their global feature vectors which misses sufficient information and lacks interpretability, detecting objects in images or videos and aligning the text with fine-grained features which relies on complicated model designs, or modeling fine-grained interaction via cross-attention upon visual and textual tokens which suffers from inferior efficiency. To address these limitations, some recent works simply aggregate the token-wise similarities to achieve fine-grained alignment, but they lack intuitive explanations as well as neglect the relationships between token-level features and global representations with high-level semantics. In this work, we rethink fine-grained cross-modal alignment and devise a new model-agnostic formulation for it. We additionally demystify the recent popular works and subsume them into our scheme. Furthermore, inspired by optimal transport theory, we introduce TokenFlow, an instantiation of the proposed scheme. By modifying only the similarity function, the performance of our method is comparable to the SoTA algorithms with heavy model designs on major video-text retrieval benchmarks. The visualization further indicates that TokenFlow successfully leverages the fine-grained information and achieves better interpretability.
Intercepting A Flying Target While Avoiding Moving Obstacles: A Unified Control Framework With Deep Manifold Learning

Real-time interception of a fast-moving object by a robotic arm in cluttered environments filled with static or dynamic obstacles permits only tens of milliseconds for reaction times, hence quite challenging and arduous for state-of-the-art robotic planning algorithms to perform multiple robotic skills, for instance, catching the dynamic object and avoiding obstacles, in parallel. This paper proposes an unified framework of robotic path planning through embedding the high-dimensional temporal information contained in the event stream to distinguish between safe and colliding trajectories into a low-dimension space manifested with a pre-constructed 2D densely connected graph. We then leverage a fast graph-traversing strategy to generate the motor commands necessary to effectively avoid the approaching obstacles while simultaneously intercepting a fast-moving objects. The most distinctive feature of our methodology is to conduct both object interception and obstacle avoidance within the same algorithm framework based on deep manifold learning. By leveraging a highly efficient diffusion-map based variational autoencoding and Extended Kalman Filter(EKF), we demonstrate the effectiveness of our approach on an autonomous 7-DoF robotic arm using only onboard sensing and computation. Our robotic manipulator was capable of avoiding multiple obstacles of different sizes and shapes while successfully capturing a fast-moving soft ball thrown by hand at normal speed in different angles. Complete video demonstrations of our experiments can be found in https://sites.google. com/view/multirobotskill/home.
Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval

There are two popular loss functions used for vision-language retrieval, i.e., triplet loss and contrastive learning loss, both of them essentially minimize the difference between the similarities of negative pairs and positive pairs. More specifically, Triplet loss with Hard Negative mining (Triplet-HN), which is widely used in existing retrieval models to improve the discriminative ability, is easy to fall into local minima in training. On the other hand, Vision-Language Contrastive learning loss (VLC), which is widely used in the vision-language pre-training, has been shown to achieve significant performance gains on vision-language retrieval, but the performance of fine-tuning with VLC on small datasets is not satisfactory. This paper proposes a unified loss of pair similarity optimization for vision-language retrieval, providing a powerful tool for understanding existing loss functions. Our unified loss includes the hard sample mining strategy of VLC and introduces the margin used by the triplet loss for better similarity separation. It is shown that both Triplet-HN and VLC are special forms of our unified loss. Compared with the Triplet-HN, our unified loss has a fast convergence speed. Compared with the VLC, our unified loss is more discriminative and can provide better generalization in downstream fine-tuning tasks. Experiments on image-text and video-text retrieval benchmarks show that our unified loss can significantly improve the performance of the state-of-the-art retrieval models.
On the modulation of low-frequency quasi-periodic oscillations in black hole transients

We studied the properties of the low-frequency quasi-periodic oscillations detected in a sample of six black hole candidates (XTE J1550-564, H 1743-322, XTE J1859+226, 4U 1630-47,GX 339-4, XTE J1650-500) observed by the Rossi XTE satellite. We analyzed the relation between the full width half maximum and the frequency of all the narrow peaks detected in power density spectra where a type-C QPO is observed. Our goal was to understand the nature of the modulation of the signal by comparing the properties of different harmonic peaks in the power density spectrum. We find that for the sources in our sample the width of the fundamental and of the first harmonic are compatible with a frequency modulation, while that of the sub-harmonic is independent of frequency, possibly indicating the presence of an additional modulation in amplitude. We compare our results with those obtained earlier from GRS 1915+105 and XTE J1550-564.
Variance Tolerance Factors For Interpreting Neural Networks

Black box models only provide results for deep learning tasks, and lack informative details about how these results were obtained. In this paper, we propose a general theory that deﬁnes a variance tolerance factor (VTF) to interpret the neural networks by ranking the importance of features and construct a novel architecture consisting of a base model and feature model to demonstrate its utility. Two feature importance ranking methods and a feature selection method based on the VTF are created. A thorough evaluation on synthetic, benchmark and real datasets is provided.
Trapped Free Surface Waves for a Lamb-Oseen Vortex Flow

Trapped surface waves have been observed in a swimming pool trapped by, and rotating around, the cores of vortices. To investigate this effect, we have numerically studied the free-surface response of a Lamb-Oseen vortex to small perturbations. The fluid has finite depth but is laterally unbounded. The numerical method used is spectrally accurate, and uses a novel non-reflecting buffer region to simulate a laterally unbounded fluid. While a variety of linear waves can arise in this flow, we focus here on surface gravity waves. We investigate the linear modes of the vortex as a function of the perturbation azimuthal mode number and the vortex rotation rate. We find that at low rotation rates, linear modes decay by radiating energy to the far field, while at higher rotation rates modes become nearly neutrally stable and trapped in the vicinity of the vortex. While trapped modes have previously been seen in shallow water surface waves due to small perturbations of a bathtub vortex, the situation considered here is qualitatively different owing to the lack of an inward flow and the dispersive nature of non-shallow-water waves. We also find that for slow vortex rotation rates, trapped waves propagate in the opposite direction to the vortex rotation, whereas, above a threshold rotation rate, waves co-rotate with the flow.
Multistate models as a framework for estimand specification in clinical trials of complex processes

Intensity‐based multistate models provide a useful framework for characterizing disease processes, the introduction of interventions, loss to followup, and other complications arising in the conduct of randomized trials studying complex life history processes. Within this framework we discuss the issues involved in the specification of estimands and show the limiting values of common estimators of marginal process features based on cumulative incidence function regression models. When intercurrent events arise we stress the need to carefully define the target estimand and the importance of avoiding targets of inference that are not interpretable in the real world. This has implications for analyses, but also the design of clinical trials where protocols may help in the interpretation of estimands based on marginal features.
Rethinking Blur Synthesis for Deep Real-World Image Deblurring

In this paper, we examine the problem of real-world image deblurring and take into account two key factors for improving the performance of the deep image deblurring model, namely, training data synthesis and network architecture design. Deblurring models trained on existing synthetic datasets perform poorly on real blurry images due to domain shift. To reduce the domain gap between synthetic and real domains, we propose a novel realistic blur synthesis pipeline to simulate the camera imaging process. As a result of our proposed synthesis method, existing deblurring models could be made more robust to handle real-world blur. Furthermore, we develop an effective deblurring model that captures non-local dependencies and local context in the feature domain simultaneously. Specifically, we introduce the multi-path transformer module to UNet architecture for enriched multi-scale features learning. A comprehensive experiment on three real-world datasets shows that the proposed deblurring model performs better than state-of-the-art methods.
DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness

Vision-based tactile sensors that can measure 3D geometry of the contacting objects are crucial for robots to perform dexterous manipulation tasks. However, the existing sensors are usually complicated to fabricate and delicate to extend. In this work, we novelly take advantage of the reflection property of semitransparent elastomer to design a robust, low-cost, and easy-to-fabricate tactile sensor named DTact. DTact measures high-resolution 3D geometry accurately from the darkness shown in the captured tactile images with only a single image for calibration. In contrast to previous sensors, DTact is robust under various illumination conditions. Then, we build prototypes of DTact that have non-planar contact surfaces with minimal extra efforts and costs. Finally, we perform two intelligent robotic tasks including pose estimation and object recognition using DTact, in which DTact shows large potential in applications.
Inertial Quasi-Newton Methods for Monotone Inclusion: Efficient Resolvent Calculus and Primal-Dual Methods

We introduce an inertial quasi-Newton Forward-Backward Splitting Algorithm to solve a class of monotone inclusion problems. While the inertial step is computationally cheap, in general, the bottleneck is the evaluation of the resolvent operator. A change of the metric makes its computation hard even for (otherwise in the standard metric) simple operators. In order to fully exploit the advantage of adapting the metric, we develop a new efficient resolvent calculus for a low-rank perturbed standard metric, which accounts exactly for quasi-Newton metrics. Moreover, we prove the convergence of our algorithms, including linear convergence rates in case one of the two considered operators is strongly monotone. Beyond the general monotone inclusion setup, we instantiate a novel inertial quasi-Newton Primal-Dual Hybrid Gradient Method for solving saddle point problems. The favourable performance of our inertial quasi-Newton PDHG method is demonstrated on several numerical experiments in image processing.
Data-driven Parsing Evaluation for Child-Parent Interactions

Abstract We present a syntactic dependency treebank for naturalistic child and child-directed spoken English. Our annotations largely follow the guidelines of the Universal Dependencies project (UD [Zeman et al., 2022]), with detailed extensions to lexical and syntactic structures unique to spontaneous spoken language, as opposed to written texts or prepared speech. Compared to existing UD-style spoken treebanks and other dependency corpora of child-parent interactions specifically, our dataset is much larger (44,744 utterances; 233,907 words) and contains data from 10 children covering a wide age range (18–66 months). We conduct thorough dependency parser evaluations using both graph-based and transition-based parsers, trained on three different types of out-of-domain written texts: news, tweets, and learner data. Out-of-domain parsers demonstrate reasonable performance for both child and parent data. In addition, parser performance for child data increases along children’s developmental paths, especially between 18 and 48 months, and gradually approaches the performance for parent data. These results are further validated with in-domain training.
Benchmarking Numerical Algorithms for Harmonic Maps into the Sphere

We numerically benchmark methods for computing harmonic maps into the unit sphere, with particular focus on harmonic maps with singularities. For the discretization we compare two different approaches, both based on Lagrange finite elements. While the first method enforces the unit-length constraint only at the Lagrange nodes, the other one adds a pointwise projection to fulfill the constraint everywhere. For the solution of the resulting algebraic problems we compare a nonconforming gradient flow with a Riemannian trust-region method. Both are energy-decreasing and can be shown to converge globally to a stationary point of the Dirichlet energy. We observe that while the nonconforming and the conforming discretizations both show similar behavior, the second-order trust-region method needs less iterations than the solver based on gradient flow.
A Fully Homomorphic Encryption Scheme for Real-Time Safe Control

Fully Homomorphic Encryption (FHE) has made it possible to perform addition and multiplication operations on encrypted data. Using FHE in control thus has the advantage that control effort for a plant can be calculated remotely without ever decrypting the exchanged information. FHE in its current form is however not practically applicable for real-time control as its computational load is very high compared to traditional encryption methods. In this paper a reformulation of the Gentry FHE scheme is proposed and applied on an FPGA to solve this problem. It is shown that the resulting FHE scheme can be implemented for real-time stabilization of an inverted double pendulum using discrete time control.
SEMICON: A Learning-to-hash Solution for Large-scale Fine-grained Image Retrieval

In this paper, we propose S uppression- E nhancing M ask based attention and I nteractive C hannel transformati ON (SEMICON) to learn binary hash codes for dealing with large-scale fine-grained image retrieval tasks. In SEMICON, we first develop a suppression-enhancing mask (SEM) based attention to dynamically localize discriminative image regions. More importantly, different from existing attention mechanism simply erasing previous discriminative regions, our SEM is developed to restrain such regions and then discover other complementary regions by considering the relation between activated regions in a stage-by-stage fashion. In each stage, the interactive channel transformation (ICON) module is afterwards designed to exploit correlations across channels of attended activation tensors. Since channels could generally correspond to the parts of fine-grained objects, the part correlation can be also modeled accordingly, which further improves fine-grained retrieval accuracy. Moreover, to be computational economy, ICON is realized by an efficient two-step process. Finally, the hash learning of our SEMICON consists of both global-and local-level branches for better representing fine-grained objects and then generating binary hash codes explicitly corresponding to multiple levels. Experiments on five benchmark fine-grained datasets show our superiority over competing methods. Codes are available at https:// github.com/NJUST-VIPGroup/SEMICON.
Single-qubit gate teleportation provides a quantum advantage

Gate-teleportation circuits are arguably among the most basic examples of computations believed to provide a quantum computational advantage: In seminal work [Quantum Inf. Comput. , 4(2):134--145], Terhal and DiVincenzo have shown that these circuits elude simulation by efficient classical algorithms under plausible complexity-theoretic assumptions. Here we consider possibilistic simulation [Phys. Rev. A 106, 062430 (2022)], a particularly weak form of this task where the goal is to output any string appearing with non-zero probability in the output distribution of the circuit. We show that even for single-qubit Clifford-gate-teleportation circuits this simulation problem cannot be solved by constant-depth classical circuits with bounded fan-in gates. Our results are unconditional and are obtained by a reduction to the problem of computing the parity, a well-studied problem in classical circuit complexity.
Deep Learning based Automatic Quantification of Urethral Plate Quality using the Plate Objective Scoring Tool (POST)

Objectives: To explore the capacity of deep learning algorithm to further streamline and optimize urethral plate (UP) quality appraisal on 2D images using the plate objective scoring tool (POST), aiming to increase the objectivity and reproducibility of UP appraisal in hypospadias repair. Methods: The five key POST landmarks were marked by specialists in a 691-image dataset of prepubertal boys undergoing primary hypospadias repair. This dataset was then used to develop and validate a deep learning-based landmark detection model. The proposed framework begins with glans localization and detection, where the input image is cropped using the predicted bounding box. Next, a deep convolutional neural network (CNN) architecture is used to predict the coordinates of the five POST landmarks. These predicted landmarks are then used to assess UP quality in distal hypospadias. Results: The proposed model accurately localized the glans area, with a mean average precision (mAP) of 99.5% and an overall sensitivity of 99.1%. A normalized mean error (NME) of 0.07152 was achieved in predicting the coordinates of the landmarks, with a mean squared error (MSE) of 0.001 and a 20.2% failure rate at a threshold of 0.1 NME. Conclusions: This deep learning application shows robustness and high precision in using POST to appraise UP quality. Further assessment using international multi-centre image-based databases is ongoing. External validation could benefit deep learning algorithms and lead to better assessments, decision-making and predictions for surgical outcomes.
Lifshitz scaling and hyperscaling violation in string theory

We explore string/M-theory constructions of holographic theories with Lifshitz scaling exponent $z$ and hyperscaling violation exponent $\theta$, finding a range of $z,\theta$-values. Some of these arise as effective metrics from dimensional reduction of certain kinds of null deformations of $AdS$ spacetimes appearing in the near horizon geometries of extremal D3-, M2- and M5-brane theories. The $AdS_5$ solution in particular gives rise to $\theta=1$ in $d=2$ (boundary) space dimensions. Other solutions arise as the IIA D2- and D4-brane solutions with appropriate null deformations, and we discuss the phase structure of these systems.
Universal Policy Tracking: Scheduling for Wireless Networks with Delayed State Observation

Numerous scheduling algorithms have been proposed to optimize various performance metrics like throughput, delay and utility in wireless networks. However, these algorithms often require instantaneous access to network state information, which is not always available. While network stability can sometimes be achieved with delayed state information, other performance metrics such as latency may degrade. Thus, instead of simply stabilizing the system, our goal is to design a framework that can mimic arbitrary scheduling algorithms with performance guarantees. A naive approach is to make decisions directly with delayed information, but we show that such methods may lead to poor performance. Instead, we propose the Universal Tracking (UT) algorithm that can mimic the actions of arbitrary scheduling algorithms under observation delay. We rigorously show that the performance gap between UT and the scheduling algorithm being tracked is bounded by constants. Our numerical experiments show that UT significantly outperforms the naive approach in various applications.
RuDSI: Graph-based Word Sense Induction Dataset for Russian

We present RuDSI, a new benchmark for word sense induction (WSI) in Russian. The dataset was created using manual annotation and semi-automatic clustering of Word Usage Graphs (WUGs). RuDSI is completely data-driven (based on texts from Russian National Corpus), with no external word senses imposed on annotators. We present and analyze RuDSI, describe our annotation workflow, show how graph clustering parameters affect the dataset, report the performance that several baseline WSI methods obtain on RuDSI and discuss possibilities for improving these scores.
On the Robustness of Ensemble-Based Machine Learning Against Data Poisoning

—Machine learning is becoming ubiquitous. From ﬁnancial to medicine, machine learning models are boosting decision-making processes and even outperforming humans in some tasks. This huge progress in terms of prediction quality does not however ﬁnd a counterpart in the security of such models and corresponding predictions, where perturbations of fractions of the training set (poisoning) can seriously undermine the model accuracy. Research on poisoning attacks and defenses even predates the introduction of deep neural networks, leading to several promising solutions. Among them, ensemble-based defenses, where different models are trained on portions of the training set and their predictions are then aggregated, are getting signiﬁcant attention, due to their relative simplicity and theoretical and practical guarantees. The work in this paper designs and implements a hash-based ensemble approach for ML robustness and evaluates its applicability and performance on random forests, a machine learning model proved to be more resistant to poisoning attempts on tabular datasets. An extensive experimental evaluation is carried out to evaluate the robustness of our approach against a variety of attacks, and compare it with a traditional monolithic model based on random forests.