Miracle-dz/UniFAR-FTUs huggingface dataset:
Hierarchy Flow For High-Fidelity Image-to-Image Translation

Image-to-image (I2I) translation comprises a wide spectrum of tasks. Here we divide this problem into three levels: strong-fidelity translation, normal-fidelity translation, and weak-fidelity translation, indicating the extent to which the content of the original image is preserved. Although existing methods achieve good performance in weak-fidelity translation, they fail to fully preserve the content in both strong- and normal-fidelity tasks, e.g. sim2real, style transfer and low-level vision. In this work, we propose Hierarchy Flow, a novel flow-based model to achieve better content preservation during translation. Specifically, 1) we first unveil the drawbacks of standard flow-based models when applied to I2I translation. 2) Next, we propose a new design, namely hierarchical coupling for reversible feature transformation and multi-scale modeling, to constitute Hierarchy Flow. 3) Finally, we present a dedicated aligned-style loss for a better trade-off between content preservation and stylization during translation. Extensive experiments on a wide range of I2I translation benchmarks demonstrate that our approach achieves state-of-the-art performance, with convincing advantages in both strong- and normal-fidelity tasks. Code and models will be at https://github.com/WeichenFan/HierarchyFlow.
MACO: A Modality Adversarial and Contrastive Framework for Modality-missing Multi-modal Knowledge Graph Completion

Recent years have seen significant advancements in multi-modal knowledge graph completion (MMKGC). MMKGC enhances knowledge graph completion (KGC) by integrating multi-modal entity information, thereby facilitating the discovery of unobserved triples in the large-scale knowledge graphs (KGs). Nevertheless, existing methods emphasize the design of elegant KGC models to facilitate modality interaction, neglecting the real-life problem of missing modalities in KGs. The missing modality information impedes modal interaction, consequently undermining the model's performance. In this paper, we propose a modality adversarial and contrastive framework (MACO) to solve the modality-missing problem in MMKGC. MACO trains a generator and discriminator adversarially to generate missing modality features that can be incorporated into the MMKGC model. Meanwhile, we design a cross-modal contrastive loss to improve the performance of the generator. Experiments on public benchmarks with further explorations demonstrate that MACO could achieve state-of-the-art results and serve as a versatile framework to bolster various MMKGC models. Our code and benchmark data are available at https://github.com/zjukg/MACO.
A Sonomyography-based Muscle Computer Interface for Individuals with Spinal Cord Injury

Impairment of hand functions in individuals with spinal cord injury (SCI) severely disrupts activities of daily living. Recent advances have enabled rehabilitation assisted by robotic devices to augment the residual function of the muscles. Traditionally, electromyography-based muscle activity sensing interfaces have been utilized to sense volitional motor intent to drive robotic assistive devices. However, the dexterity and fidelity of control that can be achieved with electromyography-based control have been limited due to inherent limitations in signal quality. We have developed and tested a muscle-computer interface (MCI) utilizing sonomyography to provide control of a virtual cursor for individuals with motor-incomplete spinal cord injury. We demonstrate that individuals with SCI successfully gained control of a virtual cursor by utilizing contractions of muscles of the wrist joint. The sonomyography-based interface enabled control of the cursor at multiple graded levels demonstrating the ability to achieve accurate and stable endpoint control. Our sonomyography-based muscle-computer interface can enable dexterous control of upper-extremity assistive devices for individuals with motor-incomplete SCI.
An Outlook into the Future of Egocentric Vision

What will the future be? We wonder! In this survey, we explore the gap between current research in egocentric vision and the ever-anticipated future, where wearable computing, with outward facing cameras and digital overlays, is expected to be integrated in our every day lives. To understand this gap, the article starts by envisaging the future through character-based stories, showcasing through examples the limitations of current technology. We then provide a mapping between this future and previously defined research tasks. For each task, we survey its seminal works, current state-of-the-art methodologies and available datasets, then reflect on shortcomings that limit its applicability to future research. Note that this survey focuses on software models for egocentric vision, independent of any specific hardware. The paper concludes with recommendations for areas of immediate explorations so as to unlock our path to the future always-on, personalised and life-enhancing egocentric vision.
HHTrack: Hyperspectral Object Tracking Based on Hybrid Attention

Hyperspectral imagery provides abundant spectral information beyond the visible RGB bands, offering rich discriminative details about objects in a scene. Leveraging such data has the potential to enhance visual tracking performance. In this paper, we propose a hyperspectral object tracker based on hybrid attention (HHTrack). The core of HHTrack is a hyperspectral hybrid attention (HHA) module that unifies feature extraction and fusion within one component through token interactions. A hyperspectral bands fusion (HBF) module is also introduced to selectively aggregate spatial and spectral signatures from the full hyperspectral input. Extensive experiments demonstrate the state-of-the-art performance of HH-Track on benchmark Near Infrared (NIR), Red Near Infrared (Red-NIR), and Visible (VIS) hyperspectral tracking datasets. Our work provides new insights into harnessing the strengths of transformers and hyperspectral fusion to advance robust object tracking.
The Hard-Constraint PINNs for Interface Optimal Control Problems

We show that the physics-informed neural networks (PINNs), in combination with some recently developed discontinuity capturing neural networks, can be applied to solve optimal control problems subject to partial differential equations (PDEs) with interfaces and some control constraints. The resulting algorithm is mesh-free and scalable to different PDEs, and it ensures the control constraints rigorously. Since the boundary and interface conditions, as well as the PDEs, are all treated as soft constraints by lumping them into a weighted loss function, it is necessary to learn them simultaneously and there is no guarantee that the boundary and interface conditions can be satisfied exactly. This immediately causes difficulties in tuning the weights in the corresponding loss function and training the neural networks. To tackle these difficulties and guarantee the numerical accuracy, we propose to impose the boundary and interface conditions as hard constraints in PINNs by developing a novel neural network architecture. The resulting hard-constraint PINNs approach guarantees that both the boundary and interface conditions can be satisfied exactly and they are decoupled from the learning of the PDEs. Its efficiency is promisingly validated by some elliptic and parabolic interface optimal control problems.
Topic-Level Bayesian Surprise and Serendipity for Recommender Systems

A recommender system that optimizes its recommendations solely to fit a user’s history of ratings for consumed items can create a filter bubble, wherein the user does not get to experience items from novel, unseen categories. One approach to mitigate this undesired behavior is to recommend items with high potential for serendipity, namely surprising items that are likely to be highly rated. In this paper, we propose a content-based formulation of serendipity that is rooted in Bayesian surprise and use it to measure the serendipity of items after they are consumed and rated by the user. When coupled with a collaborative-filtering component that identifies similar users, this enables recommending items with high potential for serendipity. To facilitate the evaluation of topic-level models for surprise and serendipity, we introduce a dataset of book reading histories extracted from Goodreads, containing over 26 thousand users and close to 1.3 million books, where we manually annotate 449 books read by 4 users in terms of their time-dependent, topic-level surprise. Experimental evaluations show that models that use Bayesian surprise correlate much better with the manual annotations of topic-level surprise than distance-based heuristics, and also obtain better serendipitous item recommendation performance.
Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI

While many Natural Language Processing (NLP) techniques have been proposed for fact-checking, both academic research and fact-checking organizations report limited adoption of such NLP work due to poor alignment with fact-checker practices, values, and needs. To address this, we investigate a co-design method, Matchmaking for AI, to enable fact-checkers, designers, and NLP researchers to collaboratively identify what fact-checker needs should be addressed by technology, and to brainstorm ideas for potential solutions. Co-design sessions we conducted with 22 professional fact-checkers yielded a set of 11 design ideas that offer a"north star", integrating fact-checker criteria into novel NLP design concepts. These concepts range from pre-bunking misinformation, efficient and personalized monitoring misinformation, proactively reducing fact-checker potential biases, and collaborative writing fact-check reports. Our work provides new insights into both human-centered fact-checking research and practice and AI co-design research.
COMPONENTS FOR ENDOMORPHISMS OF CP k : LINKING WITH THE GREEN ’ S CURRENT

Little is known about the global topology of the Fatou set U(f) for holomorphic endomorphisms f : CP → CP, when k > 1. Classical theory describes U(f) as the complement in CP of the support of a dynamically-defined closed positive (1, 1) current. Given any closed positive (1, 1) current S on CP, we give a definition of linking number between closed loops in CP \ suppS and the current S. It has the property that if lk(γ, S) 6= 0, then γ represents a non-trivial homology element in H1(CP k \ suppS). As an application, we use these linking numbers to establish that many classes of endomorphisms of CP have Fatou components with infinitely generated first homology. For example, we prove that the Fatou set has infinitely generated first homology for any polynomial endomorphism of CP for which the restriction to the line at infinity is hyperbolic and has disconnected Julia set. In addition we show that a polynomial skew product of CP has Fatou set with infinitely generated first homology if some vertical Julia set is disconnected. We then conclude with a section of concrete examples and questions for further study.
A Sequential Meta-Transfer (SMT) Learning to Combat Complexities of Physics-Informed Neural Networks: Application to Composites Autoclave Processing

Physics-Informed Neural Networks (PINNs) have gained popularity in solving nonlinear partial differential equations (PDEs) via integrating physical laws into the training of neural networks, making them superior in many scientific and engineering applications. However, conventional PINNs still fall short in accurately approximating the solution of complex systems with strong nonlinearity, especially in long temporal domains. Besides, since PINNs are designed to approximate a specific realization of a given PDE system, they lack the necessary generalizability to efficiently adapt to new system configurations. This entails computationally expensive re-training from scratch for any new change in the system. To address these shortfalls, in this work a novel sequential meta-transfer (SMT) learning framework is proposed, offering a unified solution for both fast training and efficient adaptation of PINNs in highly nonlinear systems with long temporal domains. Specifically, the framework decomposes PDE's time domain into smaller time segments to create"easier"PDE problems for PINNs training. Then for each time interval, a meta-learner is assigned and trained to achieve an optimal initial state for rapid adaptation to a range of related tasks. Transfer learning principles are then leveraged across time intervals to further reduce the computational cost.Through a composites autoclave processing case study, it is shown that SMT is clearly able to enhance the adaptability of PINNs while significantly reducing computational cost, by a factor of 100.
A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations

Modern deep neural networks, particularly recent large language models, come with massive model sizes that require significant computational and storage resources. To enable the deployment of modern models on resource-constrained environments and accelerate inference time, researchers have increasingly explored pruning techniques as a popular research direction in neural network compression. However, there is a dearth of up-to-date comprehensive review papers on pruning. To address this issue, in this survey, we provide a comprehensive review of existing research works on deep neural network pruning in a taxonomy of 1) universal/specific speedup, 2) when to prune, 3) how to prune, and 4) fusion of pruning and other compression techniques. We then provide a thorough comparative analysis of seven pairs of contrast settings for pruning (e.g., unstructured/structured) and explore emerging topics, including post-training pruning, different levels of supervision for pruning, and broader applications (e.g., adversarial robustness) to shed light on the commonalities and differences of existing methods and lay the foundation for further method development. To facilitate future research, we build a curated collection of datasets, networks, and evaluations on different applications. Finally, we provide some valuable recommendations on selecting pruning methods and prospect promising research directions. We build a repository at https://github.com/hrcheng1066/awesome-pruning.
Evolving division of labor in a response threshold model

The response threshold model explains the emergence of division of labor (i.e., task specialization) in an unstructured population by assuming that the individuals have different propensities to work on different tasks. The incentive to attend to a particular task increases when the task is left unattended and decreases when individuals work on it. Here we derive mean-field equations for the stimulus dynamics and show that they exhibit complex attractors through period-doubling bifurcation cascades when the noise disrupting the thresholds is small. In addition, we show how the fixed threshold can be set to ensure specialization in both the transient and equilibrium regimes of the stimulus dynamics. However, a complete explanation of the emergence of division of labor requires that we address the question of where the threshold variation comes from, starting from a homogeneous population. We then study a structured population scenario, where the population is divided into a large number of independent groups of equal size, and the fitness of a group is proportional to the weighted mean work performed on the tasks during a fixed period of time. Using a winner-take-all strategy to model group competition and assuming an initial homogeneous metapopulation, we find that a substantial fraction of workers specialize in each task, without the need to penalize task switching.
Stochastic Gradient Descent in the Viewpoint of Graduated Optimization

Stochastic gradient descent (SGD) method is popular for solving non-convex optimization problems in machine learning. This work investigates SGD from a viewpoint of graduated optimization, which is a widely applied approach for non-convex optimization problems. Instead of the actual optimization problem, a series of smoothed optimization problems that can be achieved in various ways are solved in the graduated optimization approach. In this work, a formal formulation of the graduated optimization is provided based on the nonnegative approximate identity, which generalizes the idea of Gaussian smoothing. Also, an asymptotic convergence result is achieved with the techniques in variational analysis. Then, we show that the traditional SGD method can be applied to solve the smoothed optimization problem. The Monte Carlo integration is used to achieve the gradient in the smoothed problem, which may be consistent with distributed computing schemes in real-life applications. From the assumptions on the actual optimization problem, the convergence results of SGD for the smoothed problem can be derived straightforwardly. Numerical examples show evidence that the graduated optimization approach may provide more accurate training results in certain cases.
Assessing the vulnerability of the rice-wheat production system in the north-west Indo-Gangetic Plains to climatic drivers

This thesis explores the spatial patterns in the vulnerability of the rice-wheat production systems of Punjab and Haryana to climate. Remote sensing monitoring is used to identify rice and wheat crop extents and to capture dynamics of the cropping system such as length of growing periods and cropland productivity. This remote sensing monitoring is integrated with analysis of climate datasets and other measures of the agricultural system to 1) identify the exposure of rice-wheat croplands to harmful climate drivers, 2) capture the sensitivity of the rice-wheat croplands to climate and to 3) inform targeted adaptations to improve climate resilience, ensure environmental sustainability and sufficient levels of production, the pillars of a climate-smart landscape. Across all India, including Punjab and Haryana, there was a fragmented spatial pattern in the occurrence, and sign, in trends of monsoon precipitation. This highlights the need for locally sensitive water resources management. Over 5 million ha of rice-wheat croplands in Punjab and Haryana were exposed to unfavourable trends in facets of monsoon precipitation; this was mainly exposure to increasing recurrence of drought years and increasing inter-annual variability in monsoon precipitation. However, crop yield-climate regression models indicated that precipitation is not influencing variability in rice or wheat crop production but growing season temperatures are. Average minimum and maximum temperature during the thermo-sensitive periods of crop development have a greater negative impact on wheat crop yield than exceedance of critical temperatures. The negative impact of warming on wheat crop production increased with later start-of-season dates. Through an integrated use of remote sensing datasets the spatial patterns in the magnitude and varying nature of the vulnerability of crop production to climate were captured. This enabled identification of location-specific stresses, such as later sowing dates, and targeting locally optimum adaptations.
Honeypot Detection Method Based on Anomalous Requests Response Differences

Honeypot detection is a popular technology in the current cyber security, which can be used to check the disguise and protection level of deployed honeypots. To address the problem of low detection accuracy of existing honeypot detection techniques, this paper proposes a honeypot detection method based on the differences of anomalous requests’ response. The method uses the anomalous request packet construction method designed in this paper to construct anomalous request packets, and sends the constructed anomalous request packets to the identity-known devices to collect the responses. Combined with the responses analysis method designed in this paper, the responses are analyzed in terms of similarity in both content and structure dimensions, which enables the evaluation of anomalous request packets in turn, and the selection of those that can consistently trigger a differential response from the honeypots to form a probing packets set based on anomaly. A deep learning model aiming at honeypot detection is designed using the responses of the identity-known devices in response to the probing packets set based on anomaly. The model and the responses of the nodes to detect to the probing packets set based on anomaly are used to detect honeypots. Experiment shows that the method is able to detect the nodes to detect with an accuracy of 96.4%.
Parvalbumin and somatostatin: biomarkers for two parallel tectothalamic pathways in the auditory midbrain

The inferior colliculus (IC) represents a crucial relay station in the auditory pathway, located in the midbrain’s tectum and primarily projecting to the thalamus. Despite the identification of distinct cell types based on various biomarkers in the IC, their specific contributions to the organization of auditory tectothalamic pathways have remained poorly understood. In this study, we demonstrate that IC neurons expressing parvalbumin (ICPV+) or somatostatin (ICSOM+) represent major, non-overlapping cell types throughout the three IC subdivisions. Strikingly, regardless of their location within the IC, these neurons predominantly project to the primary and secondary auditory thalamic nuclei, respectively. Retrograde tracing data specific to cell types indicate that ICPV+ neurons primarily receive auditory inputs, whereas ICSOM+ neurons integrate polymodal inputs that hold behavioral significance. Furthermore, ICPV+ neurons exhibit significant heterogeneity in both intrinsic electrophysiological properties and presynaptic terminal size compared to ICSOM+ neurons. Notably, approximately one quarter of ICPV+ neurons are inhibitory neurons, whereas all ICSOM+ neurons are excitatory neurons. Collectively, our findings suggest that parvalbumin and somatostatin expression in the IC can serve as biomarkers for two functionally distinct, parallel tectothalamic pathways. This discovery challenges the conventional IC subdivision-based definition of tectothalamic pathways and calls for a reassessment of their functional roles.
Plant allometry derived from Metabolic Scaling Theory and segregated by tissue functionality

Plant allometry is key for determining the role of forests in global carbon cycles, through the calculation of tree biomass using proxy measurements such as tree diameters or heights. Metabolic Scaling Theory (MST) considers the general principles that underpin allometry, but MST scaling relationships have been challenged on their lack of fit to empirical data and global applicability. Many authors have thus optimised their model forms for statistical performance over theory based approaches. We postulated that MST scaling is applicable only to the proportion of plant tissue with supportive functionality, meaning that as plants evolved tissues of specialized conductive functionality (e.g vessels) their allometry progressed into more complex relationships. Our generalised MST (gMST) models were thus created by considering conductive lumen as unsupportive area, and consequentially removing it from the original MST 2/3 scaling. According to this principle, we deducted generalized gMST relationships with mechanistically deducted coefficients. We found that the gMST height-diameter scaling outperformed current state of the art equations that are widely used within the tropics and that the model performed well across all tested ecoregions. Furthermore, the proposed aboveground biomass models performed similarly to widely used models in the literature within the tropics. The results presented indicate that the further development of generalised allo-metric models remains a research priority given the importance of assessing and monitoring global forest carbon fluxes. The height-diameter models presented can thus be of much use to the wider community in further refining carbon stock estimates globally, providing a universally applicable theoretical framework.