Miracle-dz/UniFAR-FTUs huggingface dataset:
Resilience of Entropy Model in Distributed Neural Networks

Distributed deep neural networks (DNNs) have emerged as a key technique to reduce communication overhead without sacrificing performance in edge computing systems. Recently, entropy coding has been introduced to further reduce the communication overhead. The key idea is to train the distributed DNN jointly with an entropy model, which is used as side information during inference time to adaptively encode latent representations into bit streams with variable length. To the best of our knowledge, the resilience of entropy models is yet to be investigated. As such, in this paper we formulate and investigate the resilience of entropy models to intentional interference (e.g., adversarial attacks) and unintentional interference (e.g., weather changes and motion blur). Through an extensive experimental campaign with 3 different DNN architectures, 2 entropy models and 4 rate-distortion trade-off factors, we demonstrate that the entropy attacks can increase the communication overhead by up to 95%. By separating compression features in frequency and spatial domain, we propose a new defense mechanism that can reduce the transmission overhead of the attacked input by about 9% compared to unperturbed data, with only about 2% accuracy loss. Importantly, the proposed defense mechanism is a standalone approach which can be applied in conjunction with approaches such as adversarial training to further improve robustness. Code will be shared for reproducibility.
Extrapolated Plug-and-Play Three-Operator Splitting Methods for Nonconvex Optimization with Applications to Image Restoration

This paper investigates the convergence properties and applications of the three-operator splitting method, also known as Davis-Yin splitting (DYS) method, integrated with extrapolation and Plug-and-Play (PnP) denoiser within a nonconvex framework. We first propose an extrapolated DYS method to effectively solve a class of structural nonconvex optimization problems that involve minimizing the sum of three possible nonconvex functions. Our approach provides an algorithmic framework that encompasses both extrapolated forward-backward splitting and extrapolated Douglas-Rachford splitting methods.To establish the convergence of the proposed method, we rigorously analyze its behavior based on the Kurdyka-{\L}ojasiewicz property, subject to some tight parameter conditions. Moreover, we introduce two extrapolated PnP-DYS methods with convergence guarantee, where the traditional regularization prior is replaced by a gradient step-based denoiser. This denoiser is designed using a differentiable neural network and can be reformulated as the proximal operator of a specific nonconvex functional. We conduct extensive experiments on image deblurring and image super-resolution problems, where our results showcase the advantage of the extrapolation strategy and the superior performance of the learning-based model that incorporates the PnP denoiser in terms of achieving high-quality recovery images.
The Early Cretaceous sedimentary‐tectonic attributes of Lingshan Island Basin, East Shandong Province, China: Constraints from the chemical compositions of detrital heavy minerals

North China Craton experienced significant lithospheric destruction or delamination during the Late Mesozoic, accompanied with the emergence of numerous basins and intensive volcanism. Extensional basins also formed in East Shandong Province. Even though they have been studied for decades, the relationship between basin development and tectonic processes still remains obscure. Recently, a set of Early Cretaceous strata has been found in Lingshan Island. It is characterized by translating from turbidite deposits to terrestrial strata units and ending up with a unique volcaniclastic layer within a relatively short time span. Previous studies have reached an agreement that Lingshan Island strata are formed in a distinct sedimentary background compared with the coeval sedimentary units in Jiaolai Basin. Essential information about basin development and its tectonic setting in this region during Early Cretaceous are preserved in Lingshan Island strata. In this study, it shows the characteristics of heavy mineral assemblage and the characteristics of detrital minerals' chemical component in the two representative samples. Based on these studies, it indicates that Lingshan Island Basin was developed in a transpressional tectonic setting with complex source materials, mainly from Qianliyan Uplift during the period ca. 123–120 Ma. Then, the basin evolved within an extensional setting and was filled with locally derived clastics during 120 to 105 Ma. In addition, the basin evolved during the process of progressive lithospheric thinning and asthenosphere upwelling.
Face Swap via Diffusion Model

This technical report presents a diffusion model based framework for face swapping between two portrait images. The basic framework consists of three components, i.e., IP-Adapter, ControlNet, and Stable Diffusion's inpainting pipeline, for face feature encoding, multi-conditional generation, and face inpainting respectively. Besides, I introduce facial guidance optimization and CodeFormer based blending to further improve the generation quality. Specifically, we engage a recent light-weighted customization method (i.e., DreamBooth-LoRA), to guarantee the identity consistency by 1) using a rare identifier"sks"to represent the source identity, and 2) injecting the image features of source portrait into each cross-attention layer like the text features. Then I resort to the strong inpainting ability of Stable Diffusion, and utilize canny image and face detection annotation of the target portrait as the conditions, to guide ContorlNet's generation and align source portrait with the target portrait. To further correct face alignment, we add the facial guidance loss to optimize the text embedding during the sample generation.
Quasi-calibration method for structured light system with auxiliary camera

The structured light projection technique is a representative active method for 3-D reconstruction, but many researchers face challenges with the intricate projector calibration process. To address this complexity, we employs an additional camera, temporarily referred to as the auxiliary camera, to eliminate the need for projector calibration. The auxiliary camera aids in constructing rational model equations, enabling the generation of world coordinates based on absolute phase information. Once calibration is complete, the auxiliary camera can be removed, mitigating occlusion issues and allowing the system to maintain its compact single-camera, single-projector design. Our approach not only resolves the common problem of calibrating projectors in digital fringe projection systems but also enhances the feasibility of diverse-shaped 3D imaging systems that utilize fringe projection, all without the need for the complex projector calibration process.
Joint Spatial-Temporal Calibration for Camera and Global Pose Sensor

In robotics, motion capture systems have been widely used to measure the accuracy of localization algorithms. Moreover, this infrastructure can also be used for other computer vision tasks, such as the evaluation of Visual (-Inertial) SLAM dynamic initialization, multi-object tracking, or automatic annotation. Yet, to work optimally, these functionalities require having accurate and reliable spatial-temporal calibration parameters between the camera and the global pose sensor. In this study, we provide two novel solutions to estimate these calibration parameters. Firstly, we design an offline target-based method with high accuracy and consistency. Spatial-temporal parameters, camera intrinsic, and trajectory are optimized simultaneously. Then, we propose an online target-less method, eliminating the need for a calibration target and enabling the estimation of time-varying spatial-temporal parameters. Additionally, we perform detailed observability analysis for the target-less method. Our theoretical findings regarding observability are validated by simulation experiments and provide explainable guidelines for calibration. Finally, the accuracy and consistency of two proposed methods are evaluated with hand-held real-world datasets where traditional hand-eye calibration method do not work.
Understanding Police Force Resource Allocation using Adversarial Optimal Transport with Incomplete Information

Adversarial optimal transport has been proven useful as a mathematical formulation to model resource allocation problems to maximize the efficiency of transportation with an adversary, who modifies the data. It is often the case, however, that only the adversary knows which nodes are malicious and which are not. In this paper we formulate the problem of seeking adversarial optimal transport into Bayesian games. We construct the concept of Bayesian equilibrium and design a distributed algorithm that achieve those equilibria, making our model applicable to large-scale networks. Keywords: game theory, crime control, Markov games
Mermaid Cereal: Interactions and Pattern Formation in a Macroscopic Magnetocapillary SALR System

When particles are deposited at a fluid interface they tend to aggregate by capillary attraction to minimize the overall potential energy of the system. In this work, we embed floating millimetric disks with permanent magnets to introduce a competing repulsion effect and study their pattern formation in equilibrium. The pairwise energy landscape of two disks is described by a short-range attraction and long-range repulsion (SALR) interaction potential, previously documented in a number of microscopic systems. Such competing interactions enable a variety of pairwise equilibrium states, including the possibility of a local minimum energy corresponding to a finite disk spacing. Two-dimensional (2D) experiments and simulations in confined geometries demonstrate that as the areal packing fraction is increased, the dilute repulsion-dominated lattice state becomes unstable to the spontaneous formation of localized clusters, which eventually merge into a system-spanning striped pattern. Finally, we demonstrate that the equilibrium pattern can be externally manipulated by the application of a supplemental vertical magnetic force that remotely enhances the effective capillary attraction.
A Communication-Efficient Stochastic Gradient Descent Algorithm for Distributed Nonconvex Optimization

This paper studies distributed nonconvex optimization problems with stochastic gradients for a multi-agent system, in which each agent aims to minimize the sum of all agents' cost functions by using local compressed information exchange. We propose a distributed stochastic gradient descent (SGD) algorithm, suitable for a general class of compressors. We show that the proposed algorithm achieves the linear speedup convergence rate $\mathcal{O}(1/\sqrt{nT})$ for smooth nonconvex functions, where $T$ and $n$ are the number of iterations and agents, respectively. If the global cost function additionally satisfies the Polyak--{\L}ojasiewicz condition, the proposed algorithm can linearly converge to a neighborhood of the global optimum, regardless of whether the stochastic gradient is unbiased or not. Numerical experiments are carried out to verify the efficiency of our algorithm.
Optomechanical cooling with simultaneous intracavity and extracavity squeezed light

We propose a novel and experimentally feasible approach to achieve high-efficiency ground-state cooling of a mechanical oscillator in an optomechanical system under the deeply unresolved sideband condition with the assistance of both intracavity and extracavity squeezing. In the scheme, a degenerate optical parametric amplifier is placed inside the optical cavity, generating the intracavity squeezing; besides, the optical cavity is driven by externally generated squeezing light, namely the extracavity squeezing. The quantum interference effect generated by intracavity squeezing and extracavity squeezing can completely suppress the non-resonant Stokes heating process while greatly enhancing the anti-Stokes cooling process. Therefore, the joint-squeezing scheme is capable of cooling the mechanical oscillators to their quantum ground state in a regime far away from the resolved sideband condition. Compared with other traditional optomechanical cooling schemes, the single-photon cooling rate in this joint-squeezing scheme can be tremendously enlarged by nearly three orders of magnitude. At the same time, the coupling strength required to achieve ground-state cooling can be significantly reduced. This scheme is promising for cooling large-mass and low-frequency mechanical oscillators, which provides a prerequisite for preparing and manipulating non-classical states in macroscopic quantum systems and lays a significant foundation for quantum manipulation.
A fractional-order trace-dev-div inequality

The trace-dev-div inequality in $H^s$ controls the trace in the norm of $H^s$ by that of the deviatoric part plus the $H^{s-1}$ norm of the divergence of a quadratic tensor field different from the constant unit matrix. This is well known for $s=0$ and established for orders $0\le s\le 1$ and arbitrary space dimension in this note. For mixed and least-squares finite element error analysis in linear elasticity, this inequality allows to establish robustness with respect to the Lam\'e parameter $\lambda$.
PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learning

Imitation learning has shown great potential for enabling robots to acquire complex manipulation behaviors. However, these algorithms suffer from high sample complexity in long-horizon tasks, where compounding errors accumulate over the task horizons. We present PRIME (PRimitive-based IMitation with data Efficiency), a behavior primitive-based framework designed for improving the data efficiency of imitation learning. PRIME scaffolds robot tasks by decomposing task demonstrations into primitive sequences, followed by learning a high-level control policy to sequence primitives through imitation learning. Our experiments demonstrate that PRIME achieves a significant performance improvement in multi-stage manipulation tasks, with 10-34% higher success rates in simulation over state-of-the-art baselines and 20-48% on physical hardware.
IAI MovieBot 2.0: An Enhanced Research Platform with Trainable Neural Components and Transparent User Modeling

While interest in conversational recommender systems has been on the rise, operational systems suitable for serving as research platforms for comprehensive studies are currently lacking. This paper introduces an enhanced version of the IAI MovieBot conversational movie recommender system, aiming to evolve it into a robust and adaptable platform for conducting user-facing experiments. The key highlights of this enhancement include the addition of trainable neural components for natural language understanding and dialogue policy, transparent and explainable modeling of user preferences, along with improvements in the user interface and research infrastructure.
A Cost-Effective Cooperative Exploration and Inspection Strategy for Heterogeneous Aerial System

In this paper, we propose a cost-effective strategy for heterogeneous UAV swarm systems for cooperative aerial inspection. Unlike previous swarm inspection works, the proposed method does not rely on precise prior knowledge of the environment and can complete full 3D surface coverage of objects in any shape. In this work, agents are partitioned into teams, with each drone assign a different task, including mapping, exploration, and inspection. Task allocation is facilitated by assigning optimal inspection volumes to each team, following best-first rules. A voxel map-based representation of the environment is used for pathfinding, and a rule-based path-planning method is the core of this approach. We achieved the best performance in all challenging experiments with the proposed approach, surpassing all benchmark methods for similar tasks across multiple evaluation trials. The proposed method is open source at https://github.com/ntu-aris/caric_baseline and used as the baseline of the Cooperative Aerial Robots Inspection Challenge at the 62nd IEEE Conference on Decision and Control 2023.
ROME: Memorization Insights from Text, Probability and Hidden State in Large Language Models

Probing the memorization of large language models holds significant importance. Previous works have established metrics for quantifying memorization, explored various influencing factors, such as data duplication, model size, and prompt length, and evaluated memorization by comparing model outputs with training corpora. However, the training corpora are of enormous scale and its pre-processing is time-consuming. To explore memorization without accessing training data, we propose a novel approach, named ROME, wherein memorization is explored by comparing disparities across memorized and non-memorized. Specifically, models firstly categorize the selected samples into memorized and non-memorized groups, and then comparing the demonstrations in the two groups from the insights of text, probability, and hidden state. Experimental findings show the disparities in factors including word length, part-of-speech, word frequency, mean and variance, just to name a few.
Teaching MLP More Graph Information: A Three-stage Multitask Knowledge Distillation Framework

We study the challenging problem for inference tasks on large-scale graph datasets of Graph Neural Networks: huge time and memory consumption, and try to overcome it by reducing reliance on graph structure. Even though distilling graph knowledge to student MLP is an excellent idea, it faces two major problems of positional information loss and low generalization. To solve the problems, we propose a new three-stage multitask distillation framework. In detail, we use Positional Encoding to capture positional information. Also, we introduce Neural Heat Kernels responsible for graph data processing in GNN and utilize hidden layer outputs matching for better performance of student MLP's hidden layers. To the best of our knowledge, it is the first work to include hidden layer distillation for student MLP on graphs and to combine graph Positional Encoding with MLP. We test its performance and robustness with several settings and draw the conclusion that our work can outperform well with good stability.
On the Hardness of Fair Allocation under Ternary Valuations

We study the problem of fair allocation of indivisible items when agents have ternary additive valuations -- each agent values each item at some fixed integer values $a$, $b$, or $c$ that are common to all agents. The notions of fairness we consider are max Nash welfare (MNW), when $a$, $b$, and $c$ are non-negative, and max egalitarian welfare (MEW). We show that for any distinct non-negative $a$, $b$, and $c$, maximizing Nash welfare is APX-hard -- i.e., the problem does not admit a PTAS unless P = NP. We also show that for any distinct $a$, $b$, and $c$, maximizing egalitarian welfare is APX-hard except for a few cases when $b = 0$ that admit efficient algorithms. These results make significant progress towards completely characterizing the complexity of computing exact MNW allocations and MEW allocations. En route, we resolve open questions left by prior work regarding the complexity of computing MNW allocations under bivalued valuations, and MEW allocations under ternary mixed manna.
Ups and Downs among Danish Urban Harvestmen

Abstract The opilionid fauna of Northern Europe, including Denmark, has become enriched by range expansion of several species originating from southern Europe, and this process is expected to continue into the future. I present data that describe the changing occurrence (relative abundance, locality representation) of invasive as well as native harvestman species from the urban habitat in Denmark during the 30 year period 1987 to 2017. Three invasive species and three native species have increased both in relative abundance and in the proportion of towns from which they occurred during standardized searches. Another three species showed consistently decreasing trends. Of these, Leiobunum gracile decreased dramatically from being one of the most dominant species to near extinction. Opilio parietinus and Mitopus morio were consistently but less seriously reduced. The abundance of species in the standardized searches varied idiosyncratically in relation to geography (latitude, longitude) and season. Harvestman numbers were negatively correlated with town size (number of inhabitants).
Emotion Analysis in NLP: Trends, Gaps and Roadmap for Future Directions

Emotions are a central aspect of communication. Consequently, emotion analysis (EA) is a rapidly growing field in natural language processing (NLP). However, there is no consensus on scope, direction, or methods. In this paper, we conduct a thorough review of 154 relevant NLP publications from the last decade. Based on this review, we address four different questions: (1) How are EA tasks defined in NLP? (2) What are the most prominent emotion frameworks and which emotions are modeled? (3) Is the subjectivity of emotions considered in terms of demographics and cultural factors? and (4) What are the primary NLP applications for EA? We take stock of trends in EA and tasks, emotion frameworks used, existing datasets, methods, and applications. We then discuss four lacunae: (1) the absence of demographic and cultural aspects does not account for the variation in how emotions are perceived, but instead assumes they are universally experienced in the same manner; (2) the poor fit of emotion categories from the two main emotion theories to the task; (3) the lack of standardized EA terminology hinders gap identification, comparison, and future goals; and (4) the absence of interdisciplinary research isolates EA from insights in other fields. Our work will enable more focused research into EA and a more holistic approach to modeling emotions in NLP.
Snapshot Reinforcement Learning: Leveraging Prior Trajectories for Efficiency

Deep reinforcement learning (DRL) algorithms require substantial samples and computational resources to achieve higher performance, which restricts their practical application and poses challenges for further development. Given the constraint of limited resources, it is essential to leverage existing computational work (e.g., learned policies, samples) to enhance sample efficiency and reduce the computational resource consumption of DRL algorithms. Previous works to leverage existing computational work require intrusive modifications to existing algorithms and models, designed specifically for specific algorithms, lacking flexibility and universality. In this paper, we present the Snapshot Reinforcement Learning (SnapshotRL) framework, which enhances sample efficiency by simply altering environments, without making any modifications to algorithms and models. By allowing student agents to choose states in teacher trajectories as the initial state to sample, SnapshotRL can effectively utilize teacher trajectories to assist student agents in training, allowing student agents to explore a larger state space at the early training phase. We propose a simple and effective SnapshotRL baseline algorithm, S3RL, which integrates well with existing DRL algorithms. Our experiments demonstrate that integrating S3RL with TD3, SAC, and PPO algorithms on the MuJoCo benchmark significantly improves sample efficiency and average return, without extra samples and additional computational resources.