Miracle-dz/UniFAR-FTUs huggingface dataset:
RDFC-GAN: RGB-Depth Fusion CycleGAN for Indoor Depth Completion

The raw depth image captured by indoor depth sensors usually has an extensive range of missing depth values due to inherent limitations such as the inability to perceive transparent objects and the limited distance range. The incomplete depth map with missing values burdens many downstream vision tasks, and a rising number of depth completion methods have been proposed to alleviate this issue. While most existing methods can generate accurate dense depth maps from sparse and uniformly sampled depth maps, they are not suitable for complementing large contiguous regions of missing depth values, which is common and critical in images captured in indoor environments. To overcome these challenges, we design a novel two-branch end-to-end fusion network named RDFC-GAN, which takes a pair of RGB and incomplete depth images as input to predict a dense and completed depth map. The first branch employs an encoder-decoder structure, by adhering to the Manhattan world assumption and utilizing normal maps from RGB-D information as guidance, to regress the local dense depth values from the raw depth map. In the other branch, we propose an RGB-depth fusion CycleGAN to transfer the RGB image to the fine-grained textured depth map. We adopt adaptive fusion modules named W-AdaIN to propagate the features across the two branches, and we append a confidence fusion head to fuse the two outputs of the branches for the final depth map. Extensive experiments on NYU-Depth V2 and SUN RGB-D demonstrate that our proposed method clearly improves the depth completion performance, especially in a more realistic setting of indoor environments, with the help of our proposed pseudo depth maps in training.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images

Structured text extraction is one of the most valuable and challenging application directions in the field of Document AI. However, the scenarios of past benchmarks are limited, and the corresponding evaluation protocols usually focus on the submodules of the structured text extraction scheme. In order to eliminate these problems, we organized the ICDAR 2023 competition on Structured text extraction from Visually-Rich Document images (SVRD). We set up two tracks for SVRD including Track 1: HUST-CELL and Track 2: Baidu-FEST, where HUST-CELL aims to evaluate the end-to-end performance of Complex Entity Linking and Labeling, and Baidu-FEST focuses on evaluating the performance and generalization of Zero-shot / Few-shot Structured Text extraction from an end-to-end perspective. Compared to the current document benchmarks, our two tracks of competition benchmark enriches the scenarios greatly and contains more than 50 types of visually-rich document images (mainly from the actual enterprise applications). The competition opened on 30th December, 2022 and closed on 24th March, 2023. There are 35 participants and 91 valid submissions received for Track 1, and 15 participants and 26 valid submissions received for Track 2. In this report we will presents the motivation, competition datasets, task definition, evaluation protocol, and submission summaries. According to the performance of the submissions, we believe there is still a large gap on the expected information extraction performance for complex and zero-shot scenarios. It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
The cosmic web around the Coma cluster from constrained cosmological simulations

Galaxy clusters in the Universe occupy the important position of nodes of the cosmic web. They are connected among them by filaments, elongated structures composed of dark matter, galaxies, and gas. The connection of galaxy clusters to filaments is important, as it is related to the process of matter accretion onto the former. For this reason, investigating the connections to the cosmic web of massive clusters, especially well-known ones for which a lot of information is available, is a hot topic in astrophysics. In a previous work, we performed an analysis of the filament connections of the Coma cluster of galaxies, as detected from the observed galaxy distribution. In this work we resort to a numerical simulation whose initial conditions are constrained to reproduce the local Universe, including the region of the Coma cluster to interpret our observations in an evolutionary context. We detect the filaments connected to the simulated Coma cluster and perform an accurate comparison with the cosmic web configuration we detect in observations. We perform an analysis of the halos’ spatial and velocity distributions close to the filaments in the cluster outskirts. We conclude that, although not significantly larger than the average, the flux of accreting matter on the simulated Coma cluster is significantly more collimated close to the filaments with respect to the general isotropic accretion flux. This paper is the first example of such a result and the first installment in a series of publications which will explore the build-up of the Coma cluster system in connection to the filaments of the cosmic web as a function of redshift.
Information Flow Control in Machine Learning through Modular Model Architecture

In today's machine learning (ML) models, any part of the training data can affect its output. This lack of control for information flow from training data to model output is a major obstacle in training models on sensitive data when access control only allows individual users to access a subset of data. To enable secure machine learning for access controlled data, we propose the notion of information flow control for machine learning, and develop a secure Transformer-based language model based on the Mixture-of-Experts (MoE) architecture. The secure MoE architecture controls information flow by limiting the influence of training data from each security domain to a single expert module, and only enabling a subset of experts at inference time based on an access control policy. The evaluation using a large corpus of text data shows that the proposed MoE architecture has minimal (1.9%) performance overhead and can significantly improve model accuracy (up to 37%) by enabling training on access-controlled data.
Fisher information and shape-morphing modes for solving the Fokker-Planck equation in higher dimensions

The Fokker-Planck equation describes the evolution of the probability density associated with a stochastic differential equation. As the dimension of the system grows, solving this partial differential equation (PDE) using conventional numerical methods becomes computationally prohibitive. Here, we introduce a fast, scalable, and interpretable method for solving the Fokker-Planck equation which is applicable in higher dimensions. This method approximates the solution as a linear combination of shape-morphing Gaussians with time-dependent means and covariances. These parameters evolve according to the method of reduced-order nonlinear solutions (RONS) which ensures that the approximate solution stays close to the true solution of the PDE for all times. As such, the proposed method approximates the transient dynamics as well as the equilibrium density, when the latter exists. Our approximate solutions can be viewed as an evolution on a finite-dimensional statistical manifold embedded in the space of probability densities. We show that the metric tensor in RONS coincides with the Fisher information matrix on this manifold. We also discuss the interpretation of our method as a shallow neural network with Gaussian activation functions and time-varying parameters. In contrast to existing deep learning methods, our method is interpretable, requires no training, and automatically ensures that the approximate solution satisfies all properties of a probability density.
Greedy-Mine: A Profitable Mining Attack Strategy in Bitcoin-NG

Bitcoin-NG is an extensible blockchain protocol based on the same trust model as Bitcoin. It divides each epoch into one keyblock and multiple microblocks, effectively improving the transaction processing capacity. Bitcoin-NG adopts a special incentive mechanism (i.e., the transaction fees in each epoch are split to the current and next leader) to maintain its security. However, there are some limitations to the existing incentive analysis of Bitcoin-NG in recent works. First, the incentive division method of Bitcoin-NG only includes some specific mining attack strategies of the adversary, while ignoring more stubborn attack strategies. Second, once adversaries find a whale transaction, they will deviate from the honest mining strategies to obtain an extra reward. In this paper, we are committed to solving these two limitations. First, we propose a novel mining strategy named Greedy-Mine attack. Then, we formulate a Markov reward process (MRP) model to analyze the competition of honest miners and adversaries. Furthermore, we analyze the extra reward of adversaries and summarize the mining power proportion required for malicious adversaries to launch Greedy-Mine to obtain extra returns. Meanwhile, we make a backward-compatibility progressive modification to Bitcoin-NG protocol that would raise the threshold of propagation factor from 0 to 1. Finally, we get the winning condition of adversaries when adopting Greedy-Mine, compared with honest mining. Simulation and experimental results indicate that Bitcoin-NG is not incentive compatible, which is vulnerable to Greedy-Mine attack.
Financial Numeric Extreme Labelling: A Dataset and Benchmarking for XBRL Tagging

The U.S. Securities and Exchange Commission (SEC) mandates all public companies to file periodic financial statements that should contain numerals annotated with a particular label from a taxonomy. In this paper, we formulate the task of automating the assignment of a label to a particular numeral span in a sentence from an extremely large label set. Towards this task, we release a dataset, Financial Numeric Extreme Labelling (FNXL), annotated with 2,794 labels. We benchmark the performance of the FNXL dataset by formulating the task as (a) a sequence labelling problem and (b) a pipeline with span extraction followed by Extreme Classification. Although the two approaches perform comparably, the pipeline solution provides a slight edge for the least frequent labels.
Efficient automatic design of robots

Significance AI systems have taken massive strides designing prose, artwork, gameplay, software, and proteins but have yet to master the design of complex physical machines. Here we introduce an automatic optimization method that can design self-moving machines—robots—from scratch by tracing failures in their behavior back to errors or inefficiencies in particular parts of their physical structure. Because this method improves the robot in this way, it can optimize the interdependent parts of the robot much more rapidly than the current approach, in which the designer tries different robot designs in a trial-and-error fashion. This opens the way toward bespoke AI-driven design of robots for a wide range of tasks, rapidly and on demand.
Dance Generation by Sound Symbolic Words

This study introduces a novel approach to generate dance motions using onomatopoeia as input, with the aim of enhancing creativity and diversity in dance generation. Unlike text and music, onomatopoeia conveys rhythm and meaning through abstract word expressions without constraints on expression and without need for specialized knowledge. We adapt the AI Choreographer framework and employ the Sakamoto system, a feature extraction method for onomatopoeia focusing on phonemes and syllables. Additionally, we present a new dataset of 40 onomatopoeia-dance motion pairs collected through a user survey. Our results demonstrate that the proposed method enables more intuitive dance generation and can create dance motions using sound-symbolic words from a variety of languages, including those without onomatopoeia. This highlights the potential for diverse dance creation across different languages and cultures, accessible to a wider audience. Qualitative samples from our model can be found at: https://sites.google. com/view/onomatopoeia-dance/home/.
An institutional approach to cross-national

Cross-national distance is a key concept in the field ot management. Previous research has conceptualized and measured cross-national differences mostly in terms of dyadic cultural distance, and has used the Euclidean approach to measuring it. In contrast, our goal is to disaggregate the construct of distance by proposing a set of multidimensional measures, including economic, financial, political, administrative, cultural, demographic, knowledge, and global connectedness as well as geographic distance. We ground our analysis and choice of empirical dimensions on institutional theories of national business, governance, and innovation systems. In order to overcome the methodological limitations of the Euclidean approach, we calculate dyadic distances using the Mahalanobis method, which is scaleinvariant and takes into consideration the variance-covariance matrix. We empirically analyze four different foreign expansion choices of US companies to illustrate the importance of disaggregating the distance construct and the usefulness of our distance calculations, which we make freely available to managers and scholars. journal of International Business Studies (201 0) 41, 1 460- 1 480.
Spike-based computation using classical recurrent neural networks

Spiking neural networks are a type of artificial neural networks in which communication between neurons is only made of events, also called spikes. This property allows neural networks to make asynchronous and sparse computations and therefore drastically decrease energy consumption when run on specialized hardware. However, training such networks is known to be difficult, mainly due to the non-differentiability of the spike activation, which prevents the use of classical backpropagation. This is because state-of-the-art spiking neural networks are usually derived from biologically-inspired neuron models, to which are applied machine learning methods for training. Nowadays, research about spiking neural networks focuses on the design of training algorithms whose goal is to obtain networks that compete with their non-spiking version on specific tasks. In this paper, we attempt the symmetrical approach: we modify the dynamics of a well-known, easily trainable type of recurrent neural network to make it event-based. This new RNN cell, called the Spiking Recurrent Cell, therefore communicates using events, i.e. spikes, while being completely differentiable. Vanilla backpropagation can thus be used to train any network made of such RNN cell. We show that this new network can achieve performance comparable to other types of spiking networks in the MNIST benchmark and its variants, the Fashion-MNIST and the Neuromorphic-MNIST. Moreover, we show that this new cell makes the training of deep spiking networks achievable.
Proximal Symmetric Non-negative Latent Factor Analysis: A Novel Approach to Highly-Accurate Representation of Undirected Weighted Networks

An Undirected Weighted Network (UWN) is commonly found in big data-related applications. Note that such a network's information connected with its nodes, and edges can be expressed as a Symmetric, High-Dimensional and Incomplete (SHDI) matrix. However, existing models fail in either modeling its intrinsic symmetry or low-data density, resulting in low model scalability or representation learning ability. For addressing this issue, a Proximal Symmetric Nonnegative Latent-factor-analysis (PSNL) model is proposed. It incorporates a proximal term into symmetry-aware and data density-oriented objective function for high representation accuracy. Then an adaptive Alternating Direction Method of Multipliers (ADMM)-based learning scheme is implemented through a Tree-structured of Parzen Estimators (TPE) method for high computational efficiency. Empirical studies on four UWNs demonstrate that PSNL achieves higher accuracy gain than state-of-the-art models, as well as highly competitive computational efficiency.
Many Roads Lead to Lithium: Formation Pathways For Lithium-rich Red Giants

Stellar models predict that lithium (Li) inside a star is destroyed during the first dredge-up phase, yet 1.2% of red giant stars are Li-rich. We aim to uncover possible origins of this population, by analyzing 1099 Li-rich giants (A(Li) ≥ 1.5) in GALAH DR3. To expose peculiar traits of Li-rich stars, we construct a reference sample of Li-normal (doppelgänger) stars with matched evolutionary state and fiducial iron-peak and alpha-process abundances ([Fe/H] and [Mg/Fe]). Comparing Li-rich and doppelgänger spectra reveals systematic differences in the Hα and Ca-triplet line profiles associated with the velocity broadening measurement. We also find twice as many Li-rich stars appear to be fast rotators (2% with v broad ≳ 20 km s−1) compared to doppelgängers. On average, Li-rich stars have higher abundances than their doppelgängers, for a subset of elements, and Li-rich stars at the base of RGB have higher mean s-process abundances (≥0.05 dex for Ba, Y, Zr), relative to their doppelgängers. External mass-transfer from intermediate-mass AGB companions could explain this signature. Additional companion analysis excludes binaries with mass ratios ≳0.5 at ≳7 au. Finally, we confirm a prevalence of Li-rich stars on the red clump that increases with lithium, which supports an evolutionary state mechanism for Li-enhancement. Multiple culprits, including binary spin-up and mass-transfer, are therefore likely mechanisms of Li-enrichment.
Simulation-Based Counterfactual Causal Discovery on Real World Driver Behaviour

Being able to reason about how one’s behaviour can affect the behaviour of others is a core skill required of intelligent driving agents. Despite this, the state of the art struggles to meet the need of agents to discover causal links between themselves and others. Observational approaches struggle because of the non-stationarity of causal links in dynamic environments, and the sparsity of causal interactions while requiring the approaches to work in an online fashion. Meanwhile interventional approaches are impractical as a vehicle cannot experiment with its actions on a public road. To counter the issue of non-stationarity we reformulate the problem in terms of extracted events, while the previously mentioned restriction upon interventions can be overcome with the use of counterfactual simulation. We present three variants of the proposed counterfactual causal discovery method and evaluate these against state of the art observational temporal causal discovery methods across 3396 causal scenes extracted from a real world driving dataset. We find that the proposed method significantly outperforms the state of the art on the proposed task quantitatively and can offer additional insights by comparing the outcome of an alternate series of decisions in a way that observational and interventional approaches cannot.
Quantum defogging: temporal photon number fluctuation correlation in time-variant fog scattering medium

The conventional McCartney model simplifies fog as a scattering medium with space-time invariance, as the time-variant nature of fog is a pure noise for classical optical imaging. In this letter, an opposite finding to traditional idea is reported. The time parameter is incorporated into the McCartney model to account for photon number fluctuation introduced by time-variant fog. We demonstrated that the randomness of ambient photons in the time domain results in the absence of a stable correlation, while the scattering photons are the opposite. This difference can be measured by photon number fluctuation correlation when two conditions are met. A defogging image is reconstructed from the target's information carried by scattering light. Thus, the noise introduced by time-variant fog is eliminated by itself. Distinguishable images can be obtained even when the target is indistinguishable by conventional cameras, providing a prerequisite for subsequent high-level computer vision tasks.
A Robust Likelihood Model for Novelty Detection

Current approaches to novelty or anomaly detection are based on deep neural networks. Despite their effectiveness, neural networks are also vulnerable to imperceptible deformations of the input data. This is a serious issue in critical applications, or when data alterations are generated by an adversarial attack. While this is a known problem that has been studied in recent years for the case of supervised learning, the case of novelty detection has received very limited attention. Indeed, in this latter setting the learning is typically unsupervised because outlier data is not available during training, and new approaches for this case need to be investigated. We propose a new prior that aims at learning a robust likelihood for the novelty test, as a defense against attacks. We also integrate the same prior with a state-of-the-art novelty detection approach. Because of the geometric properties of that approach, the resulting robust training is computationally very efficient. An initial evaluation of the method indicates that it is effective at improving performance with respect to the standard models in the absence and presence of attacks.
Switching Autoregressive Low-rank Tensor Models

An important problem in time-series analysis is modeling systems with time-varying dynamics. Probabilistic models with joint continuous and discrete latent states offer interpretable, efficient, and experimentally useful descriptions of such data. Commonly used models include autoregressive hidden Markov models (ARHMMs) and switching linear dynamical systems (SLDSs), each with its own advantages and disadvantages. ARHMMs permit exact inference and easy parameter estimation, but are parameter intensive when modeling long dependencies, and hence are prone to overfitting. In contrast, SLDSs can capture long-range dependencies in a parameter efficient way through Markovian latent dynamics, but present an intractable likelihood and a challenging parameter estimation task. In this paper, we propose switching autoregressive low-rank tensor (SALT) models, which retain the advantages of both approaches while ameliorating the weaknesses. SALT parameterizes the tensor of an ARHMM with a low-rank factorization to control the number of parameters and allow longer range dependencies without overfitting. We prove theoretical and discuss practical connections between SALT, linear dynamical systems, and SLDSs. We empirically demonstrate quantitative advantages of SALT models on a range of simulated and real prediction tasks, including behavioral and neural datasets. Furthermore, the learned low-rank tensor provides novel insights into temporal dependencies within each discrete state.
Distributed Flocking Control of Aerial Vehicles Based on a Markov Random Field

The distributed flocking control of collective aerial vehicles has extraordinary advantages in scalability and reliability, etc. However, it is still challenging to design a reliable, efficient, and responsive flocking algorithm. In this paper, a distributed predictive flocking framework is presented based on a Markov random field (MRF). The MRF is used to characterize the optimization problem that is eventually resolved by discretizing the input space. Potential functions are employed to describe the interactions between aerial vehicles and as indicators of flight performance. The dynamic constraints are taken into account in the candidate feasible trajectories which correspond to random variables. Numerical simulation shows that compared with some existing latest methods, the proposed algonthm has better-flocking cohesion and control efficiency performances. Experiments are also conducted to demonstrate the feasibility of the proposed algorithm.