Miracle-dz/UniFAR-FTUs huggingface dataset:
Talking Head(?) Anime from a Single Image 4: Improved Model and Its Distillation

We study the problem of creating a character model that can be controlled in real time from a single image of an anime character. A solution to this problem would greatly reduce the cost of creating avatars, computer games, and other interactive applications. Talking Head Anime 3 (THA3) is an open source project that attempts to directly address the problem. It takes as input (1) an image of an anime character's upper body and (2) a 45-dimensional pose vector and outputs a new image of the same character taking the specified pose. The range of possible movements is expressive enough for personal avatars and certain types of game characters. However, the system is too slow to generate animations in real time on common PCs, and its image quality can be improved. In this paper, we improve THA3 in two ways. First, we propose new architectures for constituent networks that rotate the character's head and body based on U-Nets with attention that are widely used in modern generative models. The new architectures consistently yield better image quality than the THA3 baseline. Nevertheless, they also make the whole system much slower: it takes up to 150 milliseconds to generate a frame. Second, we propose a technique to distill the system into a small network (less than 2 MB) that can generate 512x512 animation frames in real time (under 30 FPS) using consumer gaming GPUs while keeping the image quality close to that of the full system. This improvement makes the whole system practical for real-time applications.
On the Convergence Rate of Linear Datalogo over Stable Semirings

Datalogo is an extension of Datalog, where instead of a program being a collection of union of conjunctive queries over the standard Boolean semiring, a program may now be a collection of sum-sum-product queries over an arbitrary commutative partially ordered pre-semiring. Datalogo is more powerful than Datalog in that its additional algebraic structure alows for supporting recursion with aggregation. At the same time, Datalogo retains the syntactic and semantic simplicity of Datalog: Datalogo has declarative least fixpoint semantics. The least fixpoint can be found via the na\"ive evaluation algorithm that repeatedly applies the immediate sequence opeator until no further change is possible. It was shown that, when the underlying semiring is $p$-stable, then the naive evaluation of any Datalogo program over the semiring converges in a finite number of steps. However, the upper bounds on the rate of convergence were exponential in the number of ground IDB atoms. This paper establishes polynomial upper bounds on the convergence rate of the na\"ive algorithm on {\bf linear} Datalogo programs, which is quite common in practice. In particular, the main result of this paper is that the convergence rate of linear Datalogo programs under any $p$-stable semiring is $O(pn^3)$. Furthermore, we show a matching lower bound by constructing a $p$-stable semiring and a linear Datalogo program that requires $\Omega(pn^3)$ iterations for the na\"ive iteration algorithm to converge. Next, we study the convergence rate in terms of the number of elements in the semiring for linear Datalogo programs. When $L$ is the number of elements, the convergence rate is bounded by $O(pn \log L)$. This significantly improves the convergence rate for small $L$. We show a nearly matching lower bound as well.
360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries

Portable 360$^\circ$ cameras are becoming a cheap and efficient tool to establish large visual databases. By capturing omnidirectional views of a scene, these cameras could expedite building environment models that are essential for visual localization. However, such an advantage is often overlooked due to the lack of valuable datasets. This paper introduces a new benchmark dataset, 360Loc, composed of 360$^\circ$ images with ground truth poses for visual localization. We present a practical implementation of 360$^\circ$ mapping combining 360$^\circ$ images with lidar data to generate the ground truth 6DoF poses. 360Loc is the first dataset and benchmark that explores the challenge of cross-device visual positioning, involving 360$^\circ$ reference frames, and query frames from pinhole, ultra-wide FoV fisheye, and 360$^\circ$ cameras. We propose a virtual camera approach to generate lower-FoV query frames from 360$^\circ$ images, which ensures a fair comparison of performance among different query types in visual localization tasks. We also extend this virtual camera approach to feature matching-based and pose regression-based methods to alleviate the performance loss caused by the cross-device domain gap, and evaluate its effectiveness against state-of-the-art baselines. We demonstrate that omnidirectional visual localization is more robust in challenging large-scale scenes with symmetries and repetitive structures. These results provide new insights into 360-camera mapping and omnidirectional visual localization with cross-device queries.
The Carbon-deficient Evolution of TRAPPIST-1c

Transiting planets orbiting M dwarfs provide the best opportunity to study the atmospheres of rocky planets with current facilities. As JWST enters its second year of science operations, an important initial endeavor is to determine whether these rocky planets have atmospheres at all. M dwarfs are thought to pose a major threat to planetary atmospheres due to their high magnetic activity over timescales of several billion years, and might completely strip atmospheres. Several Cycle 1 and 2 General Observers and Guaranteed Time Observations programs are testing this hypothesis, observing a series of rocky planets to determine whether they retained their atmospheres. A key case study is TRAPPIST-1c, which receives almost the same bolometric flux as Venus. We might therefore expect TRAPPIST-1c to possess a thick, CO2-dominated atmosphere. Instead, Zieba et al. show that it has little to no CO2 in its atmosphere. To interpret these results, we run coupled time-dependent simulations of planetary outgassing and atmospheric escape to model the evolution of TRAPPIST-1c's atmosphere. We find that the stellar wind stripping that is expected to occur on TRAPPIST-1c over its lifetime can only remove up to ∼16 bar of CO2, less than the modern CO2 inventory of either Earth or Venus. Therefore, we infer that TRAPPIST-1c either formed volatile-poor, as compared to Earth and Venus, or lost a substantial amount of CO2 during an early phase of hydrodynamic hydrogen escape. Finally, we scale our results for the other TRAPPIST-1 planets, finding that the more distant TRAPPIST-1 planets may readily retain atmospheres.
Shadows Don't Lie and Lines Can't Bend! Generative Models don't know Projective Geometry...for now

Generative models can produce impressively realistic images. This paper demonstrates that generated images have geometric features different from those of real images. We build a set of collections of generated images, prequalified to fool simple, signal-based classifiers into believing they are real. We then show that prequalified generated images can be identified reliably by classifiers that only look at geometric properties. We use three such classifiers. All three classifiers are denied access to image pixels, and look only at derived geometric features. The first classifier looks at the perspective field of the image, the second looks at lines detected in the image, and the third looks at relations between detected objects and shadows. Our procedure detects generated images more reliably than SOTA local signal based detectors, for images from a number of distinct generators. Saliency maps suggest that the classifiers can identify geometric problems reliably. We conclude that current generators cannot reliably reproduce geometric properties of real images.
WyCryst: Wyckoff Inorganic Crystal Generator Framework

Generative design marks a significant data-driven advancement in the exploration of novel inorganic materials, which entails learning the symmetry equivalent to the crystal structure prediction (CSP) task and subsequent learning of their target properties. Generative models have been developed in the last few years that use custom Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and diffusion models. While periodicity and global Euclidian symmetry in three dimensions through translations, rotations and reflections have recently been accounted for, symmetry constraints within allowed space groups have not. This is especially important because the final step involves energy relaxation on the generated crystal structures to find the relaxed crystal structure, typically using Density Functional Theory (DFT). To address this explicitly, we introduce a generative design framework (WyCryst), composed of three pivotal components: 1) a Wyckoff position based inorganic crystal representation, 2) a property-directed VAE model and 3) an automated DFT workflow for structure refinement. Our model selectively generates materials that follow the ground truth of unit cell space group symmetry by encoding the Wyckoff representation for each space group. We successfully reproduce a variety of existing materials: CaTiO3 (space group, SG No. 62 and 221), CsPbI3 (SG No. 221), BaTiO3 (SG No. 160), and CuInS2 (SG No.122) for both ground state as well as polymorphic structure predictions. We also generate several new ternary materials not found in the inorganic materials database (Materials Project), which are proved to be stable, retaining their symmetry, and we also check their phonon stability, using our automated DFT workflow highlighting the validity of our approach. We believe our symmetry-aware WyCryst takes a vital step towards AI-driven inorganic materials discovery.
Interannual Variability of the Kuroshio Extension System and Its Impact on the Wintertime SST Field

Altimetry data from the first 7-yr TOPEX/Poseidon (T/P) mission (October 1992‐September 1999) are analyzed to investigate the interannual variability in regions of the Kuroshio Extension and its southern recirculation gyre. Large-scale, interannual changes of the Kuroshio Extension system are characterized by the oscillation between an elongated state and a contracted state. In the elongated state, the Kuroshio Extension has a larger eastward surface transport, a greater zonal penetration, and a more northerly zonal-mean path. All these characteristics are closely connected to the presence of an intense, zonally elongated southern recirculation gyre. In its contracted state, the Kuroshio Extension has a smaller eastward surface transport, a more southerly mean path, and is accompanied by a weaker southern recirculation gyre. During the T/P period, the Kuroshio Extension changed from the elongated state in 1992‐93 to the contracted state in 1996‐97, and back to the elongated state since late 1997. In addition to these large-scale changes, the mesoscale eddy field also exhibited prominent interannual changes in the Kuroshio Extension region. In the upstream region between the Izu Ridge and the Shatsky Rise, the eddy kinetic energy level was generally low while the Kuroshio Extension was in its elongated state, and high while in the contracted state. Downstream of the Shatsky Rise, on the other hand, the EKE level was high (low) when the Kuroshio Extension was in the elongated (contracted) state. The large-scale, interannual changes in the Kuroshio Extension system have a significant impact on the regional wintertime SST anomaly field: the warm (cold) wintertime SST anomalies tend to persist in years when the Kuroshio Extension is in its elongated (contracted) state. A diagnostic analysis of the surface ocean heat balance indicates that the nonseasonal geostrophic advection by the ocean circulation works to reduce (increase) the wintertime SST anomalies when the Kuroshio Extension changes from an elongated (contracted) state to a contracted (elongated) state. The SST anomalies associated with the large-scale changes of the Kuroshio Extension have an area-averaged, peak-topeak amplitude of . 18C and appear independent of the interannual SST changes in the tropical Pacific Ocean.
Potentials and fields of a charge set suddenly from rest into uniform motion

The fact that electromagnetic effects propagate at the speed of light suggests how the Lorenz-gauge scalar and vector potentials of a uniformly moving point charge must be modified when the charge was initially at rest and then set suddenly into uniform motion. The modified potentials are shown to satisfy the requisite inhomogeneous wave equations. The gauge function of the transformation of these potentials to the Coulomb gauge is calculated in closed form. It is validated by confirming that the Coulomb-gauge vector potential that is calculated using it yields together with the Coulomb-gauge scalar potential the same electric and magnetic fields as those calculated with the Lorenz-gauge potentials.
Direct Fabrication of Atomically Defined Pores in MXenes

Controlled fabrication of nanopores in atomically thin two-dimensional material offers the means to create robust membranes needed for ion transport, nanofiltration, and DNA sensing. Techniques for creating nanopores have relied upon either plasma etching or direct irradiation using electrons or ions; however, aberration-corrected scanning transmission electron microscopy (STEM) offers the advantage of combining a highly energetic, sub-angstrom sized electron beam for atomic manipulation along with atomic resolution imaging. Here, we utilize a method for automated nanopore fabrication with real-time atomic visualization to enhance our mechanistic understanding of beam-induced transformations. Additionally, an electron beam simulation technique, Electron-Beam Simulator (E-BeamSim) was developed to observe the atomic movements and interactions resulting from electron beam irradiation. Using the 2D MXene Ti3C2Tx, we explore the influence of temperature on nanopore fabrication by tracking atomic transformation pathways and find that at room temperature, electron beam irradiation induces random displacement of atoms and results in a pileup of titanium atoms at the nanopore edge. This pileup was confirmed and demonstrated in E-BeamSim simulations around the small, milled area in the MXene monolayer. At elevated temperatures, the surface functional groups on MXene are effectively removed, and the mobility of atoms increases, which results in atomic transformations that lead to the selective removal of atoms layer by layer. Through controllable manufacture using e-beam milling fabrication, the production and then characterization of the fabricated defects can be better understood for future work. This work can lead to the development of defect engineering techniques within functionalized MXene layers.
Design of platform trials with a change in the control treatment arm

Platform trials are a more efficient way of testing multiple treatments compared to running separate trials. In this paper we consider platform trials where, if a treatment is found to be superior to the control, it will become the new standard of care (and the control in the platform). The remaining treatments are then tested against this new control. In such a setting, one can either keep the information on both the new standard of care and the other active treatments before the control is changed or one could discard this information when testing for benefit of the remaining treatments. We will show analytically and numerically that retaining the information collected before the change in control can be detrimental to the power of the study. Specifically, we consider the overall power, the probability that the active treatment with the greatest treatment effect is found during the trial. We also consider the conditional power of the active treatments, the probability a given treatment can be found superior against the current control. We prove when, in a multi-arm multi-stage trial where no arms are added, retaining the information is detrimental to both overall and conditional power of the remaining treatments. This loss of power is studied for a motivating example. We then discuss the effect on platform trials in which arms are added later. On the basis of these observations we discuss different aspects to consider when deciding whether to run a continuous platform trial or whether one may be better running a new trial.
Path integral molecular dynamics approximations of quantum canonical observables

Mean-field molecular dynamics based on path integrals is used to approximate canonical quantum observables for particle systems consisting of nuclei and electrons. A computational bottleneck is the sampling from the Gibbs density of the electron operator, which due to the fermion sign problem has a computational complexity that scales exponentially with the number of electrons. In this work we construct an algorithm that approximates the mean-field Hamiltonian by path integrals for fermions. The algorithm is based on the determinant of a matrix with components based on Brownian bridges connecting permuted electron coordinates. The computational work for $n$ electrons is $\mathcal O(n^3)$, which reduces the computational complexity associated with the fermion sign problem. We analyze a bias resulting from this approximation and provide a computational error indicator. It remains to rigorously explain the surprisingly high accuracy.
Exploring Multi-Reader Buffers and Channel Placement During Dataflow Network Mapping to Heterogeneous Many-Core Systems

This paper presents an approach for reducing the memory requirements of periodically executed dataflow applications, while minimizing the period when deployed on a many-core target. Often, implementations of dataflow applications suffer from data duplication if identical data has to be processed by multiple actors. In fact, multi-cast (also called fork) actors can produce huge memory overheads when storing and communicating copies of the same data. As a remedy, so-called Multi-Reader Buffers (MRBs) can be utilized to forward identical data to multiple actors in a First In First Out (FIFO) manner while storing each data item only once by sharing. However, using MRBs may increase the achievable period due to contention when accessing the shared data. This paper proposes a novel multi-objective design space exploration approach that selectively replaces multi-cast actors with MRBs and explores actor and FIFO channel mappings to find trade-offs between the objectives of period, memory footprint, and core cost. In distinction to the state-of-the-art, our approach considers (i) memory-size constraints for on-chip memories, (ii) hierarchical memories to implement the buffers, e.g., tile-local memories, (iii) supports heterogeneous many-core platforms, i.e., core-type dependent actor execution times, and (iv) optimizes the buffer placement and overall scheduling to minimize the execution period by proposing a novel combined actor and communication scheduling heuristic for period minimization called Communication-Aware Periodic Scheduling on Heterogeneous Many-core Systems (CAPS-HMS). Our results show that the explored Pareto fronts improve a hypervolume indicator over a reference approach by up to 66 % for small to mid-size applications and 90 % for large applications. Moreover, selectively replacing multi-cast actors with corresponding MRBs proves to be always superior to never or always replacing them. Finally, it is shown that the quality of the explored Pareto fronts does not degrade when replacing the efficient scheduling heuristic CAPS-HMS by an Integer Linear Program (ILP) solver that requires orders of magnitude higher solver times and thus cannot be applied to large scale dataflow network problems.
U-Net v2: Rethinking the Skip Connections of U-Net for Medical Image Segmentation

In this paper, we introduce U-Net v2, a new robust and efficient U-Net variant for medical image segmentation. It aims to augment the infusion of semantic information into low-level features while simultaneously refining high-level features with finer details. For an input image, we begin by extracting multi-level features with a deep neural network encoder. Next, we enhance the feature map of each level by infusing semantic information from higher-level features and integrating finer details from lower-level features through Hadamard product. Our novel skip connections empower features of all the levels with enriched semantic characteristics and intricate details. The improved features are subsequently transmitted to the decoder for further processing and segmentation. Our method can be seamlessly integrated into any Encoder-Decoder network. We evaluate our method on several public medical image segmentation datasets for skin lesion segmentation and polyp segmentation, and the experimental results demonstrate the segmentation accuracy of our new method over state-of-the-art methods, while preserving memory and computational efficiency. Code is available at: https://github.com/yaoppeng/U-Net\_v2
Pose Anything: A Graph-Based Approach for Category-Agnostic Pose Estimation

Traditional 2D pose estimation models are limited by their category-specific design, making them suitable only for predefined object categories. This restriction becomes particularly challenging when dealing with novel objects due to the lack of relevant training data. To address this limitation, category-agnostic pose estimation (CAPE) was introduced. CAPE aims to enable keypoint localization for arbitrary object categories using a single model, requiring minimal support images with annotated keypoints. This approach not only enables object pose generation based on arbitrary keypoint definitions but also significantly reduces the associated costs, paving the way for versatile and adaptable pose estimation applications. We present a novel approach to CAPE that leverages the inherent geometrical relations between keypoints through a newly designed Graph Transformer Decoder. By capturing and incorporating this crucial structural information, our method enhances the accuracy of keypoint localization, marking a significant departure from conventional CAPE techniques that treat keypoints as isolated entities. We validate our approach on the MP-100 benchmark, a comprehensive dataset comprising over 20,000 images spanning more than 100 categories. Our method outperforms the prior state-of-the-art by substantial margins, achieving remarkable improvements of 2.16% and 1.82% under 1-shot and 5-shot settings, respectively. Furthermore, our method's end-to-end training demonstrates both scalability and efficiency compared to previous CAPE approaches.
Optimal EEG Electrode Set for Emotion Recognition From Brain Signals: An Empirical Quest

The human brain is a complex organ, still completely undiscovered, that controls almost all the parts of the body. Apart from survival, the human brain stimulates emotions. Recent research indicates that brain signals can be very effective for emotion recognition. However, which parts of the brain exhibit most of the emotions is still under-explored. In this study, we empirically analyze the contribution of each part of the brain in exhibiting emotions. We use the DEAP dataset to find the most optimal electrode set which eventually leads to the effective brain part associated with emotions. We use Fast Fourier Transformation for effective feature extraction and a 1D-CNN with residual connection for classification. Though 32 electrodes from the DEAP dataset got an accuracy of 97.34%, only 12 electrodes (F7, P8, O1, F8, C4, T7, PO3, Fp1, Fp2, O2, P3, and Fz) achieve 95.81% accuracy. This study also shows that adding more than 10 electrodes does not improve performance significantly. Moreover, the frontal lobe is the most important for recognizing emotion.