Miracle-dz/UniFAR-FTUs huggingface dataset:
Joint Representation Learning for Text and 3D Point Cloud

Recent advancements in vision-language pre-training (e.g. CLIP) have shown that vision models can benefit from language supervision. While many models using language modality have achieved great success on 2D vision tasks, the joint representation learning of 3D point cloud with text remains under-explored due to the difficulty of 3D-Text data pair acquisition and the irregularity of 3D data structure. In this paper, we propose a novel Text4Point framework to construct language-guided 3D point cloud models. The key idea is utilizing 2D images as a bridge to connect the point cloud and the language modalities. The proposed Text4Point follows the pre-training and fine-tuning paradigm. During the pre-training stage, we establish the correspondence of images and point clouds based on the readily available RGB-D data and use contrastive learning to align the image and point cloud representations. Together with the well-aligned image and text features achieved by CLIP, the point cloud features are implicitly aligned with the text embeddings. Further, we propose a Text Querying Module to integrate language information into 3D representation learning by querying text embeddings with point cloud features. For fine-tuning, the model learns task-specific 3D representations under informative language guidance from the label set without 2D images. Extensive experiments demonstrate that our model shows consistent improvement on various downstream tasks, such as point cloud semantic segmentation, instance segmentation, and object detection. The code will be available here: https://github.com/LeapLabTHU/Text4Point
Beating the Best: Improving on AlphaFold2 at Protein Structure Prediction

The goal of Protein Structure Prediction (PSP) problem is to predict a protein's 3D structure (confirmation) from its amino acid sequence. The problem has been a ‘holy grail’ of science since the Noble prize-winning work of Anfinsen demonstrated that protein conformation was determined by sequence. A recent and important step towards this goal was the development of AlphaFold2, currently the best PSP method. AlphaFold2 is probably the highest profile application of AI to science. Both AlphaFold2 and RoseTTAFold (another impressive PSP method) have been published and placed in the public domain (code & models). Stacking is a form of ensemble machine learning ML in which multiple baseline models are first learnt, then a meta-model is learnt using the outputs of the baseline level model to form a model that outperforms the base models. Stacking has been successful in many applications. We developed the ARStack PSP method by stacking AlphaFold2 and RoseTTAFold. ARStack significantly outperforms AlphaFold2. We rigorously demonstrate this using two sets of non-homologous proteins, and a test set of protein structures published after that of AlphaFold2 and RoseTTAFold. As more high quality prediction methods are published it is likely that ensemble methods will increasingly outperform any single method.
A Simple Method for Predicting N H Variability in Active Galactic Nuclei

The unified model of active galactic nuclei (AGNs) includes a geometrically thick obscuring medium to explain the differences between type I and type II AGNs as an effect of inclination angle. This medium is often referred to as the torus and is thought to be “clumpy” as the line-of-sight column density, N H, has been observed to vary in time for many sources. We present a method which uses a variation in the hardness ratio to predict whether an AGN has experienced N H variability across different observations. We define two sets of hard and soft bands that are chosen to be sensitive to the energies most affected by changes in N H. We calculate hardness ratios for Chandra and XMM-Newton observations of a sample of 12 sources with multiple observations, and compare the predictions of this method to the N H values obtained from spectral fitting with physically motivated torus models (borus02, MYTorus, and UXCLUMPY). We also provide a calibrated correction factor that allows comparison between Chandra and XMM-Newton observations which is otherwise not possible due to differences in the instrument response functions. The sensitivity of this method can be easily adjusted. As we decrease the sensitivity, we find that the false positive rate becomes small while the true positive rate remains above 0.5. We also test the method on simulated data and show that it remains reliable for observations with as few as 100 counts. Therefore, we conclude that the method proposed in this work is effective in preselecting sources for variability studies.
FPANet: Frequency-based Video Demoireing using Frame-level Post Alignment

Interference between overlapping gird patterns creates moire patterns, degrading the visual quality of an image that captures a screen of a digital display device by an ordinary digital camera. Removing such moire patterns is challenging due to their complex patterns of diverse sizes and color distortions. Existing approaches mainly focus on filtering out in the spatial domain, failing to remove a large-scale moire pattern. In this paper, we propose a novel model called FPANet that learns filters in both frequency and spatial domains, improving the restoration quality by removing various sizes of moire patterns. To further enhance, our model takes multiple consecutive frames, learning to extract frame-invariant content features and outputting better quality temporally consistent images. We demonstrate the effectiveness of our proposed method with a publicly available large-scale dataset, observing that ours outperforms the state-of-the-art approaches, including ESDNet, VDmoire, MBCNN, WDNet, UNet, and DMCNN, in terms of the image and video quality metrics, such as PSNR, SSIM, LPIPS, FVD, and FSIM.
AppealMod: Shifting Effort from Moderators to Users Making Appeals

As content moderation becomes a central aspect of all social media platforms and online communities, interest has grown in how to make moderation decisions contestable. On social media platforms where individual communities moderate their own activities, the responsibility to address user appeals falls on volunteers from within the community. While there is a growing body of work devoted to understanding and supporting the volunteer moderators’ workload, little is known about their practice of handling user appeals. Through a collaborative and iterative design process with Reddit moderators, we found that moderators spend considerable effort in investigating user ban appeals and desired to directly engage with users and retain their agency over each decision. To fulfill their needs, we designed and built AppealMod, a system that asks users to put more effort in their appeals, by providing additional information, before their appeals are reviewed by human moderators. In addition to giving moderators more information, we expected the friction in the appeal process would lead to a selection effect among users, with many insincere and toxic appeals being abandoned before getting any attention from human moderators. To evaluate our system, we conducted a field experiment in a Reddit community of over 29 million users that lasted for four months. As a result of the selection effect, moderators viewed only 30% of initial appeals and less than 10% of the toxically worded appeals; yet they granted roughly the same number of appeals. Overall, our system is effective at reducing moderator workload and minimizing their exposure to toxic content while honoring their preference for direct engagement and agency in appeals.
Information-theoretical analysis of the neural code for decoupled face representation

Processing faces accurately and efficiently is a key capability of humans and other animals that engage in sophisticated social tasks. Recent studies reported a decoupled coding for faces in the primate inferotemporal cortex, with two separate neural populations coding for the geometric position of (texture-free) facial landmarks and for the image texture at fixed landmark positions, respectively. Here, we formally assess the efficiency of this decoupled coding by appealing to the information-theoretic notion of description length, which quantifies the amount of information that is saved when encoding novel facial images, with a given precision. We show that despite decoupled coding describes the facial images in terms of two sets of principal components (of landmark shape and image texture), it is more efficient (i.e., yields more information compression) than the encoding in terms of the image principal components only, which corresponds to the widely used eigenface method. The advantage of decoupled coding over eigenface coding increases with image resolution and is especially prominent when coding variants of training set images that only differ in facial expressions. Moreover, we demonstrate that decoupled coding entails better performance in three different tasks: the representation of facial images, the (daydream) sampling of novel facial images, and the recognition of facial identities and gender. In summary, our study provides a first principle perspective on the efficiency and accuracy of the decoupled coding of facial stimuli reported in the primate inferotemporal cortex.
Deep Unsupervised Phase-based 3D Incompressible Motion Estimation in Tagged-MRI

Tagged magnetic resonance imaging (MRI) has been used for decades to observe and quantify the detailed motion of deforming tissue. However, this technique faces several challenges such as tag fading, large motion, long computation times, and diﬃculties in obtaining diﬀeomorphic incompressible ﬂow ﬁelds. To address these issues, this paper presents a novel unsupervised phase-based 3D motion estimation technique for tagged MRI. We introduce two key innovations. First, we apply a sinusoidal transformation to the harmonic phase input, which enables end-to-end training and avoids the need for phase interpolation. Second, we propose a Jacobian determinant-based learning objective to encourage incompressible ﬂow ﬁelds for deforming biological tissues. Our method eﬃciently estimates 3D motion ﬁelds that are accurate, dense, and approximately diﬀeomorphic and incompressible. The eﬃcacy of the method is assessed using human tongue motion during speech, and includes both healthy controls and patients that have undergone glossectomy. We show that the method outperforms existing approaches, and also exhibits improvements in speed, robustness to tag fading, and large tongue motion.
Extensions for Shared Resource Orchestration in Kubernetes to Support RT-Cloud Containers

— Industries are considering the adoption of cloud computing for real-time applications due to current improvements in network latencies and the advent of Fog and Edge computing. To create an RT-cloud capable of hosting real-time applications, it is increasingly signi ﬁ cant to improve the entire stack, including containerization of applications, and their deployment and orchestration across nodes. However, state-of-the-art orchestrators (e.g., Kubernetes) and underlying container engines are designed for general purpose applications. They ignore orchestration and management of shared resources (e.g. memory bandwidth, cache, shared-interconnect) making them unsuitable for use with an RT-cloud. Taking inspiration from existing resource management architectures for multicore nodes, such as ACTORS, and for distributed mixed-criticality systems, such as the DREAMS, we propose a series of extensions in the way shared resources are orchestrated by Kubernetes and managed by the underlying Linux layers. Our approach allows ﬁ ne-grained monitoring and allocation of low-level shared resources on nodes to provide better isolation to real-time containers and supports dynamic orchestration and balancing of containers across the nodes based on the availability and demand of shared resources.
Three aspects of the MSTCI problem

Consider a connected graph G and let T be a spanning tree of G . Every edge e ∈ G − T induces a cycle in T ∪ { e } . The intersection of two distinct such cycles is the set of edges of T that belong to both cycles. The MSTCI problem consists in ﬁnding a spanning tree that has the least number of such non-empty intersections and the instersection number is the number of non-empty intersections of a solution. In this article we consider three aspects of the problem in a general context (i.e. for arbitrary connected graphs). The ﬁrst presents two lower bounds of the intersection number. The second compares the intersection number of graphs that diﬀer in one edge. The last is an attempt to generalize a recent result for graphs with a universal vertex.
Coupling spin defects in hexagonal boron nitride to a microwave cavity

Optically addressable spin defects in hexagonal boron nitride (hBN) have become a promising platform for quantum sensing. While sensitivity of these defects is limited by their interactions with the spin environment in hBN, inefficient microwave delivery can further reduce their sensitivity. Here, we design and fabricate a microwave double arc resonator for efficient transferring of the microwave field at 3.8 GHz. The spin transitions in the ground state of VB− are coupled to the frequency of the microwave cavity, which result in enhanced optically detected magnetic resonance (ODMR) contrast. In addition, the linewidth of the ODMR signal further reduces, achieving a magnetic field sensitivity as low as 42.4 μT/√Hz. Our robust and scalable device engineering is promising for future employment of spin defects in hBN for quantum sensing.
Threats, Vulnerabilities, and Controls of Machine Learning Based Systems: A Survey and Taxonomy

In this article, we propose the Artificial Intelligence Security Taxonomy to systematize the knowledge of threats, vulnerabilities, and security controls of ML-based systems. We first classify the damage caused by attacks against ML-based systems, define ML-specific security, and discuss its characteristics. Next, we enumerate all relevant assets and stakeholders and provide a general taxonomy for ML-specific threats. Then, we collect a wide range of security controls against ML-specific threats through an extensive review of recent literature. Finally, we classify the vulnerabilities and controls of an ML-based system in terms of each vulnerable asset in the system’s entire lifecycle.
Revisiting mass-radius relationships for exoplanet populations: a machine learning insight

The growing number of exoplanet discoveries and advances in machine learning techniques have opened new avenues for exploring and understanding the characteristics of worlds beyond our Solar System. In this study, we employ efficient machine learning approaches to analyze a dataset comprising 762 confirmed exoplanets and eight Solar System planets, aiming to characterize their fundamental quantities. By applying different unsupervised clustering algorithms, we classify the data into two main classes: ‘small’ and ‘giant’ planets, with cut-off values at Rp = 8.13R⊕ and Mp = 52.48M⊕. This classification reveals an intriguing distinction: giant planets have lower densities, suggesting higher H-He mass fractions, while small planets are denser, composed mainly of heavier elements. We apply various regression models to uncover correlations between physical parameters and their predictive power for exoplanet radius. Our analysis highlights that planetary mass, orbital period, and stellar mass play crucial roles in predicting exoplanet radius. Among the models evaluated, the Support Vector Regression consistently outperforms others, demonstrating its promise for obtaining accurate planetary radius estimates. Furthermore, we derive parametric equations using the M5P and Markov Chain Monte Carlo methods. Notably, our study reveals a noteworthy result: small planets exhibit a positive linear mass-radius relation, aligning with previous findings. Conversely, for giant planets, we observe a strong correlation between planetary radius and the mass of their host stars, which might provide intriguing insights into the relationship between giant planet formation and stellar characteristics.
Observable Signature of Merging Stellar-mass Black Holes in Active Galactic Nuclei

The origin of stellar-mass black hole mergers discovered through gravitational waves is being widely debated. Mergers in the disks of active galactic nuclei (AGNs) represent a promising source of origin, with possible observational clues in the gravitational-wave data. Beyond gravitational waves, a unique signature of AGN-assisted mergers is electromagnetic emission from the accreting black holes. Here we show that jets launched by accreting black holes merging in an AGN disk can be detected as peculiar transients by infrared, optical, and X-ray observatories. We further show that this emission mechanism can explain the possible associations between gravitational-wave events and the optical transient ZTF 19abanrhr and the proposed gamma-ray counterparts GW150914-GBM and LVT151012-GBM. We demonstrate how these associations, if genuine, can be used to reconstruct the properties of these events’ environments. Searching for infrared and X-ray counterparts to similar electromagnetic transients in the future, once host galaxies are localized by optical observations, could provide a smoking-gun signature of the mergers’ AGN origin.
A GRU-Based Dynamic Generative Factor Model for CVaR Portfolio Optimization

The dynamic portfolio construction problem requires dynamic modeling of the joint distribution of multivariate stock returns. To achieve this, we pro-pose a dynamic generative factor model which uses random variable transformation as an implicit way of distribution modeling and relies on the GRU network for the dynamic modeling. The proposed model captures the dynamic dependence among multivariate stock returns, especially focusing on the tail-side properties. We also propose a two-step iterative algorithm to train the model and then predict the time-varying model parameters (including the time-invariant tail parameters). At each time, we can easily generate new samples from the learned generative model, and we further perform CVaR portfolio optimization with the samples to form a dynamic portfolio strategy. Numerical experiments on stock data show that our strategy us-ing the proposed model leads to a wiser investment that promises a higher reward while presenting lower tail risk and smaller maximum drawdown.
TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks

The apparent "black box" nature of neural networks is a barrier to adoption in applications where explainability is essential. This paper presents TAME (Trainable Attention Mechanism for Explanations)1, a method for generating explanation maps with a multi-branch hierarchical attention mechanism. TAME combines a target model’s feature maps from multiple layers using an attention mechanism, transforming them into an explanation map. TAME can easily be applied to any convolutional neural network (CNN) by streamlining the optimization of the attention mechanism’s training method and the selection of target model’s feature maps. After training, explanation maps can be computed in a single forward pass. We apply TAME to two widely used models, i.e. VGG-16 and ResNet-50, trained on ImageNet and show improvements over previous top-performing methods. We also provide a comprehensive ablation study comparing the performance of different variations of TAME’s architecture.
Transit timing variation analysis of the low-mass brown dwarf KELT-1 b

We investigate whether there is a variation in the orbital period of the short-period brown dwarf-mass KELT-1 b, which is one of the best candidates to observe orbital decay. We obtain 19 high-precision transit light curves of the target using six different telescopes. We add all precise and complete transit light curves from open databases and the literature, as well as the available TESS observations from sectors 17 and 57, to form a transit timing variation (TTV) diagram spanning more than 10 years of observations. The analysis of the TTV diagram, however, is inconclusive in terms of a secular or periodic variation, hinting that the system might have synchronized. We update the transit ephemeris and determine an informative lower limit for the reduced tidal quality parameter of its host star of Q$_{\star }^{\prime } > (8.5 \pm 3.9) \times 10^{6}$ assuming that the stellar rotation is not yet synchronised. Using our new photometric observations, published light curves, the TESS data, archival radial velocities and broadband magnitudes, we also update the measured parameters of the system. Our results are in good agreement with those found in previous analyses.
Constraining Fundamental Parameters in Modified Gravity Using Gaia-DR2 Massive White Dwarf Observations

Various experiments and observations have led researchers to suggest different bounds on fundamental constants like the fine-structure constant and the proton-to-electron mass ratio. These bounds differ mostly due to the energy scale of the systems where the experiments are performed. In this article, we obtain bounds on these parameters in the modified gravity context using Gaia-DR2 massive white dwarf data and show that the bounds alter as the gravity theory changes. This exploration not only indicates strong support for nonnegligible influences of modified gravity in astrophysical scenarios in high-density regimes but also reveals that the bounds on the fundamental parameters can be much stronger under alternate gravity theories.
A variational autoencoder-based nonnegative matrix factorisation model for deep dictionary learning

Construction of dictionaries using nonnegative matrix factorisation (NMF) has extensive applications in signal processing and machine learning. With the advances in deep learning, training compact and robust dictionaries using deep neural networks, i.e., dictionaries of deep features, has been proposed. In this study, we propose a probabilistic generative model which employs a variational autoencoder (VAE) to perform nonnegative dictionary learning. In contrast to the existing VAE models, we cast the model under a statistical framework with latent variables obeying a Gamma distribution and design a new loss function to guarantee the nonnegative dictionaries. We adopt an acceptance-rejection sampling reparameterization trick to update the latent variables iteratively. We apply the dictionaries learned from VAE-NMF to two signal processing tasks, i.e., enhancement of speech and extraction of muscle synergies. Experimental results demonstrate that VAE-NMF performs better in learning the latent nonnegative dictionaries in comparison with state-of-the-art methods.
Causal Falsification of Digital Twins

Digital twins are virtual systems designed to predict how a real-world process will evolve in response to interventions. This modelling paradigm holds substantial promise in many applications, but rigorous procedures for assessing their accuracy are essential for safety-critical settings. We consider how to assess the accuracy of a digital twin using real-world data. We formulate this as causal inference problem, which leads to a precise definition of what it means for a twin to be"correct"appropriate for many applications. Unfortunately, fundamental results from causal inference mean observational data cannot be used to certify that a twin is correct in this sense unless potentially tenuous assumptions are made, such as that the data are unconfounded. To avoid these assumptions, we propose instead to find situations in which the twin is not correct, and present a general-purpose statistical procedure for doing so. Our approach yields reliable and actionable information about the twin under only the assumption of an i.i. d. dataset of observational trajectories, and remains sound even if the data are confounded. We apply our methodology to a large-scale, real-world case study involving sepsis modelling within the Pulse Physiology Engine, which we assess using the MIMIC-III dataset of ICU patients.