Miracle-dz/UniFAR-FTUs huggingface dataset:
Quasar UV Luminosity Function at 3.5 < z < 5.0 from SDSS Deep Imaging Data

We present a well-designed sample of more than 1000 type 1 quasars at 3.5 < z < 5 and derive UV quasar luminosity functions (QLFs) in this redshift range. These quasars were selected using the Sloan Digital Sky Survey (SDSS) imaging data in the Stripe 82 and overlap regions with repeat imaging observations that are about 1 mag fainter than the SDSS single-epoch data. The follow-up spectroscopic observations were conducted by the SDSS-III Baryon Oscillation Spectroscopic Survey (BOSS) as one of the BOSS ancillary programs. Reaching i ∼ 21.5 mag, our sample bridges previous samples from brighter and deeper surveys. We use a 1/V a method to derive binned QLFs at 3.6 < z < 4.0, 4.0 < z < 4.5, and 4.5 < z < 4.9 and then use a double power-law model to parameterize the QLFs. We also combine our data with literature QLFs to better constrain the QLFs across a much wider luminosity baseline. The faint- and bright-end slopes of the QLFs in this redshift range are around −1.7 and −3.7, respectively, with uncertainties from 0.2 to 0.3 to >0.5. The evolution of the QLFs from z ∼ 5 to 3.5 can be described by a pure density evolution model (∝10 kz ) with a parameter k similar to that at 5 < z < 7, suggesting a nearly uniform evolution of the quasar density at z = 3.5–7.
Burton’s Curse: The Impact of Bulk Flows on the Galactic Longitude–Velocity Diagram and the Illusion of a Continuous Perseus Arm

In this work, we demonstrate that the Perseus Arm is not a continuous structure of molecular gas in the second quadrant. We first show that the observed, distanced-resolved velocity structure of the Galaxy in the outer disk is capable of creating illusory spiral arms, as was first proposed by Burton. Second, we measure the distances to a collection of CO clouds at velocities consistent with the Perseus arm with 135° < l < 160°. We find these distances using 3D dust maps from Green et al. We determine that these molecular clouds do not preferentially lie at the distance of a purported Perseus arm but rather extend over 3 kpc in distance, with some evidence for a closer, high pitch angle structure between 1 and 1.5 kpc away. Finally, we demonstrate that velocity perturbations of the amplitude found near the Perseus arm can wreak havoc on our interpretation of the longitude–velocity diagram for more than half of the Milky Way disk.
Efficient Geometry-aware 3D Generative Adversarial Networks

Unsupervised generation of high-quality multi-view-consistent images and 3D shapes using only collections of single-view 2D photographs has been a long-standing challenge. Existing 3D GANs are either compute intensive or make approximations that are not 3D-consistent; the former limits quality and resolution of the generated images and the latter adversely affects multi-view consistency and shape quality. In this work, we improve the computational efficiency and image quality of 3D GANs without overly relying on these approximations. We introduce an expressive hybrid explicit implicit network architecture that, together with other design choices, synthesizes not only high-resolution multi-view-consistent images in real time but also produces high-quality 3D geometry. By decoupling feature generation and neural rendering, our framework is able to leverage state-of-the-art 2D CNN generators, such as StyleGAN2, and inherit their efficiency and expressiveness. We demonstrate state-of-the-art 3D-aware synthesis with FFHQ and AFHQ Cats, among other experiments.
CentSmoothie: Central-Smoothing Hypergraph Neural Networks for Predicting Drug-Drug Interactions

—Predicting drug-drug interactions (DDI) is the problem of predicting side effects (unwanted outcomes) of a pair of drugs using drug information and known side effects of many pairs. This problem can be formulated as predicting labels (i.e. side effects) for each pair of nodes in a DDI graph, of which nodes are drugs and edges are interacting drugs with known labels. State-of-the-art methods for this problem are graph neural networks (GNNs), which leverage neighborhood information in the graph to learn node representations. For DDI, however, there are many labels with complicated relationships due to the nature of side effects. Usual GNNs often ﬁx labels as one-hot vectors that do not reﬂect label relationships and potentially do not obtain the highest performance in the difﬁcult cases of infrequent labels. In this paper, we formulate DDI as a hypergraph where each hyperedge is a triple: two nodes for drugs and one node for a label. We then present CentSmoothie , a hypergraph neural network that learns representations of nodes and labels altogether with a novel ’central-smoothing’ formulation. We empirically demonstrate the performance advantages of CentSmoothie in simulations as well as real datasets.
Dynamic Human Evaluation for Relative Model Comparisons

Collecting human judgements is currently the most reliable evaluation method for natural language generation systems. Automatic metrics have reported flaws when applied to measure quality aspects of generated text and have been shown to correlate poorly with human judgements. However, human evaluation is time and cost-intensive, and we lack consensus on designing and conducting human evaluation experiments. Thus there is a need for streamlined approaches for efficient collection of human judgements when evaluating natural language generation systems. Therefore, we present a dynamic approach to measure the required number of human annotations when evaluating generated outputs in relative comparison settings. We propose an agent-based framework of human evaluation to assess multiple labelling strategies and methods to decide the better model in a simulation and a crowdsourcing case study. The main results indicate that a decision about the superior model can be made with high probability across different labelling strategies, where assigning a single random worker per task requires the least overall labelling effort and thus the least cost.
Model Stealing Attacks Against Inductive Graph Neural Networks

Many real-world data come in the form of graphs. Graph neural networks (GNNs), a new family of machine learning (ML) models, have been proposed to fully leverage graph data to build powerful applications. In particular, the inductive GNNs, which can generalize to unseen data, become mainstream in this direction. Machine learning models have shown great potential in various tasks and have been deployed in many real-world scenarios. To train a good model, a large amount of data as well as computational resources are needed, leading to valuable intellectual property. Previous research has shown that ML models are prone to model stealing attacks, which aim to steal the functionality of the target models. However, most of them focus on the models trained with images and texts. On the other hand, little attention has been paid to models trained with graph data, i.e., GNNs. In this paper, we fill the gap by proposing the first model stealing attacks against inductive GNNs. We systematically define the threat model and propose six attacks based on the adversary’s background knowledge and the responses of the target models. Our evaluation on six benchmark datasets shows that the proposed model stealing attacks against GNNs achieve promising performance.1
Putting People in their Place: Monocular Regression of 3D People in Depth

Given an image with multiple people, our goal is to directly regress the pose and shape of all the people as well as their relative depth. Inferring the depth of a person in an image, however, is fundamentally ambiguous without knowing their height. This is particularly problematic when the scene contains people of very different sizes, e.g. from infants to adults. To solve this, we need several things. First, we develop a novel method to infer the poses and depth of multiple people in a single image. While previous work that estimates multiple people does so by reasoning in the image plane, our method, called BEV, adds an additional imaginary Bird's-Eye-View representation to explicitly reason about depth. BEV reasons simultaneously about body centers in the image and in depth and, by combing these, estimates 3D body position. Unlike prior work, BEV is a single-shot method that is end-to-end differentiable. Second, height varies with age, making it impossible to resolve depth without also estimating the age of people in the image. To do so, we exploit a 3D body model space that lets BEV infer shapes from infants to adults. Third, to train BEV, we need a new dataset. Specifically, we create a “Relative Human” (RH) dataset that includes age labels and relative depth relationships between the people in the images. Extensive experiments on RH and AGORA demonstrate the effectiveness of the model and training scheme. BEV out-performs existing methods on depth reasoning, child shape estimation, and robustness to occlusion. The code11https://github.com/Arthur151/ROMP and dataset22https://github.com/Arthur151/Relative_Human are released for research purposes.
3D Question Answering

Visual question answering (VQA) has experienced tremendous progress in recent years. However, most efforts have only focused on 2D image question-answering tasks. In this article, we extend VQA to its 3D counterpart, 3D question answering (3DQA), which can facilitate a machine's perception of 3D real-world scenarios. Unlike 2D image VQA, 3DQA takes the color point cloud as input and requires both appearance and 3D geometrical comprehension to answer the 3D-related questions. To this end, we propose a novel transformer-based 3DQA framework “3DQA-TR”, which consists of two encoders to exploit the appearance and geometry information, respectively. Finally, the multi-modal information about the appearance, geometry, and linguistic question can attend to each other via a 3D-linguistic Bert to predict the target answers. To verify the effectiveness of our proposed 3DQA framework, we further develop the first 3DQA dataset “ScanQA”, which builds on the ScanNet dataset and contains over 10 K question-answer pairs for 806 scenes. To the best of our knowledge, ScanQA is the first large-scale dataset with natural-language questions and free-form answers in 3D environments that is fully human-annotated. We also use several visualizations and experiments to investigate the astonishing diversity of the collected questions and the significant differences between this task from 2D VQA and 3D captioning. Extensive experiments on this dataset demonstrate the obvious superiority of our proposed 3DQA framework over state-of-the-art VQA frameworks and the effectiveness of our major designs. Our code and dataset will be made publicly available to facilitate research in this direction. The code and data are available at http://shuquanye.com/3DQA_website/.
Assessing $TESS$'s Yield of Rocky Planets Around Nearby M Dwarfs

Terrestrial planets are easier to detect around M dwarfs than other types of stars, making them promising for next-generation atmospheric characterization studies. The TESS mission has greatly increased the number of known M dwarf planets that we can use to perform population studies, allowing us to explore how the rocky planet occurrence rate varies with host radius, following in the footsteps of past work with Kepler data. In this paper, we use simulations to assess TESS’s yield of small ( 0 . 5 R ⊕ < R p < 2 R ⊕ ) planet candidates around nearby ( d < 30 pc) M dwarfs. We highlight the underappreciated fact that while TESS was indeed expected to ﬁnd a large number of planets around M dwarfs overall, it was not expected to have a high planetary yield for the latest M dwarfs. Furthermore, we ﬁnd that TESS has detected fewer planets around stars with R (cid:63) < 0 . 3 R (cid:12) than even was expected (11 observed vs. 24 ± 5 expected). We ﬁnd evidence that the photometric noise of stars in the TESS bandpass increases with decreasing radius for M dwarfs. However, this trend cannot explain the observed distribution of planets. Our main conclusions are: (1) the planet occurrence rate likely doesn’t increase, and may decrease for the latest M dwarfs; and (2) there are at least 17, and potentially three times that number, transiting planets around nearby late M dwarfs that still won’t be detected by the end of TESS’s 4th year.
Spatiotemporal ETAS model with a renewal main‐shock arrival process

We propose a spatiotemporal point process model that enhances the classical Epidemic‐Type Aftershock Sequence (ETAS) model. This is achieved with the introduction of a renewal main‐shock arrival process and we call this extension the renewal ETAS (RETAS) model. This modification is similar in spirit to the renewal Hawkes (RHawkes) process but the conditional intensity process supports a spatial component. It empowers the main‐shock intensity to reset upon the arrival of main‐shocks. This allows for heavier clustering of main‐shocks than the classical spatiotemporal ETAS model. We introduce a likelihood evaluation algorithm for parameter estimation and provide a novel procedure to evaluate the fitted model's goodness‐of‐fit (GOF) based on a sequential application of the Rosenblatt transformation. A simulation algorithm for the RETAS model is outlined and used to validate the numerical performance of the likelihood evaluation algorithm and GOF test procedure. We illustrate the proposed model and methods on various earthquake catalogues around the world each with distinctly different seismic activity. These catalogues demonstrate the RETAS model's additional flexibility in comparison to the classical spatiotemporal ETAS model and emphasizes the potential for superior modelling and forecasting of seismicity.
Antibody-Mediated Immunogenicity Against Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Following Priming, Boosting, and Hybrid Immunity: Insights From 11 Months of Follow-up of a Healthcare Worker Cohort in Israel, December 2020–October 2021

Abstract Background: We determined circulating anti-S SARS-CoV-2 IgG antibody titres in a vaccinated healthcare workers (HCWs) cohort from Northern Israel in the 11 months following primary vaccination according to age, ethnicity, boosting timing and previous infection status. Methods: All consenting HCWs were invited to have their circulating IgG levels measured before vaccination and at 6 subsequent timepoints. All HCWs with suspected COVID-19 were PCR tested. We described trends in circulating IgG geometric mean concentration by age, ethnicity, timing of boosting and previous infection status and compared strata using Kruskall-Wallis tests. Results: Among 985 vaccinated HCWs. IgG titres gradually decreased in all groups over the study duration. Younger or previously infected individuals had higher initial IgG levels (p<0.001 in both cases); differences substantially decreased or disappeared at 7-9 months, before boosting. Pre-infection IgG levels in infected participants were similar to levels measured at the same timepoint in HCWs who remained uninfected (p>0.3). IgG GMC in those boosted 6-7 months after dose 2 was lower compared with those boosted 8-9 months after (1999-vs 2736, p=0.02). Conclusions: Immunity waned 6 months post-priming in all age groups and in previously infected individuals, reversed by boosting. IgG titres decrease among previously infected individuals and the proportion of reinfected individuals in this group, comparable to the proportion of breakthrough infection in previously uninfected individuals suggests individuals with hybrid immunity (infection+vaccination) may also require further doses. Our study also highlights the difficulty in determining protective IgG levels and the need to clarify the optimal timing in 3 dose regimens
Weighted analytic regularity for the integral fractional Laplacian in polygons

We prove weighted analytic regularity of solutions to the Dirichlet problem for the integral fractional Laplacian in polygons with analytic right-hand side. We localize the problem through the Caffarelli-Silvestre extension and study the tangential differentiability of the extended solutions, followed by bootstrapping based on Caccioppoli inequalities on dyadic decompositions of vertex, edge, and edge-vertex neighborhoods.
Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime

Understanding the black-box prediction for neural networks is challenging. To achieve this, early studies have designed influence function (IF) to measure the effect of removing a single training point on neural networks. However, the classic implicit Hessian-vector product (IHVP) method for calculating IF is fragile, and theoretical analysis of IF in the context of neural networks is still lacking. To this end, we utilize the neural tangent kernel (NTK) theory to calculate IF for the neural network trained with regularized mean-square loss, and prove that the approximation error can be arbitrarily small when the width is sufficiently large for two-layer ReLU networks. We analyze the error bound for the classic IHVP method in the over-parameterized regime to understand when and why it fails or not. In detail, our theoretical analysis reveals that (1) the accuracy of IHVP depends on the regularization term, and is pretty low under weak regularization; (2) the accuracy of IHVP has a significant correlation with the probability density of corresponding training points. We further borrow the theory from NTK to understand the IFs better, including quantifying the complexity for influential samples and depicting the variation of IFs during the training dynamics. Numerical experiments on real-world data confirm our theoretical results and demonstrate our findings.
Understanding Feature Transfer Through Representation Alignment

—Training with the true labels of a dataset as opposed to randomized labels leads to faster optimization and better generalization. This difference is attributed to a notion of alignment between inputs and labels in natural datasets. We ﬁnd that training neural networks with different architectures and optimizers on random or true labels enforces the same relationship between the hidden representations and the training labels, elucidating why neural network representations have been so successful for transfer. We ﬁrst highlight why aligned features promote transfer and show in a classic synthetic transfer problem that alignment is the determining factor for positive and negative transfer to similar and dissimilar tasks. We then investigate a variety of neural network architectures and ﬁnd that (a) alignment emerges across a variety of different architectures and optimizers, with more alignment arising from depth (b) alignment increases for layers closer to the output and (c) existing high-performance deep CNNs exhibit high levels of alignment.
Identifying Class Specific Filters with L1 Norm Frequency Histograms in Deep CNNs

Interpretability of Deep Neural Networks has become a major area of exploration. Although these networks have achieved state of the art accuracy in many tasks, it is extremely diﬃcult to interpret and explain their decisions. In this work we analyze the ﬁnal and penultimate layers of Deep Convolutional Networks and provide an eﬃcient method for identifying subsets of features that contribute most towards the network’s decision for a class. We demonstrate that the number of such features per class is much lower in comparison to the dimension of the ﬁnal layer and therefore the decision surface of Deep CNNs lies on a low dimensional manifold and is proportional to the network depth. Our methods allow to decompose the ﬁnal layer into separate subspaces which is far more interpretable and has a lower computational cost as compared to the ﬁnal layer of the full network. We also release our code at https://github.com/ akshaybadola/cnn-class-speciﬁc-ﬁlters-with-histogram
Taming Overconfident Prediction on Unlabeled Data from Hindsight

Minimizing prediction uncertainty on unlabeled data is a key factor to achieve good performance in semi-supervised learning (SSL). The prediction uncertainty is typically expressed as the entropy computed by the transformed probabilities in output space. Most existing works distill low-entropy prediction by either accepting the determining class (with the largest probability) as the true label or suppressing subtle predictions (with the smaller probabilities). Unarguably, these distillation strategies are usually heuristic and less informative for model training. From this discernment, this article proposes a dual mechanism, named adaptive sharpening (ADS), which first applies a soft-threshold to adaptively mask out determinate and negligible predictions, and then seamlessly sharpens the informed predictions, distilling certain predictions with the informed ones only. More importantly, we theoretically analyze the traits of ADS by comparing it with various distillation strategies. Numerous experiments verify that ADS significantly improves state-of-the-art SSL methods by making it a plug-in. Our proposed ADS forges a cornerstone for future distillation-based SSL research.
Detail‐Aware Deep Clothing Animations Infused with Multi‐Source Attributes

This paper presents a novel learning‐based clothing deformation method to generate rich and reasonable detailed deformations for garments worn by bodies of various shapes in various animations. In contrast to existing learning‐based methods, which require numerous trained models for different garment topologies or poses and are unable to easily realize rich details, we use a unified framework to produce high fidelity deformations efficiently and easily. Specifically, we first found that the fit between the garment and the body has an important impact on the degree of folds. We then designed an attribute parser to generate detail‐aware encodings and infused them into the graph neural network, therefore enhancing the discrimination of details under diverse attributes. Furthermore, to achieve better convergence and avoid overly smooth deformations, we proposed to reconstruct output to mitigate the complexity of the learning task. Experimental results show that our proposed deformation method achieves better performance over existing methods in terms of generalization ability and quality of details.
Sensor placement on Canton Tower for health monitoring using asynchronous-climb monkey algorithm

Heuristic optimization algorithms have become a popular choice for solving complex and intricate sensor placement problems which are difficult to solve by traditional methods. This paper proposes a novel and interesting methodology called the asynchronous-climb monkey algorithm (AMA) for the optimum design of sensor arrays for a structural health monitoring system. Different from the existing algorithms, the dual-structure coding method is designed and adopted for the representation of the design variables. The asynchronous-climb process is incorporated in the proposed AMA that can adjust the trajectory of each individual dynamically in the search space according to its own experience and other monkeys. The concept of ‘monkey king’ is introduced in the AMA, which reflects the Darwinian principle of natural selection and can create an interaction network to correctly guide the movement of other monkeys. Numerical experiments are carried out using two different objective functions by considering the Canton Tower in China with or without the antenna mast to evaluate the performance of the proposed algorithm. Investigations have indicated that the proposed AMA exhibits faster convergence characteristics and can generate sensor configurations superior in all instances when compared to the conventional monkey algorithm. For structures with stiffness mutation such as the Canton Tower, the sensor placement needs to be considered for each part separately.