Miracle-dz/UniFAR-FTUs huggingface dataset:
Continual Novelty Detection

Novelty Detection methods identify samples that are not representative of a model's training set thereby flagging misleading predictions and bringing a greater flexibility and transparency at deployment time. However, research in this area has only considered Novelty Detection in the offline setting. Recently, there has been a growing realization in the computer vision community that applications demand a more flexible framework - Continual Learning - where new batches of data representing new domains, new classes or new tasks become available at different points in time. In this setting, Novelty Detection becomes more important, interesting and challenging. This work identifies the crucial link between the two problems and investigates the Novelty Detection problem under the Continual Learning setting. We formulate the Continual Novelty Detection problem and present a benchmark, where we compare several Novelty Detection methods under different Continual Learning settings. We show that Continual Learning affects the behaviour of novelty detection algorithms, while novelty detection can pinpoint insights in the behaviour of a continual learner. We further propose baselines and discuss possible research directions. We believe that the coupling of the two problems is a promising direction to bring vision models into practice.
Numerical influence of ReLU'(0) on backpropagation

In theory, the choice of ReLU (cid:48) (0) in [0 , 1] for a neural network has a negligible inﬂuence both on backpropagation and training. Yet, in the real world, 32 bits default precision combined with the size of deep learning problems makes it a hyperparameter of training methods. We investigate the importance of the value of ReLU (cid:48) (0) for several precision levels (16, 32, 64 bits), on various networks (fully connected, VGG, ResNet) and datasets (MNIST, CIFAR10, SVHN). We observe considerable variations of backpropagation outputs which occur around half of the time in 32 bits precision. The effect disappears with double precision, while it is systematic at 16 bits. For vanilla SGD training, the choice ReLU (cid:48) (0) = 0 seems to be the most efﬁcient. We also evidence that reconditioning approaches as batch-norm or ADAM tend to buffer the inﬂuence of ReLU (cid:48) (0) ’s value. Overall, the message we want to convey is that algorithmic differentiation of nonsmooth problems potentially hides parameters that could be tuned advantageously.
Fold2Seq: A Joint Sequence(1D)-Fold(3D) Embedding-based Generative Model for Protein Design

Designing novel protein sequences for a desired 3D topological fold is a fundamental yet nontrivial task in protein engineering. Challenges exist due to the complex sequence-fold relationship, as well as the difficulties to capture the diversity of the sequences (therefore structures and functions) within a fold. To overcome these challenges, we propose Fold2Seq, a novel transformer-based generative framework for designing protein sequences conditioned on a specific target fold. To model the complex sequence-structure relationship, Fold2Seq jointly learns a sequence embedding using a transformer and a fold embedding from the density of secondary structural elements in 3D voxels. On test sets with single, high-resolution and complete structure inputs for individual folds, our experiments demonstrate improved or comparable performance of Fold2Seq in terms of speed, coverage, and reliability for sequence design, when compared to existing state-of-the-art methods that include data-driven deep generative models and physics-based RosettaDesign. The unique advantages of fold-based Fold2Seq, in comparison to a structure-based deep model and RosettaDesign, become more evident on three additional real-world challenges originating from low-quality, incomplete, or ambiguous input structures. Source code and data are available at https://github.com/IBM/fold2seq.
HyperNeRF

Neural Radiance Fields (NeRF) are able to reconstruct scenes with unprecedented fidelity, and various recent works have extended NeRF to handle dynamic scenes. A common approach to reconstruct such non-rigid scenes is through the use of a learned deformation field mapping from coordinates in each input image into a canonical template coordinate space. However, these deformation-based approaches struggle to model changes in topology, as topological changes require a discontinuity in the deformation field, but these deformation fields are necessarily continuous. We address this limitation by lifting NeRFs into a higher dimensional space, and by representing the 5D radiance field corresponding to each individual input image as a slice through this "hyper-space". Our method is inspired by level set methods, which model the evolution of surfaces as slices through a higher dimensional surface. We evaluate our method on two tasks: (i) interpolating smoothly between "moments", i.e., configurations of the scene, seen in the input images while maintaining visual plausibility, and (ii) novel-view synthesis at fixed moments. We show that our method, which we dub HyperNeRF, outperforms existing methods on both tasks. Compared to Nerfies, HyperNeRF reduces average error rates by 4.1% for interpolation and 8.6% for novel-view synthesis, as measured by LPIPS. Additional videos, results, and visualizations are available at hypernerf.github. io.
Time--dependent equations on networks

We study well posedness of time–dependent Hamilton–Jacobi equations on a network, coupled with a continuous initial datum and a flux limiter. We show existence and uniqueness of solutions as well as stability properties. The novelty of our approach is that comparison results are proved linking the equation to a suitable semidiscrete problem, bypassing doubling variable method. Further, we do not need special test functions, and perform tests relative to the equations on different arcs separately.
Radial Migration from the Metallicity Gradient of Open Clusters and Outliers

Radial migration is an important process in the evolution of the Galactic disk. The metallicity gradient of open clusters and its outliers provide an effective way to examine this process. In this work, we compile metallicity, age, and kinematic parameters for 225 open clusters and carry out a quantitative analysis of radial migration via the calculated migration distances. Based on clusters with age <0.5 Gyr, we obtain a present-day metallicity gradient of −0.074 ± 0.007 dex kpc−1. Three sequences are distributed along this gradient, and clusters in the upper, middle, and lower groups are found to be old outward migrators, in situ clusters, and inward migrators, respectively. The migration distance increases with age, but the time when it is most effective is probably less than 3 Gyr. The metallicity gradient breaks at guiding center radius R g ∼ 11.5 kpc, which is caused by the lack of young open clusters in the outer disk and the presence of old outward migrators in the upper sequence. This shows that this boundary is related to the different effects of radial migration between the inner and outer disks. We also found many special open clusters in and near the outer disk with R > 11 kpc and a steeper metallicity gradient from the inner disk of R g < 7 kpc, which tells of a complicated evolution history of the Galactic disk caused by different effects of stellar radial migration.
Long short-term relevance learning

To incorporate prior knowledge as well as measurement uncertainties in the traditional long short term memory (LSTM) neural networks, an efficient sparse Bayesian training algorithm is introduced to the network architecture. The proposed scheme automatically determines relevant neural connections and adapts accordingly, in contrast to the classical LSTM solution. Due to its flexibility, the new LSTM scheme is less prone to overfitting, and hence can approximate time dependent solutions by use of a smaller data set. On a structural nonlinear finite element application we show that the self-regulating framework does not require prior knowledge of a suitable network architecture and size, while ensuring satisfying accuracy at reasonable computational cost.
A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear-Equality-Constrained Optimization with Rank-Deficient Jacobians

A sequential quadratic optimization algorithm is proposed for solving smooth nonlinear-equality-constrained optimization problems in which the objective function is defined by an expectation. The algorithmic structure of the proposed method is based on a step decomposition strategy that is known in the literature to be widely effective in practice, wherein each search direction is computed as the sum of a normal step (toward linearized feasibility) and a tangential step (toward objective decrease in the null space of the constraint Jacobian). However, the proposed method is unique from others in the literature in that it both allows the use of stochastic objective gradient estimates and possesses convergence guarantees even in the setting in which the constraint Jacobians may be rank-deficient. The results of numerical experiments demonstrate that the algorithm offers superior performance when compared with popular alternatives. Funding: This material is based upon work supported by the U.S. National Science Foundation’s Division of Computing and Communication Foundations under award [CF-1740796], by the Office of Naval Research under award [N00014-21-1-2532], and by the National Science Foundation under award [2030859] to the Computing Research Association for the CIFellows Project.
Spatial-Temporal Graph ODE Networks for Traffic Flow Forecasting

Spatial-temporal forecasting has attracted tremendous attention in a wide range of applications, and traffic flow prediction is a canonical and typical example. The complex and long-range spatial-temporal correlations of traffic flow bring it to a most intractable challenge. Existing works typically utilize shallow graph convolution networks (GNNs) and temporal extracting modules to model spatial and temporal dependencies respectively. However, the representation ability of such models is limited due to: (1) shallow GNNs are incapable to capture long-range spatial correlations, (2) only spatial connections are considered and a mass of semantic connections are ignored, which are of great importance for a comprehensive understanding of traffic networks. To this end, we propose Spatial-Temporal Graph Ordinary Differential Equation Networks (STGODE). 1 Specifically, we capture spatial-temporal dynamics through a tensor-based ordinary differential equation (ODE), as a result, deeper networks can be constructed and spatial-temporal features are utilized synchronously. To understand the network more comprehensively, semantical adjacency matrix is considered in our model, and a well-design temporal dialated convolution structure is used to capture long term temporal dependencies. We evaluate our model on multiple real-world traffic datasets and superior performance is achieved over state-of-the-art baselines.
Robust Decisions for Heterogeneous Agents via Certainty Equivalents

We study the problem of a planner who resolves risk-return trade-offs - like financial investment decisions - on behalf of a collective of agents with heterogeneous risk preferences. The planner's objective is a two-stage utility functional where an outer utility function is applied to the distribution of the agents' certainty equivalents from a given decision. Assuming lognormal risks and heterogeneous power utility preferences for the agents, we characterize optimal behavior in a setting where the planner can let each agent choose between different options from a fixed menu of possible decisions, leading to a grouping of the agents by risk preferences. These optimal decision menus are derived first for the case where the planner knows the distribution of preferences exactly and then for a case where he faces uncertainty about this distribution, only having access to upper and lower bounds on agents' relative risk aversion. Finally, we provide tight bounds on the welfare loss from offering a finite menu of choices rather than fully personalized decisions.
Unusual Electronic Structure of Dirac Material BaMnSb$_2$ Revealed by Angle-Resolved Photoemission Spectroscopy

Hongtao Rong(戎洪涛)1,2,†, Liqin Zhou(周丽琴)1,2,†, Junbao He(何俊宝)1,2,3,†, Chunyao Song(宋春尧)1,2,†, Yu Xu(徐煜)1,2, Yongqing Cai(蔡永青)1,2, Cong Li(李聪)1,2, Qingyan Wang(王庆艳)1,2, Lin Zhao(赵林)1,2,4, Guodong Liu(刘国东)1,2,4, Zuyan Xu(许祖彦)5, Genfu Chen(陈根富)1,2, Hongming Weng(翁红明)1,2, and Xingjiang Zhou(周兴江)1,2,4,6,‡ 1National Laboratory for Superconductivity, Beijing National Laboratory for Condensed Matter Physics, Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China 2University of Chinese Academy of Sciences, Beijing 100049, China 3Henan International Joint Laboratory of MXene Materials Microstructure, College of Physics and Electronic Engineering, Nanyang Normal University, Nanyang 473061, China 4Songshan Lake Materials Laboratory, Dongguan 523808, China 5Technical Institute of Physics and Chemistry, Chinese Academy of Sciences, Beijing 100190, China 6Beijing Academy of Quantum Information Sciences, Beijing 100193, China
New Subgrouping of Multiple Stellar Populations in NGC 2808 Based on Low-resolution Spectroscopy

We performed low-resolution spectroscopy for the red giant branch stars in an intriguing globular cluster (GC) NGC 2808, which hosts subpopulations with extreme helium and light-element abundances. In order to trace N, C, and Ca abundance differences among subpopulations, we measured CN, CH, and Ca ii H&K spectral indices, respectively. We identified four subpopulations (G1, G2, G3, and G4) from CN and CH strength, with CN-weak/CH-strong G1, CN-intermediate/CH-strong G2, CN-strong/CH-intermediate G3, and CN-strong/CH-weak G4. Compared to [Na/O] from high-resolution spectroscopy, we show that CN index can more clearly separate G1 and G2. Since CN traces N abundance in a GC, it implies that G1 and G2 would show a larger difference in [N/Fe] compared to [Na/Fe], as predicted by chemical evolution models. Later generation stars G3 and G4, however, are better separated with high-resolution spectroscopy. We also found that G4 shows a stronger Ca ii H&K line strength compared to that of G1, but we suspect this to be a result of unusually strong He enhancement and/or Mg depletion in G4 of this GC. This work illustrates that combining low- and high-resolution spectroscopic studies can improve the separation of subpopulations in GCs.
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Mental health conditions remain underdiagnosed even in countries with common access to advanced medical care. The ability to accurately and efficiently predict mood from easily collectible data has several important implications for the early detection, intervention, and treatment of mental health disorders. One promising data source to help monitor human behavior is daily smartphone usage. However, care must be taken to summarize behaviors without identifying the user through personal (e.g., personally identifiable information) or protected (e.g., race, gender) attributes. In this paper, we study behavioral markers of daily mood using a recent dataset of mobile behaviors from adolescent populations at high risk of suicidal behaviors. Using computational models, we find that language and multimodal representations of mobile typed text (spanning typed characters, words, keystroke timings, and app usage) are predictive of daily mood. However, we find that models trained to predict mood often also capture private user identities in their intermediate representations. To tackle this problem, we evaluate approaches that obfuscate user identity while remaining predictive. By combining multimodal representations with privacy-preserving learning, we are able to push forward the performance-privacy frontier.
TwiBot-20: A Comprehensive Twitter Bot Detection Benchmark

Twitter has become a vital social media platform while an ample amount of malicious Twitter bots exist and induce undesirable social effects. Successful Twitter bot detection proposals are generally supervised, which rely heavily on large-scale datasets. However, existing benchmarks generally suffer from low levels of user diversity, limited user information and data scarcity. Therefore, these datasets are not sufficient to train and stably benchmark bot detection measures. To alleviate these problems, we present TwiBot-20, a massive Twitter bot detection benchmark, which contains 229,573 users, 33,488,192 tweets, 8,723,736 user property items and 455,958 follow relationships. TwiBot-20 covers diversified bots and genuine users to better represent the real-world Twittersphere. TwiBot-20 also includes three modals of user information to support both binary classification of single users and community-aware approaches. To the best of our knowledge, TwiBot-20 is the largest Twitter bot detection benchmark to date. We reproduce competitive bot detection methods and conduct a thorough evaluation on TwiBot-20 and two other public datasets. Experiment results demonstrate that existing bot detection measures fail to match their previously claimed performance on TwiBot-20, which suggests that Twitter bot detection remains a challenging task and requires further research efforts.
A Simple and Strong Baseline: Progressively Region-based Scene Text Removal Networks

Existing scene text removal methods mainly train an elaborate network with paired images to realize the function of text localization and background reconstruction simultaneously, but there exists two problems: 1) lacking the exhaustive erasure of text region and 2) causing the excessive erasure to text-free areas. To handle these issues, this paper provides a novel ProgrEssively Region-based scene Text eraser (PERT), which introduces region-based modiﬁcation strategy to progressively erase the pixels in only text region. Firstly, PERT decomposes the STR task to several erasing stages. As each stage aims to take a further step toward the text-removed image rather than directly regress to the ﬁnal result, the decomposed operation reduces the learning difﬁ-culty in each stage, and an exhaustive erasure result can be obtained by iterating over lightweight erasing blocks with shared parameters. Then, PERT introduces a region-based modiﬁcation strategy to ensure the integrity of text-free areas by decoupling text localization from erasure process to guide the removal. Beneﬁting from the simplicity architecture, PERT is a simple and strong baseline, and is easy to be followed and developed. Extensive experiments demonstrate that PERT obtains the state-of-the-art results on both synthetic and real-world datasets. Code is available at https://github.com/wangyuxin87/PERT .
Coarse injectivity, hierarchical hyperbolicity and semihyperbolicity

We relate three classes of nonpositively curved metric spaces: hierarchically hyperbolic spaces, coarsely injective spaces, and strongly shortcut spaces. We show that every hierarchically hyperbolic space admits a new metric that is coarsely injective. The new metric is quasi-isometric to the original metric and is preserved under automorphisms of the hierarchically hyperbolic space. We show that every coarsely injective metric space of uniformly bounded geometry is strongly shortcut. Consequently, hierarchically hyperbolic groups -- including mapping class groups of surfaces -- are coarsely injective and coarsely injective groups are strongly shortcut. Using these results, we deduce several important properties of hierarchically hyperbolic groups, including that they are semihyperbolic, have solvable conjugacy problem, have finitely many conjugacy classes of finite subgroups, and that their finitely generated abelian subgroups are undistorted. Along the way we show that hierarchically quasiconvex subgroups of hierarchically hyperbolic groups have bounded packing.
Molecular Examination of Differentially Expressed Genes in the Brains of Experimental Autoimmune Encephalomyelitis Mice Post Herceptin Treatment

Objective Herceptin (trastuzumab) is an approved drug for treating HER2+ breast cancer patients, but its use for other diseases is not established. We sought to investigate the effects of Herceptin on ameliorating experimental autoimmune encephalomyelitis (EAE) and to examine its effects on the expression of various genes. Methods We used in-silico analysis of publicly available data, qRT-PCR, and immunohistochemistry (IHC) to determine the expression of HER2+ cells in the brains of EAE mice. IHC was also utilized to determine the anti-inflammatory effects of Herceptin. The ability of Herceptin to alleviate the EAE clinical score was measured in these mice. Bioinformatics analysis of publicly available data and qRT-PCR were performed to investigate the differentially expressed genes that were either up-regulated or down-regulated during the high clinical score (HCS) of the disease. Results We observed that HER2/Erbb2, the receptor for Herceptin is upregulated in the brains of EAE mice when the brains were examined at the HCS stage. Further, we demonstrated that Herceptin ameliorates the EAE disease, increasing re-myelination, reducing brain inflammation, CD3+ T cell accumulation, and HER2+ cells in the brains of these mice. Molecular analysis demonstrated the expression of different genes that were either up-regulated or down-regulated during the HCS of the disease. Our combined bioinformatics and qRT-PCR analyses show increased mRNA expression of Atp6v0d2, C3, C3ar1, Ccl3, Ccl6, Cd74, Clec7a, Cybb, H2-Aa, Hspb1, Lilr4b, Lilrb4a, Mpeg1, Ms4a4a, Ms4a6c, Saa3, Serpina3n and Timp1, at HCS. Except for the mRNA levels of Cd74 and Clec7a which were increased at HCS when Herceptin was used in both prophylactic and therapeutic regimens, the levels of other described mRNAs were reduced. Conclusion These novel findings show that Herceptin ameliorates the clinical score in EAE mice and are the first to investigate in detail the differential gene expression post-treatment with the drug.
Does the type of cryoprobe affect oncological and functional outcomes in men with clinically localized prostate cancer treated with primary whole gland prostate cryoablation?

Abstract Background: This study aimed to compare the oncological and functional outcomes of primary whole gland cryoablation of the prostate using the variable ice cryoprobe (V-Probe®) and the conventional fixed-size ice probe. Materials and methods: We reviewed the Cryo On-Line Data Registry for men who were treated with primary whole gland prostate cryoablation from 2000 through 2017. A multivariate Cox proportional hazards model was used to compare timing to biochemical recurrence between the V-Probe® and fixed-size ice probe after adjusting for preoperative prostate-specific antigen (PSA), neoadjuvant androgen deprivation therapy, preoperative Gleason score, and preoperative T stage. Results: A total of 1124 men were included. Median age, Gleason score, and pretreatment PSA were 70 years (interquartile range [IQR]: 65–74 years), 7 (IQR: 6–7) and 5.9 ng/mL (IQR: 4.6–8.1 ng/mL), respectively. The median follow-up time was 25.0 months (IQR: 11.2–48.6 months). V-Probes® were used in 269 (23.9%) cases and fixed-size ice probes in 858 (76.1%) cases. After adjusting for clinical T stage, PSA, neoadjuvant androgen deprivation therapy and preoperative Gleason score, on the multivariate Cox regression model, we found that there was no significant difference between the type of probe and timing to biochemical recurrence (p = 0.35). On multivariate logistic regression, using the V-Probe® was associated with a 91% increase in postoperative urinary retention compared to the fixed-size ice probe (p = 0.003). Conclusions: The use of the V-Probe® versus conventional fixed-size ice probe was not associated with a difference in biochemical recurrence in patients undergoing primary cryoablation of the prostate.