Miracle-dz/UniFAR-FTUs huggingface dataset:
MCMAE: Masked Convolution Meets Masked Autoencoders

Vision Transformers (ViT) become widely-adopted architectures for various vision tasks. Masked auto-encoding [2, 1, 28, 55] for feature pretraining and multi-scale hybrid convolution-transformer architectures [12, 21, 49, 34, 57] can further unleash the potentials of ViT, leading to state-of-the-art performances on image classification, detection and semantic segmentation. In this paper, our MCMAE framework demonstrates that multi-scale hybrid convolution-transformer can learn more discriminative representations via the mask auto-encoding scheme. However, directly using the original masking strategy leads to the heavy computational cost and pretraining-finetuning discrepancy. To tackle the issue, we adopt the masked convolution to prevent information leakage in the convolution blocks. A simple block-wise masking strategy is proposed to ensure computational efficiency. We also propose to more directly supervise the multi-scale features of the encoder to boost multi-scale features. MCMAE-Base improves ImageNet-1K finetuning accuracy by 1.4% compared with MAE-Base. On object detection, MCMAE-Base finetuned for only 25 epochs surpasses MAE-Base fined-tuned for 100 epochs by 2.9% AP box and 2.2% AP mask respectively. Code and pretrained models are available at https://github.com/Alpha-VL/ConvMAE .
Geodesic Self-Attention for 3D Point Clouds

Due to the outstanding competence in capturing long-range relationships, self-attention mechanism has achieved remarkable progress in point cloud tasks. Never-theless, point cloud object often has complex non-Euclidean spatial structures, with the behavior changing dynamically and unpredictably. Most current self-attention modules highly rely on the dot product multiplication in Euclidean space, which cannot capture internal non-Euclidean structures of point cloud objects, especially the long-range relationships along the curve of the implicit manifold surface represented by point cloud objects. To address this problem, in this paper, we introduce a novel metric on the Riemannian manifold to capture the long-range geometrical dependencies of point cloud objects to replace traditional self-attention modules, namely, the G eodesic S elf-A ttention (GSA) module. Our approach achieves state-of-the-art performance compared to point cloud Transformers [13, 10, 44, 26] on object classiﬁcation, few-shot classiﬁcation and part segmentation benchmarks.
Global Optimal K-Medoids Clustering of One Million Samples

We study the deterministic global optimization of the K-Medoids clustering problem. This work proposes a branch and bound (BB) scheme, in which a tailored Lagrangian relaxation method proposed in the 1970s is used to provide a lower bound at each BB node. The lower bounding method already guarantees the maximum gap at the root node. A closed-form solution to the lower bound can be derived analytically without explicitly solving any optimization problems, and its computation can be easily parallelized. Moreover, with this lower bounding method, finite convergence to the global optimal solution can be guaranteed by branching only on the regions of medoids. We also present several tailored bound tightening techniques to reduce the search space and computational cost. Extensive computational studies on 28 machine learning datasets demonstrate that our algorithm can provide a provable global optimal solution with an optimality gap of 0.1% within 4 hours on datasets with up to one million samples. Besides, our algorithm can obtain better or equal objective values than the heuristic method. A theoretical proof of global convergence for our algorithm is also presented.
Brain activity of professional investors signals future stock performance

A major aspiration of investors is to better forecast stock performance. Interestingly, emerging ‘neuroforecasting’ research suggests that brain activity associated with anticipatory reward relates to market behavior and population-wide preferences, including stock price dynamics. In this study we extend these findings to professional investors processing comprehensive real-world information on stock investment options while making predictions of long-term stock performance. Using functional magnetic resonance imaging (fMRI), we sampled investors’ neural responses to investment cases and assessed whether these responses relate to future performance on the stock market. We find that our sample of investors could not successfully predict future market performance of the investment cases, confirming that stated preferences do not predict the market. Stock metrics of the investment cases were not predictive of future stock performance either. However, as investors processed case information, nucleus accumbens (NAcc) activity was higher for investment cases that ended up overperforming in the market. These findings remained robust, even when controlling for stock metrics and investors’ predictions made in the scanner. Cross-validated prediction analysis indicated that NAcc activity could significantly predict future stock performance out-of-sample above chance. Our findings resonate with recent neuroforecasting studies and suggest that brain activity of professional investors may help in forecasting future stock performance. Significance Statement The investors’ dream of forecasting the stock market is typically considered to be just that: An unrealistic aspiration. However, we find that forecasting stock performance may in fact not be completely unattainable. Results of our neuroimaging experiment reveal that professional investors fail to accurately predict long-term stock performance. However, while processing complex information pertaining to investment cases, brain activity in a region associated with reward anticipation was increased for stocks that would end up overperforming in the future market. Remarkably, this effect held after controlling for the stock information presented in the investment cases. Our findings add to recent work in ‘neuroforecasting’, demonstrating that market behavior can be forecasted by brain activity of a small sample, here of professional investors.
Diverging maternal and infant cord antibody functions from SARS-CoV-2 infection and vaccination in pregnancy

Immunization in pregnancy is a critical tool that can be leveraged to protect the infant with an immature immune system but how vaccine-induced antibodies transfer to the placenta and protect the maternal-fetal dyad remains unclear. Here, we compare matched maternal-infant cord blood from individuals who in pregnancy received mRNA COVID-19 vaccine, were infected by SARS-CoV-2, or had the combination of these two immune exposures. We find that some but not all antibody neutralizing activities and Fc effector functions are enriched with vaccination compared to infection. Preferential transport to the fetus of Fc functions and not neutralization is observed. Immunization compared to infection enriches IgG1-mediated antibody functions with changes in antibody post-translational sialylation and fucosylation that impact fetal more than maternal antibody functional potency. Thus, vaccine enhanced antibody functional magnitude, potency and breadth in the fetus are driven more by antibody glycosylation and Fc effector functions compared to maternal responses, highlighting prenatal opportunities to safeguard newborns as SARS-CoV-2 becomes endemic. One Sentence Summary SARS-CoV-2 vaccination in pregnancy induces diverging maternal and infant cord antibody functions
β-ionone regulates Arabidopsis thaliana transcriptome and increases its resistance against Botrytis cinerea

Carotenoids are isoprenoid pigments vital for photosynthesis. Moreover, they are the precursor of apocarotenoids that include the phytohormones abscisic acid (ABA) and strigolactones (SLs), and retrograde signaling molecules and growth regulators, such as β-cyclocitral and zaxinone. The apocarotenoid β-ionone (β-I) was previously reported to exert antimicrobial effects. Here, we showed that the application of this scent to Arabidopsis plants at micromolar concentrations caused a global reprogramming of gene expression, affecting thousands of transcripts involved in stress tolerance, growth, hormone metabolism, pathogen defense and photosynthesis. These changes, along with modulating the levels of the phytohormones ABA, jasmonic acid and salicylic acid, led to enhanced Arabidopsis resistance to Botrytis cinerea (B.c.), one of the most aggressive and widespread pathogenic fungi affecting numerous plant hosts and causing severe losses of postharvest fruits. Pre-treatment of tobacco and tomato plants with β-I followed by inoculation with B.c. confirms the conserved effect of β-I and induced immune responses in leaves and fruits. Moreover, there was reduced susceptibility to B.c. in LYCOPENE β-CYCLASE- expressing tomato fruits possessing elevated levels of the endogenous β-I, indicating beneficial biological activities of this compound in planta. Our work unraveled β-I as a further carotenoid-derived regulatory metabolite and opens up new possibilities to control B.c. infection by establishing this natural volatile as an environmentally friendly bio-fungicide.
Post-hoc estimators for learning to defer to an expert

Many practical settings allow a classiﬁer to defer predictions to one or more costly experts . For example, the learning to defer paradigm allows a classiﬁer to defer to a human expert, at some monetary cost. Similarly, the adaptive inference paradigm allows a base model to defer to one or more large models, at some computational cost. The goal in these settings is to learn classiﬁcation and deferral mechanisms to optimise a suitable accuracy-cost tradeo � . To achieve this, a central issue studied in prior work is the design of a coherent loss function for both mechanisms. In this work, we demonstrate that existing losses can underﬁt the training set when there is a non-trivial deferral cost, owing to an implicit application of a high level of label smoothing. To resolve this, we propose two post-hoc estimators that ﬁt a deferral function on top of a base model, either by threshold correction, or by learning when the base model’s error rate exceeds the cost of deferring to the expert. Both approaches are equipped with theoretical guarantees, and empirically yield e � ective accuracy-cost tradeo � s on learning to defer and adaptive inference benchmarks.
Multi temporal multispectral UAV remote sensing allows for yield assessment across European wheat varieties already in tillering stage

High throughput field phenotyping techniques employing multispectral cameras allow to extract a variety of variables and features to predict yield and yield related traits, but little is known about which types of multispectral features may allow to forecast yield potential in the early growth phase. In this study, we hypothesized that the best features for predicting yield in an early stage might be different from the best predictors for the late growth stages. Based on a variety testing trial of 19 European wheat varieties in 2021, multispectral images were taken on 19 dates ranging from tillering to harvest by an unmanned aerial vehicle measuring reflectance in five bands, including visible bands, Red-edge and the near-infrared (NIR). Orthomosaic images were created, and then the single band reflectances, vegetation indices (VI) and texture features (TF) based on a gray level correlation matrix (GLCM) were extracted. We evaluated the performance of these three types of features for yield prediction and classification at different growth stages by, i) using features on each of the measurement dates, ii) smoothing features across the 19 dates, and iii) combining features across the directly adjacent dates, in combination with the random forest models. Our results showed that, for most features, measurements at the flowering stage showed the best performance and the Red reflectance was able to predict yield with a RMSE of 47.4 g m-2 (R2 = 0.63), the best VI was NDRE predicting yield with a RMSE of 47.9 g m-2 (R2 = 0.63), the best TF was contrast predicting yield with a RMSE of 57.2 g m-2 (R2 = 0.46) at the booting stage. Combining dates improved yield prediction in all dates and made the prediction errors more stable across dates. Rather than the Red-edge band, visible bands especially the Red band enabled to distinguish between the high- and low-yielding varieties already in the tillering stage, with a total accuracy of 76.7%. The study confirms our hypothesis and further implies that, in the early stages, the visible bands may be more effective than Red-edge bands in assessing the yield potential in a range of testing varieties.
SAP: Synteny-aware gene function prediction for bacteria using protein embeddings

Motivation Today, we know the function of only a small fraction of the protein sequences predicted from genomic data. This problem is even more salient for bacteria, which represent some of the most phylogenetically and metabolically diverse taxa on Earth. This low rate of bacterial gene annotation is compounded by the fact that most function prediction algorithms have focused on eukaryotes, and conventional annotation approaches rely on the presence of similar sequences in existing databases. However, often there are no such sequences for novel bacterial proteins. Thus, we need improved gene function prediction methods tailored for prokaryotes. Recently, transformer-based language models - adopted from the natural language processing field - have been used to obtain new representations of proteins, to replace amino acid sequences. These representations, referred to as protein embeddings, have shown promise for improving annotation of eukaryotes, but there have been only limited applications on bacterial genomes. Results To predict gene functions in bacteria, we developed SAP, a novel synteny-aware gene function prediction tool based on protein embeddings from state-of-the-art protein language models. SAP also leverages the unique operon structure of bacteria through conserved synteny. SAP outperformed both conventional sequence-based annotation methods and state-of-the-art methods on multiple bacterial species, including for distant homolog detection, where the sequence similarity to the proteins in the training set was as low as 40%. Using SAP to identify gene functions across diverse enterococci, of which some species are major clinical threats, we identified 11 previously unrecognized putative novel toxins, with potential significance to human and animal health. Availability https://github.com/AbeelLab/sap Contact t.abeel@tudelft.nl Supplementary information Supplementary data are available at Bioinformatics online.
Characterizing Spatiotemporal Population Receptive Fields in Human Visual Cortex with fMRI

The use of fMRI and computational modeling has advanced understanding of spatial characteristics of population receptive fields (pRFs) in human visual cortex. However, we know relatively little about the spatiotemporal characteristics of pRFs because neurons' temporal properties are one to two orders of magnitude faster than fMRI BOLD responses. Here, we developed an image-computable framework to estimate spatiotemporal pRFs from fMRI data. First, we developed a simulation software that predicts fMRI responses to a time-varying visual input given a spatiotemporal pRF model and solves the model parameters. The simulator revealed that ground-truth spatiotemporal parameters can be accurately recovered at the millisecond resolution from synthesized fMRI responses. Then, using fMRI and a novel stimulus paradigm, we mapped spatiotemporal pRFs in individual voxels across human visual cortex in 10 participants (both females and males). We find that a compressive spatiotemporal (CST) pRF model better explains fMRI responses than a conventional spatial pRF model across visual areas spanning the dorsal, lateral, and ventral streams. Further, we find three organizational principles of spatiotemporal pRFs: (1) from early to later areas within a visual stream, spatial and temporal windows of pRFs progressively increase in size and show greater compressive nonlinearities, (2) later visual areas show diverging spatial and temporal windows across streams, and (3) within early visual areas (V1–V3), both spatial and temporal windows systematically increase with eccentricity. Together, this computational framework and empirical results open exciting new possibilities for modeling and measuring fine-grained spatiotemporal dynamics of neural responses using fMRI.
Randomized Channel Shuffling: Minimal-Overhead Backdoor Attack Detection without Clean Datasets

Deep neural networks (DNNs) typically require massive data to train on, which is a hurdle for numerous practical domains. Facing the data shortfall, one viable option is to acquire domain-specific training data from external uncensored sources, such as open webs or third-party data collectors. However, the quality of such acquired data is often not rigorously scrutinized, and one cannot easily rule out the risk of “poisoned” examples being included in such unreliable datasets, resulting in unreliable trained models which pose potential risks to many high-stake applications. While existing options usually suffer from high computational costs or assumptions on clean data access, this paper attempts to detect backdoors for potential victim models with minimal prior knowledge. In particular, provided with a trained model, users are assumed to (1) have no prior knowledge of whether it is already poisoned, or what the target class/percentage of samples is poisoned, and (2) have no access to a clean sample set from the same training distribution, nor any trusted model trained on such clean data. To tackle this challenging scenario, we first observe the contrasting channel-level statistics between the backdoor trigger and clean image features, and consequently, how they can be differentiated by progressive channel shuffling. We then propose the randomized channel shuffling method for backdoor-targeted class detection, which requires only a few feed-forward passes. It thus incurs minimal overheads and demands no clean sample nor prior knowledge. We further explore a “full” clean data-free setting, where neither the target class detection nor the trigger recovery can access the clean data. Extensive experiments are conducted with three datasets (CIFAR-10, GTSRB, Tiny ImageNet), three architectures (AlexNet, ResNet-20, SENet-18), and three attacks (BadNets[1], clean label attack [2], and WaNet
Engineered cytokine/antibody fusion proteins improve delivery of IL-2 to pro-inflammatory cells and promote antitumor activity

Progress in cytokine engineering is driving therapeutic translation by overcoming the inherent limitations of these proteins as drugs. The interleukin-2 (IL-2) cytokine harbors great promise as an immune stimulant for cancer treatment. However, the cytokine’s concurrent activation of both pro-inflammatory immune effector cells and anti-inflammatory regulatory T cells, its toxicity at high doses, and its short serum half-life have limited clinical application. One promising approach to improve the selectivity, safety, and longevity of IL-2 is complexation with anti-IL-2 antibodies that bias the cytokine towards the activation of immune effector cells (i.e., effector T cells and natural killer cells). Although this strategy shows therapeutic potential in preclinical cancer models, clinical translation of a cytokine/antibody complex is complicated by challenges in formulating a multi-protein drug and concerns about complex stability. Here, we introduce a versatile approach to designing intramolecularly assembled single-agent fusion proteins (immunocytokines, ICs) comprising IL-2 and a biasing anti-IL-2 antibody that directs the cytokine’s activities towards immune effector cells. We establish the optimal IC construction and further engineer the cytokine/antibody affinity to improve immune biasing function. We demonstrate that our IC preferentially activates and expands immune effector cells, leading to superior antitumor activity compared to natural IL-2 without inducing toxicities associated with IL-2 administration. Collectively, this work presents a roadmap for the design and translation of immunomodulatory cytokine/antibody fusion proteins. One Sentence Summary We developed an IL-2/antibody fusion protein that expands immune effector cells and shows superior tumor suppression and toxicity profile versus IL-2.
ULNeF: Untangled Layered Neural Fields for Mix-and-Match Virtual Try-On

Recent advances in neural models have shown great results for virtual try-on (VTO) problems, where a 3D representation of a garment is deformed to fit a target body shape. However, current solutions are limited to a single garment layer, and cannot address the combinatorial complexity of mixing different garments. Motivated by this limitation, we investigate the use of neural fields for mix-and-match VTO, and identify and solve a fundamental challenge that existing neural-field methods cannot address: the interaction between layered neural fields. To this end, we propose a neural model that untangles layered neural fields to represent collision-free garment surfaces. The key ingredient is a neural untangling projection operator that works directly on the layered neural fields, not on explicit surface representations. Algorithms to resolve object-object interaction are inherently limited by the use of explicit geometric representations, and we show how methods that work directly on neural implicit representations could bring a change of paradigm and open the door to radically different approaches.
Learning Generalizable Part-based Feature Representation for 3D Point Clouds

Deep networks on 3D point clouds have achieved remarkable success in 3D classification, while they are vulnerable to geometry variations caused by inconsistent data acquisition procedures. This results in a challenging 3D domain generalization (3DDG) problem, that is to generalize a model trained on source domain to an unseen target domain. Based on the observation that local geometric structures are more generalizable than the whole shape, we propose to reduce the geometry shift by a generalizable part-based feature representation and design a novel part-based domain generalization network (PDG) for 3D point cloud classification. Specifically, we build a part-template feature space shared by source and target domains. Shapes from distinct domains are first organized to part-level features and then represented by part-template features. The transformed part-level features, dubbed aligned part-based representations, are then aggregated by a part-based feature aggregation module. To improve the robustness of the part-based representations, we further propose a contrastive learning framework upon part-based shape representation. Experiments and ablation studies on 3DDA and 3DDG benchmarks justify the efficacy of the proposed approach for domain generalization, compared with the previous state-of-the-art methods. Our code will be available on http://github.com/weixmath/PDG .