pwc-archive/papers-with-abstracts huggingface dataset:
Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution
Authors: Yizi Zhang, Yanchen Wang, Donato Jimenez-Beneto, Zixuan Wang, Mehdi Azabou, Blake Richards, Olivier Winter, International Brain Laboratory, Eva Dyer, Liam Paninski, Cole Hurwitz
Date: 2024-07-19 00:00:00
Tasks: Activity Prediction, Multi-Task Learning, Prediction

Neuroscience research has made immense progress over the last decade, but our understanding of the brain remains fragmented and piecemeal: the dream of probing an arbitrary brain region and automatically reading out the information encoded in its neural activity remains out of reach. In this work, we build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. We introduce a novel self-supervised modeling approach for population activity in which the model alternates between masking out and reconstructing neural activity across different time steps, neurons, and brain regions. To evaluate our approach, we design unsupervised and supervised prediction tasks using the International Brain Laboratory repeated site dataset, which is comprised of Neuropixels recordings targeting the same brain locations across 48 animals and experimental sessions. The prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding. We demonstrate that our multi-task-masking (MtM) approach significantly improves the performance of current state-of-the-art population models and enables multi-task learning. We also show that by training on multiple animals, we can improve the generalization ability of the model to unseen animals, paving the way for a foundation model of the brain at single-cell, single-spike resolution.
LORTSAR: Low-Rank Transformer for Skeleton-based Action Recognition
Authors: Soroush Oraki, Harry Zhuang, Jie Liang
Date: 2024-07-19 00:00:00
Tasks: Action Recognition, Computational Efficiency, Model Compression, Skeleton Based Action Recognition, Temporal Action Localization

The complexity of state-of-the-art Transformer-based models for skeleton-based action recognition poses significant challenges in terms of computational efficiency and resource utilization. In this paper, we explore the application of Singular Value Decomposition (SVD) to effectively reduce the model sizes of these pre-trained models, aiming to minimize their resource consumption while preserving accuracy. Our method, LORTSAR (LOw-Rank Transformer for Skeleton-based Action Recognition), also includes a fine-tuning step to compensate for any potential accuracy degradation caused by model compression, and is applied to two leading Transformer-based models, "Hyperformer" and "STEP-CATFormer". Experimental results on the "NTU RGB+D" and "NTU RGB+D 120" datasets show that our method can reduce the number of model parameters substantially with negligible degradation or even performance increase in recognition accuracy. This confirms that SVD combined with post-compression fine-tuning can boost model efficiency, paving the way for more sustainable, lightweight, and high-performance technologies in human action recognition.
The Collection of a Human Robot Collaboration Dataset for Cooperative Assembly in Glovebox Environments
Authors: Shivansh Sharma, Mathew Huang, Sanat Nair, Alan Wen, Christina Petlowany, Juston Moore, Selma Wanna, Mitch Pryor
Date: 2024-07-19 00:00:00
Tasks: Hand Segmentation, Segmentation

Industry 4.0 introduced AI as a transformative solution for modernizing manufacturing processes. Its successor, Industry 5.0, envisions humans as collaborators and experts guiding these AI-driven manufacturing solutions. Developing these techniques necessitates algorithms capable of safe, real-time identification of human positions in a scene, particularly their hands, during collaborative assembly. Although substantial efforts have curated datasets for hand segmentation, most focus on residential or commercial domains. Existing datasets targeting industrial settings predominantly rely on synthetic data, which we demonstrate does not effectively transfer to real-world operations. Moreover, these datasets lack uncertainty estimations critical for safe collaboration. Addressing these gaps, we present HAGS: Hand and Glove Segmentation Dataset. This dataset provides challenging examples to build applications toward hand and glove segmentation in industrial human-robot collaboration scenarios as well as assess out-of-distribution images, constructed via green screen augmentations, to determine ML-classifier robustness. We study state-of-the-art, real-time segmentation models to evaluate existing methods. Our dataset and baselines are publicly available.
Advancing Melanoma Diagnosis with Self-Supervised Neural Networks: Evaluating the Effectiveness of Different Techniques
Authors: Srivishnu Vusirikala, Suraj Rajendran
Date: 2024-07-19 00:00:00
Tasks: Contrastive Learning, Melanoma Diagnosis

We investigate the potential of self-supervision in improving the accuracy of deep learning models trained to classify melanoma patches. Various self-supervision techniques such as rotation prediction, missing patch prediction, and corruption removal were implemented and assessed for their impact on the convolutional neural network's performance. Preliminary results suggest a positive influence of self-supervision methods on the model's accuracy. The study notably demonstrates the efficacy of the corruption removal method in enhancing model performance. Despite observable improvements, we conclude that the self-supervised models have considerable potential for further enhancement, achievable through training over more epochs or expanding the dataset. We suggest exploring other self-supervision methods like Bootstrap Your Own Latent (BYOL) and contrastive learning in future research, emphasizing the cost-benefit trade-off due to their resource-intensive nature. The findings underline the promise of self-supervision in augmenting melanoma detection capabilities of deep learning models.
The Research of Group Re-identification from Multiple Cameras
Authors: Hao Xiao
Date: 2024-07-19 00:00:00
Tasks: Graph Matching, Pedestrian Detection

Object re-identification is of increasing importance in visual surveillance. Most existing works focus on re-identify individual from multiple cameras while the application of group re-identification (Re-ID) is rarely discussed. We redefine Group Re-identification as a process which includes pedestrian detection, feature extraction, graph model construction, and graph matching. Group re-identification is very challenging since it is not only interfered by view-point and human pose variations in the traditional re-identification tasks, but also suffered from the challenges in group layout change and group member variation. To address the above challenges, this paper introduces a novel approach which leverages the multi-granularity information inside groups to facilitate group re-identification. We first introduce a multi-granularity Re-ID process, which derives features for multi-granularity objects (people/people-subgroups) in a group and iteratively evaluates their importances during group Re-ID, so as to handle group-wise misalignments due to viewpoint change and group dynamics. We further introduce a multi-order matching scheme. It adaptively selects representative people/people-subgroups in each group and integrates the multi-granularity information from these people/people-subgroups to obtain group-wise matching, hence achieving a more reliable matching score between groups. Experimental results on various datasets demonstrate the effectiveness of our approach.
A Comparative Study of Transfer Learning for Emotion Recognition using CNN and Modified VGG16 Models
Authors: Samay Nathani
Date: 2024-07-19 00:00:00
Tasks: Diversity, Emotion Recognition, Transfer Learning

Emotion recognition is a critical aspect of human interaction. This topic garnered significant attention in the field of artificial intelligence. In this study, we investigate the performance of convolutional neural network (CNN) and Modified VGG16 models for emotion recognition tasks across two datasets: FER2013 and AffectNet. Our aim is to measure the effectiveness of these models in identifying emotions and their ability to generalize to different and broader datasets. Our findings reveal that both models achieve reasonable performance on the FER2013 dataset, with the Modified VGG16 model demonstrating slightly increased accuracy. When evaluated on the Affect-Net dataset, performance declines for both models, with the Modified VGG16 model continuing to outperform the CNN. Our study emphasizes the importance of dataset diversity in emotion recognition and discusses open problems and future research directions, including the exploration of multi-modal approaches and the development of more comprehensive datasets.
Are handcrafted filters helpful for attributing AI-generated images?
Authors: Jialiang Li, Haoyue Wang, Sheng Li, Zhenxing Qian, Xinpeng Zhang, Athanasios V. Vasilakos
Date: 2024-07-19 00:00:00
Tasks: Attribute, Image Attribution, Image Generation

Recently, a vast number of image generation models have been proposed, which raises concerns regarding the misuse of these artificial intelligence (AI) techniques for generating fake images. To attribute the AI-generated images, existing schemes usually design and train deep neural networks (DNNs) to learn the model fingerprints, which usually requires a large amount of data for effective learning. In this paper, we aim to answer the following two questions for AI-generated image attribution, 1) is it possible to design useful handcrafted filters to facilitate the fingerprint learning? and 2) how we could reduce the amount of training data after we incorporate the handcrafted filters? We first propose a set of Multi-Directional High-Pass Filters (MHFs) which are capable to extract the subtle fingerprints from various directions. Then, we propose a Directional Enhanced Feature Learning network (DEFL) to take both the MHFs and randomly-initialized filters into consideration. The output of the DEFL is fused with the semantic features to produce a compact fingerprint. To make the compact fingerprint discriminative among different models, we propose a Dual-Margin Contrastive (DMC) loss to tune our DEFL. Finally, we propose a reference based fingerprint classification scheme for image attribution. Experimental results demonstrate that it is indeed helpful to use our MHFs for attributing the AI-generated images. The performance of our proposed method is significantly better than the state-of-the-art for both the closed-set and open-set image attribution, where only a small amount of images are required for training.
Quantifying vitamin D intake among Aboriginal and Torres Strait Islander peoples in Australia
Authors: Belinda Neo, Dale Tilbrook, Noel Nannup, Alison Daly, Eleanor Dunlop, John Jacky, Carol Michie, Cindy Prior, Brad Farrant, Carrington C. J. Shepherd, Lucinda J. Black
Date: 2024-07-17 00:00:00
Tasks: Nutrition

Background/Objective: Vitamin D deficiency (serum 25-hydroxyvitamin D [25(OH)D] concentration <50 nmol/L) is prevalent among Aboriginal and Torres Strait Islander peoples in Australia. Alternative to sun exposure (the primary source of vitamin D), vitamin D can also be obtained from food (e.g., fish, eggs, and meat) and supplements. However, vitamin D intake among Aboriginal and Torres Strait Islander peoples is currently unknown. We aimed to provide the first quantification of vitamin D intake using nationally representative data from Aboriginal and Torres Strait Islander peoples. Methods: We used food consumption data collected in the 2012-2013 National Aboriginal and Torres Strait Islander Nutrition and Physical Activity Survey (n = 4,109) and vitamin D food composition data to quantify mean absolute vitamin D intake by sex, age group, and remoteness of location. Differences in mean vitamin D intake between sexes and between remoteness of location were assessed using the 95% confidence interval (95% CI). Results: The mean (standard deviation (SD)) vitamin D intake among Aboriginal and Torres Strait Islander peoples was 2.9 (3.0) {\mu}g/day. Males had a statistically significantly higher mean (SD) [95% CI] vitamin D intake (3.2 (3.1) [3.0-3.4] {\mu}g/day) than females (2.6 (2.7) [2.4-2.7] {\mu}g/day). There were no statistically significant differences between mean (SD) [95% CI] vitamin D intake in non-remote (2.9 (2.2) [2.7-3.1] {\mu}g/day) and remote areas (2.8 (4.8) [2.6-3.0] {\mu}g/day). Conclusions: Vitamin D intake among Aboriginal and Torres Strait Islander peoples is low. Food-based public health strategies could be developed to promote higher vitamin D intake among this population.
SoftCVI: Contrastive variational inference with self-generated soft labels
Authors: Daniel Ward, Mark Beaumont, Matteo Fasiolo
Date: 2024-07-22 00:00:00
Tasks: Bayesian Inference, Variational Inference

Estimating a distribution given access to its unnormalized density is pivotal in Bayesian inference, where the posterior is generally known only up to an unknown normalizing constant. Variational inference and Markov chain Monte Carlo methods are the predominant tools for this task; however, both are often challenging to apply reliably, particularly when the posterior has complex geometry. Here, we introduce Soft Contrastive Variational Inference (SoftCVI), which allows a family of variational objectives to be derived through a contrastive estimation framework. The approach parameterizes a classifier in terms of a variational distribution, reframing the inference task as a contrastive estimation problem aiming to identify a single true posterior sample among a set of samples. Despite this framing, we do not require positive or negative samples, but rather learn by sampling the variational distribution and computing ground truth soft classification labels from the unnormalized posterior itself. The objectives have zero variance gradient when the variational approximation is exact, without the need for specialized gradient estimators. We empirically investigate the performance on a variety of Bayesian inference tasks, using both simple (e.g. normal) and expressive (normalizing flow) variational distributions. We find that SoftCVI can be used to form objectives which are stable to train and mass-covering, frequently outperforming inference with other variational approaches.
Dual Test-time Training for Out-of-distribution Recommender System
Authors: Xihong Yang, Yiqi Wang, Jin Chen, Wenqi Fan, Xiangyu Zhao, En Zhu, Xinwang Liu, Defu Lian
Date: 2024-07-22 00:00:00
Tasks: Recommendation Systems

Deep learning has been widely applied in recommender systems, which has achieved revolutionary progress recently. However, most existing learning-based methods assume that the user and item distributions remain unchanged between the training phase and the test phase. However, the distribution of user and item features can naturally shift in real-world scenarios, potentially resulting in a substantial decrease in recommendation performance. This phenomenon can be formulated as an Out-Of-Distribution (OOD) recommendation problem. To address this challenge, we propose a novel Dual Test-Time-Training framework for OOD Recommendation, termed DT3OR. In DT3OR, we incorporate a model adaptation mechanism during the test-time phase to carefully update the recommendation model, allowing the model to specially adapt to the shifting user and item features. To be specific, we propose a self-distillation task and a contrastive task to assist the model learning both the user's invariant interest preferences and the variant user/item characteristics during the test-time phase, thus facilitating a smooth adaptation to the shifting features. Furthermore, we provide theoretical analysis to support the rationale behind our dual test-time training framework. To the best of our knowledge, this paper is the first work to address OOD recommendation via a test-time-training strategy. We conduct experiments on three datasets with various backbones. Comprehensive experimental results have demonstrated the effectiveness of DT3OR compared to other state-of-the-art baselines.
Distance-based mutual congestion feature selection with genetic algorithm for high-dimensional medical datasets
Authors: Hossein Nematzadeh, Joseph Mani, Zahra Nematzadeh, Ebrahim Akbari, Radziah Mohamad
Date: 2024-07-22 00:00:00
Tasks: Binary Classification, feature selection

Feature selection poses a challenge in small-sample high-dimensional datasets, where the number of features exceeds the number of observations, as seen in microarray, gene expression, and medical datasets. There isn't a universally optimal feature selection method applicable to any data distribution, and as a result, the literature consistently endeavors to address this issue. One recent approach in feature selection is termed frequency-based feature selection. However, existing methods in this domain tend to overlook feature values, focusing solely on the distribution in the response variable. In response, this paper introduces the Distance-based Mutual Congestion (DMC) as a filter method that considers both the feature values and the distribution of observations in the response variable. DMC sorts the features of datasets, and the top 5% are retained and clustered by KMeans to mitigate multicollinearity. This is achieved by randomly selecting one feature from each cluster. The selected features form the feature space, and the search space for the Genetic Algorithm with Adaptive Rates (GAwAR) will be approximated using this feature space. GAwAR approximates the combination of the top 10 features that maximizes prediction accuracy within a wrapper scheme. To prevent premature convergence, GAwAR adaptively updates the crossover and mutation rates. The hybrid DMC-GAwAR is applicable to binary classification datasets, and experimental results demonstrate its superiority over some recent works. The implementation and corresponding data are available at https://github.com/hnematzadeh/DMC-GAwAR
Inverted Activations: Reducing Memory Footprint in Neural Network Training
Authors: Georgii Novikov, Ivan Oseledets
Date: 2024-07-22 00:00:00

The scaling of neural networks with increasing data and model sizes necessitates the development of more efficient deep learning algorithms. A significant challenge in neural network training is the memory footprint associated with activation tensors, particularly in pointwise nonlinearity layers that traditionally save the entire input tensor for the backward pass, leading to substantial memory consumption. In this paper, we propose a modification to the handling of activation tensors in pointwise nonlinearity layers. Our method involves saving the output tensor instead of the input tensor during the forward pass. Since the subsequent layer typically also saves its input tensor, this approach reduces the total memory required by storing only one tensor between layers instead of two. This optimization is especially beneficial for transformer-based architectures like GPT, BERT, Mistral, and Llama. To enable this approach, we utilize the inverse function of the nonlinearity during the backward pass. As the inverse cannot be computed analytically for most nonlinearities, we construct accurate approximations using simpler functions. Experimental results demonstrate that our method significantly reduces memory usage without affecting training accuracy or computational performance. Our implementation is provided as a drop-in replacement for standard nonlinearity layers in the PyTorch framework, facilitating easy adoption without requiring architectural modifications.
Multiple Importance Sampling for Stochastic Gradient Estimation
Authors: Corentin Salaün, Xingchang Huang, Iliyan Georgiev, Niloy J. Mitra, Gurprit Singh
Date: 2024-07-22 00:00:00

We introduce a theoretical and practical framework for efficient importance sampling of mini-batch samples for gradient estimation from single and multiple probability distributions. To handle noisy gradients, our framework dynamically evolves the importance distribution during training by utilizing a self-adaptive metric. Our framework combines multiple, diverse sampling distributions, each tailored to specific parameter gradients. This approach facilitates the importance sampling of vector-valued gradient estimation. Rather than naively combining multiple distributions, our framework involves optimally weighting data contribution across multiple distributions. This adapted combination of multiple importance yields superior gradient estimates, leading to faster training convergence. We demonstrate the effectiveness of our approach through empirical evaluations across a range of optimization tasks like classification and regression on both image and point cloud datasets.
Conformity to continuous and discrete ordinal traits
Authors: Elisa Heinrich Mora, Kaleda K. Denton, Michael E. Palmer, Marcus W. Feldman
Date: 2024-07-18 00:00:00

Models of conformity and anti-conformity have typically focused on cultural traits with nominal (unordered) variants, such as baby names, strategies (cooperate/defect), or the presence/absence of an innovation. There have been fewer studies of conformity to "ordinal" cultural traits with ordered variants, such as level of cooperation (low to high) or fraction of time spent on a task (0 to 1). In these latter studies, conformity is conceptualized as a preference for the mean trait value in a population even if no members of the population have variants near this mean; e.g., 50% of the population has variant 0 and 50% has variant 1, producing a mean of 0.5. Here, we introduce models of conformity to ordinal traits, which can be either discrete or continuous and linear (with minimum and maximum values) or circular (without boundaries). In these models, conformists prefer to adopt more popular cultural variants, even if these variants are far from the population mean. To measure a variant's "popularity" in cases where no two individuals share precisely the same variant on a continuum, we introduce a metric called $k$-dispersal; this takes into account a variant's distance to its $k$ closest neighbors, with more "popular" variants having lower distances to their neighbors. We demonstrate through simulations that conformity to ordinal traits need not produce a homogeneous population, as has previously been claimed. Under some combinations of parameter values, conformity sustains substantial trait variation over many generations. Anti-conformist transmission may produce high levels of polarization.
Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
Authors: Georgy Tyukin, Gbetondji J-S Dovonon, Jean Kaddour, Pasquale Minervini
Date: 2024-07-22 00:00:00
Tasks: All

The inference demand for LLMs has skyrocketed in recent months, and serving models with low latencies remains challenging due to the quadratic input length complexity of the attention layers. In this work, we investigate the effect of dropping MLP and attention layers at inference time on the performance of Llama-v2 models. We find that dropping dreeper attention layers only marginally decreases performance but leads to the best speedups alongside dropping entire layers. For example, removing 33\% of attention layers in a 13B Llama2 model results in a 1.8\% drop in average performance over the OpenLLM benchmark. We also observe that skipping layers except the latter layers reduces performances for more layers skipped, except for skipping the attention layers.