pwc-archive/papers-with-abstracts huggingface dataset:
Identification of Novel Diagnostic Neuroimaging Biomarkers for Autism Spectrum Disorder Through Convolutional Neural Network-Based Analysis of Functional, Structural, and Diffusion Tensor Imaging Data Towards Enhanced Autism Diagnosis
Authors: Annie Adhikary
Date: 2023-05-30 00:00:00
Tasks: Brain Lesion Segmentation From Mri, Classification with Binary Neural Network, Diagnostic, Motion Correction In Multishot Mri, MRI Reconstruction, MRI segmentation, Multi-Subject Fmri Data Alignment

Autism spectrum disorder is one of the leading neurodevelopmental disorders in our world, present in over 1% of the population and rapidly increasing in prevalence, yet the condition lacks a robust, objective, and efficient diagnostic. Clinical diagnostic criteria rely on subjective behavioral assessments, which are prone to misdiagnosis as they face limitations in terms of their heterogeneity, specificity, and biases. This study proposes a novel convolutional neural network-based classification tool that aims to identify the potential of different neuroimaging features as autism biomarkers. The model is constructed using a set of sequential layers specifically designed to extract relevant features from brain imaging data. Trained and tested on over 300,000 distinct features across three imaging types, the model shows promise in classifying individuals with autism from typical controls, outperforming metrics of current gold standard diagnostics by achieving an accuracy of 95.4% on a dataset of 1,111 samples with 521 autistic subjects (260 male and 261 female) and 590 controls (297 male and 293 female). 32 optimal features from the training data were identified and classified as candidate biomarkers using an independent samples t-test, in which functional features such as connectivity and the time series of signal intensity from each voxel exhibited the highest mean value differences between individuals with autism and typical control subjects. The p-values of these biomarkers were < 0.001, proving the statistical significance of the results and indicating that this research could pave the way towards the usage of neuroimaging in conjunction with behavioral criteria in clinics.
Neural correlates of cognitive ability and visuo-motor speed: validation of IDoCT on UK Biobank Data
Authors: Valentina Giunchiglia, Sharon Curtis, Stephen Smith, Naomi Allen, Adam Hampshire
Date: 2023-05-30 00:00:00

Automated online and App-based cognitive assessment tasks are becoming increasingly popular in large-scale cohorts and biobanks due to advantages in affordability, scalability and repeatability. However, the summary scores that such tasks generate typically conflate the cognitive processes that are the intended focus of assessment with basic visuomotor speeds, testing device latencies and speed-accuracy tradeoffs. This lack of precision presents a fundamental limitation when studying brain-behaviour associations. Previously, we developed a novel modelling approach that leverages continuous performance recordings from large-cohort studies to achieve an iterative decomposition of cognitive tasks (IDoCT), which outputs data-driven estimates of cognitive abilities, and device and visuomotor latencies, whilst recalibrating trial-difficulty scales. Here, we further validate the IDoCT approach with UK BioBank imaging data. First, we examine whether IDoCT can improve ability distributions and trial-difficulty scales from an adaptive picture-vocabulary task (PVT). Then, we confirm that the resultant visuomotor and cognitive estimates associate more robustly with age and education than the original PVT scores. Finally, we conduct a multimodal brain-wide association study with free-text analysis to test whether the brain regions that predict the IDoCT estimates have the expected differential relationships with visuomotor vs. language and memory labels within the broader imaging literature. Our results support the view that the rich performance timecourses recorded during computerised cognitive assessments can be leveraged with modelling frameworks like IDoCT to provide estimates of human cognitive abilities that have superior distributions, re-test reliabilities and brain-wide associations.
Wind turbine power and land cover effects on cumulative bat deaths
Authors: Aristides Moustakas, Panagiotis Georgiakakis, Elzbieta Kret, Eleftherios Kapsalis
Date: 2023-05-30 00:00:00

Wind turbines (WT) cause bird and bat mortalities which depend on the WT and landscape features. The effects of WT features and environmental variables at different spatial scales associated to bat deaths in a mountainous and forested area in Thrace, NE Greece were investigated. Initially, we sought to quantify the most lethal WT characteristic between tower height, rotor diameter and power. The scale of interaction distance between bat deaths and the land cover characteristics surrounding the WTs was quantified. A statistical model was trained and validated against bat deaths and WT, land cover and topography features. Variance partitioning between bat deaths and the explanatory covariates was conducted. The trained model was used to predict bat deaths attributed to existing and future wind farm development in the region. Results indicated that the optimal interaction distance between WT and surrounding land cover was 5 km, the larger distance than the ones examined. WT power, natural land cover type and distance from water explained 40 %, 15 % and 11 % respectively of the total variance in bat deaths by WTs. The model predicted that operating but not surveyed WTs comprise of 377.8% and licensed but not operating yet will contribute to 210.2% additional deaths than the ones recorded. Results indicate that among all WT features and land cover characteristics, wind turbine power is the most significant factor associated to bat deaths. Results indicated that WTs located within 5 km buffer comprised of natural land cover types have substantial higher deaths. More WT power will result in more deaths. Wind turbines should not be licensed in areas where natural land cover at a radius of 5km exceeds 50%. These results are discussed in the climate-land use-biodiversity-energy nexus.
Neighbor Joining And Leaf Status
Authors: Mathias Weller
Date: 2023-05-30 00:00:00

The Neighbor Joining Algorithm is among the most fundamental algorithmic results in computational biology. However, its definition and correctness proof are not straightforward. In particular, ''the question ''what does the NJ method seek to do?'' has until recently proved somewhat elusive'' [Gascuel \& Steel, 2006]. While a rigorous mathematical analysis is now available, it is still considered somewhat hard to follow and its proof tedious at best. In this work, we present an alternative interpretation of the goal of the Neighbor Joining algorithm by proving that it chooses to merge the two taxa u and v that maximize the ''leaf-status'', that is, the sum of distances of all leaves to the unique u-v-path.
Automated Feedback Generation for a Chemistry Database and Abstracting Exercise
Authors: Oscar Morris, Russell Morris
Date: 2023-05-22 00:00:00

Timely feedback is an important part of teaching and learning. Here we describe how a readily available neural network transformer (machine-learning) model (BERT) can be used to give feedback on the structure of the response to an abstracting exercise where students are asked to summarise the contents of a published article after finding it from a publication database. The dataset contained 207 submissions from two consecutive years of the course, summarising a total of 21 different papers from the primary literature. The model was pre-trained using an available dataset (approx. 15,000 samples) and then fine-tuned on 80% of the submitted dataset. This fine tuning was seen to be important. The sentences in the student submissions are characterised into three classes - background, technique and observation - which allows a comparison of how each submission is structured. Comparing the structure of the students' abstract a large collection of those from the PubMed database shows that students in this exercise concentrate more on the background to the paper and less on the techniques and results than the abstracts to papers themselves. The results allowed feedback for each submitted assignment to be automatically generated.
REFinD: Relation Extraction Financial Dataset
Authors: Simerjot Kaur, Charese Smiley, Akshat Gupta, Joy Sain, Dongsheng Wang, Suchetha Siddagangappa, Toyin Aguda, Sameena Shah
Date: 2023-05-22 00:00:00
Tasks: Articles, General Knowledge, Information Retrieval, Natural Language Inference, Question Answering, Relation, Relation Extraction, Retrieval

A number of datasets for Relation Extraction (RE) have been created to aide downstream tasks such as information retrieval, semantic search, question answering and textual entailment. However, these datasets fail to capture financial-domain specific challenges since most of these datasets are compiled using general knowledge sources such as Wikipedia, web-based text and news articles, hindering real-life progress and adoption within the financial world. To address this limitation, we propose REFinD, the first large-scale annotated dataset of relations, with $\sim$29K instances and 22 relations amongst 8 types of entity pairs, generated entirely over financial documents. We also provide an empirical evaluation with various state-of-the-art models as benchmarks for the RE task and highlight the challenges posed by our dataset. We observed that various state-of-the-art deep learning models struggle with numeric inference, relational and directional ambiguity.
Regex-augmented Domain Transfer Topic Classification based on a Pre-trained Language Model: An application in Financial Domain
Authors: Vanessa Liao, Syed Shariyar Murtaza, Yifan Nie, Jimmy Lin
Date: 2023-05-23 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, text-classification, Text Classification, Topic Classification

A common way to use large pre-trained language models for downstream tasks is to fine tune them using additional layers. This may not work well if downstream domain is a specialized domain whereas the large language model has been pre-trained on a generic corpus. In this paper, we discuss the use of regular expression patterns employed as features for domain knowledge during the process of fine tuning, in addition to domain specific text. Our experiments on real scenario production data show that this method of fine tuning improves the downstream text classification tasks as compared to fine tuning only on domain specific text. We also show that the use of attention network for fine tuning improves results compared to simple linear layers.
Adaptive Sparsity Level during Training for Efficient Time Series Forecasting with Transformers
Authors: Zahra Atashgahi, Mykola Pechenizkiy, Raymond Veldhuis, Decebal Constantin Mocanu
Date: 2023-05-28 00:00:00
Tasks: Time Series, Time Series Forecasting

Efficient time series forecasting has become critical for real-world applications, particularly with deep neural networks (DNNs). Efficiency in DNNs can be achieved through sparse connectivity and reducing the model size. However, finding the sparsity level automatically during training remains challenging due to the heterogeneity in the loss-sparsity tradeoffs across the datasets. In this paper, we propose \enquote{\textbf{P}runing with \textbf{A}daptive \textbf{S}parsity \textbf{L}evel} (\textbf{PALS}), to automatically seek a decent balance between loss and sparsity, all without the need for a predefined sparsity level. PALS draws inspiration from sparse training and during-training methods. It introduces the novel "expand" mechanism in training sparse neural networks, allowing the model to dynamically shrink, expand, or remain stable to find a proper sparsity level. In this paper, we focus on achieving efficiency in transformers known for their excellent time series forecasting performance but high computational cost. Nevertheless, PALS can be applied directly to any DNN. To this aim, we demonstrate its effectiveness also on the DLinear model. Experimental results on six benchmark datasets and five state-of-the-art (SOTA) transformer variants show that PALS substantially reduces model size while maintaining comparable performance to the dense model. More interestingly, PALS even outperforms the dense model, in \textcolor{blue}{12} and \textcolor{blue}{14} cases out of 30 cases in terms of MSE and MAE loss, respectively, while reducing \textcolor{blue}{65\%} parameter count and \textcolor{blue}{63\%} FLOPs on average. Our code and supplementary material are available on Github\footnote{\tiny \url{https://github.com/zahraatashgahi/PALS}}.
AnoRand: A Semi Supervised Deep Learning Anomaly Detection Method by Random Labeling
Authors: Mansour Zoubeirou A Mayaki, Michel Riveill
Date: 2023-05-28 00:00:00
Tasks: Anomaly Detection, Semi-supervised Anomaly Detection, Supervised Anomaly Detection, Unsupervised Anomaly Detection

Anomaly detection or more generally outliers detection is one of the most popular and challenging subject in theoretical and applied machine learning. The main challenge is that in general we have access to very few labeled data or no labels at all. In this paper, we present a new semi-supervised anomaly detection method called \textbf{AnoRand} by combining a deep learning architecture with random synthetic label generation. The proposed architecture has two building blocks: (1) a noise detection (ND) block composed of feed forward ferceptron and (2) an autoencoder (AE) block. The main idea of this new architecture is to learn one class (e.g. the majority class in case of anomaly detection) as well as possible by taking advantage of the ability of auto encoders to represent data in a latent space and the ability of Feed Forward Perceptron (FFP) to learn one class when the data is highly imbalanced. First, we create synthetic anomalies by randomly disturbing (add noise) few samples (e.g. 2\%) from the training set. Second, we use the normal and the synthetic samples as input to our model. We compared the performance of the proposed method to 17 state-of-the-art unsupervised anomaly detection method on synthetic datasets and 57 real-world datasets. Our results show that this new method generally outperforms most of the state-of-the-art methods and has the best performance (AUC ROC and AUC PR) on the vast majority of reference datasets. We also tested our method in a supervised way by using the actual labels to train the model. The results show that it has very good performance compared to most of state-of-the-art supervised algorithms.
Approximation Rate of the Transformer Architecture for Sequence Modeling
Authors: Haotian Jiang, Qianxiao Li
Date: 2023-05-29 00:00:00

The Transformer architecture is widely applied in sequence modeling applications, yet the theoretical understanding of its working principles remains limited. In this work, we investigate the approximation rate for single-layer Transformers with one head. We consider a class of non-linear relationships and identify a novel notion of complexity measures to establish an explicit Jackson-type approximation rate estimate for the Transformer. This rate reveals the structural properties of the Transformer and suggests the types of sequential relationships it is best suited for approximating. In particular, the results on approximation rates enable us to concretely analyze the differences between the Transformer and classical sequence modeling methods, such as recurrent neural networks.
Forward and Inverse Approximation Theory for Linear Temporal Convolutional Networks
Authors: Haotian Jiang, Qianxiao Li
Date: 2023-05-29 00:00:00
Tasks: Temporal Sequences

We present a theoretical analysis of the approximation properties of convolutional architectures when applied to the modeling of temporal sequences. Specifically, we prove an approximation rate estimate (Jackson-type result) and an inverse approximation theorem (Bernstein-type result), which together provide a comprehensive characterization of the types of sequential relationships that can be efficiently captured by a temporal convolutional architecture. The rate estimate improves upon a previous result via the introduction of a refined complexity measure, whereas the inverse approximation theorem is new.
Bringing regularized optimal transport to lightspeed: a splitting method adapted for GPUs
Date: 2023-05-29 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Domain Adaptation, GPU

We present an efficient algorithm for regularized optimal transport. In contrast to previous methods, we use the Douglas-Rachford splitting technique to develop an efficient solver that can handle a broad class of regularizers. The algorithm has strong global convergence guarantees, low per-iteration cost, and can exploit GPU parallelization, making it considerably faster than the state-of-the-art for many problems. We illustrate its competitiveness in several applications, including domain adaptation and learning of generative models.
SANE: The phases of gradient descent through Sharpness Adjusted Number of Effective parameters
Authors: Lawrence Wang, Stephen J. Roberts
Date: 2023-05-29 00:00:00

Modern neural networks are undeniably successful. Numerous studies have investigated how the curvature of loss landscapes can affect the quality of solutions. In this work we consider the Hessian matrix during network training. We reiterate the connection between the number of "well-determined" or "effective" parameters and the generalisation performance of neural nets, and we demonstrate its use as a tool for model comparison. By considering the local curvature, we propose Sharpness Adjusted Number of Effective parameters (SANE), a measure of effective dimensionality for the quality of solutions. We show that SANE is robust to large learning rates, which represent learning regimes that are attractive but (in)famously unstable. We provide evidence and characterise the Hessian shifts across "loss basins" at large learning rates. Finally, extending our analysis to deeper neural networks, we provide an approximation to the full-network Hessian, exploiting the natural ordering of neural weights, and use this approximation to provide extensive empirical evidence for our claims.
Towards a Better Understanding of Representation Dynamics under TD-learning
Authors: Yunhao Tang, Rémi Munos
Date: 2023-05-29 00:00:00
Tasks: Reinforcement Learning (RL), Representation Learning, Value prediction

TD-learning is a foundation reinforcement learning (RL) algorithm for value prediction. Critical to the accuracy of value predictions is the quality of state representations. In this work, we consider the question: how does end-to-end TD-learning impact the representation over time? Complementary to prior work, we provide a set of analysis that sheds further light on the representation dynamics under TD-learning. We first show that when the environments are reversible, end-to-end TD-learning strictly decreases the value approximation error over time. Under further assumptions on the environments, we can connect the representation dynamics with spectral decomposition over the transition matrix. This latter finding establishes fitting multiple value functions from randomly generated rewards as a useful auxiliary task for representation learning, as we empirically validate on both tabular and Atari game suites.
DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm
Authors: Yunhao Tang, Tadashi Kozuno, Mark Rowland, Anna Harutyunyan, Rémi Munos, Bernardo Ávila Pires, Michal Valko
Date: 2023-05-29 00:00:00

Multi-step learning applies lookahead over multiple time steps and has proved valuable in policy evaluation settings. However, in the optimal control case, the impact of multi-step learning has been relatively limited despite a number of prior efforts. Fundamentally, this might be because multi-step policy improvements require operations that cannot be approximated by stochastic samples, hence hindering the widespread adoption of such methods in practice. To address such limitations, we introduce doubly multi-step off-policy VI (DoMo-VI), a novel oracle algorithm that combines multi-step policy improvements and policy evaluations. DoMo-VI enjoys guaranteed convergence speed-up to the optimal policy and is applicable in general off-policy learning settings. We then propose doubly multi-step off-policy actor-critic (DoMo-AC), a practical instantiation of the DoMo-VI algorithm. DoMo-AC introduces a bias-variance trade-off that ensures improved policy gradient estimates. When combined with the IMPALA architecture, DoMo-AC has showed improvements over the baseline algorithm on Atari-57 game benchmarks.
Generalized Disparate Impact for Configurable Fairness Solutions in ML
Authors: Luca Giuliani, Eleonora Misino, Michele Lombardi
Date: 2023-05-29 00:00:00
Tasks: Fairness

We make two contributions in the field of AI fairness over continuous protected attributes. First, we show that the Hirschfeld-Gebelein-Renyi (HGR) indicator (the only one currently available for such a case) is valuable but subject to a few crucial limitations regarding semantics, interpretability, and robustness. Second, we introduce a family of indicators that are: 1) complementary to HGR in terms of semantics; 2) fully interpretable and transparent; 3) robust over finite samples; 4) configurable to suit specific applications. Our approach also allows us to define fine-grained constraints to permit certain types of dependence and forbid others selectively. By expanding the available options for continuous protected attributes, our approach represents a significant contribution to the area of fair artificial intelligence.
BigVideo: A Large-scale Video Subtitle Translation Dataset for Multimodal Machine Translation
Authors: Liyan Kang, Luyang Huang, Ningxin Peng, Peihao Zhu, Zewei Sun, Shanbo Cheng, Mingxuan Wang, Degen Huang, Jinsong Su
Date: 2023-05-23 00:00:00
Tasks: Contrastive Learning, Machine Translation, Multimodal Machine Translation, NMT, Sentence, Translation

We present a large-scale video subtitle translation dataset, BigVideo, to facilitate the study of multi-modality machine translation. Compared with the widely used How2 and VaTeX datasets, BigVideo is more than 10 times larger, consisting of 4.5 million sentence pairs and 9,981 hours of videos. We also introduce two deliberately designed test sets to verify the necessity of visual information: Ambiguous with the presence of ambiguous words, and Unambiguous in which the text context is self-contained for translation. To better model the common semantics shared across texts and videos, we introduce a contrastive learning method in the cross-modal encoder. Extensive experiments on the BigVideo show that: a) Visual information consistently improves the NMT model in terms of BLEU, BLEURT, and COMET on both Ambiguous and Unambiguous test sets. b) Visual information helps disambiguation, compared to the strong text baseline on terminology-targeted scores and human evaluation. Dataset and our implementations are available at https://github.com/DeepLearnXMU/BigVideo-VMT.