pwc-archive/papers-with-abstracts huggingface dataset:
Neuromorphic circuit for temporal odor encoding in turbulent environments
Authors: Shavika Rastogi, Nik Dennler, Michael Schmuker, André van Schaik
Date: 2024-12-28 00:00:00

Natural odor environments present turbulent and dynamic conditions, causing chemical signals to fluctuate in space, time, and intensity. While many species have evolved highly adaptive behavioral responses to such variability, the emerging field of neuromorphic olfaction continues to grapple with the challenge of efficiently sampling and identifying odors in real-time. In this work, we investigate Metal-Oxide (MOx) gas sensor recordings of constant airflow-embedded artificial odor plumes. We discover a data feature that is representative of the presented odor stimulus at a certain concentration - irrespective of temporal variations caused by the plume dynamics. Further, we design a neuromorphic electronic nose front-end circuit for extracting and encoding this feature into analog spikes for gas detection and concentration estimation. The design is inspired by the spiking output of parallel neural pathways in the mammalian olfactory bulb. We test the circuit for gas recognition and concentration estimation in artificial environments, where either single gas pulses or pre-recorded odor plumes were deployed in a constant flow of air. For both environments, our results indicate that the gas concentration is encoded in -- and inversely proportional to the time difference of analog spikes emerging out of two parallel pathways, similar to the spiking output of a mammalian olfactory bulb. The resulting neuromorphic nose could enable data-efficient, real-time robotic plume navigation systems, advancing the capabilities of odor source localization in applications such as environmental monitoring and search-and-rescue.
Improving SSVEP BCI Spellers With Data Augmentation and Language Models
Authors: Joseph Zhang, Ruiming Zhang, Kipngeno Koech, David Hill, Kateryna Shapovalenko
Date: 2024-12-28 00:00:00
Tasks: Brain Computer Interface, Data Augmentation, EEG, Language Modeling, Language Modelling, SSVEP

Steady-State Visual Evoked Potential (SSVEP) spellers are a promising communication tool for individuals with disabilities. This Brain-Computer Interface utilizes scalp potential data from (electroencephalography) EEG electrodes on a subject's head to decode specific letters or arbitrary targets the subject is looking at on a screen. However, deep neural networks for SSVEP spellers often suffer from low accuracy and poor generalizability to unseen subjects, largely due to the high variability in EEG data. In this study, we propose a hybrid approach combining data augmentation and language modeling to enhance the performance of SSVEP spellers. Using the Benchmark dataset from Tsinghua University, we explore various data augmentation techniques, including frequency masking, time masking, and noise injection, to improve the robustness of deep learning models. Additionally, we integrate a language model (CharRNN) with EEGNet to incorporate linguistic context, significantly enhancing word-level decoding accuracy. Our results demonstrate accuracy improvements of up to 2.9 percent over the baseline, with time masking and language modeling showing the most promise. This work paves the way for more accurate and generalizable SSVEP speller systems, offering improved communication solutions for individuals with disabilities.
Children's Acquisition of Tail-recursion Sequences: A Review of Locative Recursion and Possessive Recursion as Examples
Authors: Xiaoyi Wang, Chenxi Fu, Caimei Yang, Ziman Zhuang
Date: 2024-12-28 00:00:00
Tasks: Language Acquisition

Recursion is the nature of human natural language. Since Chomsky proposed generative grammar, many scholars have studied recursion either theoretically or empirically. However, by observing children's acquisition of tail recursion sequences, we can verify the nativism of language supported by universal grammar and reveal the cognitive mechanism of human brain. To date, our understanding of children's acquisition path of recursion and influencing factors still remain controversial. This systematic review summarizes the research of tail recursive sequence by taking possessive recursion and locative recursion as examples, focusing on the experimental methods, acquisition paths, and influencing factors of tail recursive sequence. The current behavioural experiments reveal that, the debate about children's performance revolves around: 1) Gradual acquisition or synchronous acquisition. 2) symmetry or asymmetry between the acquisition of locative recursion sequences and possessive recursion sequences. We presume that children can acquire recursion quickly in a short period of time thanks to the language acquisition device, though there are also scholars who believe that a third factor also plays a role.
Self-Assembly of a Biologically Plausible Learning Circuit
Authors: Qianli Liao, Liu Ziyin, Yulu Gan, Brian Cheung, Mark Harnett, Tomaso Poggio
Date: 2024-12-28 00:00:00
Tasks: Anatomy

Over the last four decades, the amazing success of deep learning has been driven by the use of Stochastic Gradient Descent (SGD) as the main optimization technique. The default implementation for the computation of the gradient for SGD is backpropagation, which, with its variations, is used to this day in almost all computer implementations. From the perspective of neuroscientists, however, the consensus is that backpropagation is unlikely to be used by the brain. Though several alternatives have been discussed, none is so far supported by experimental evidence. Here we propose a circuit for updating the weights in a network that is biologically plausible, works as well as backpropagation, and leads to verifiable predictions about the anatomy and the physiology of a characteristic motif of four plastic synapses between ascending and descending cortical streams. A key prediction of our proposal is a surprising property of self-assembly of the basic circuit, emerging from initial random connectivity and heterosynaptic plasticity rules.
Low coordinate degree algorithms II: Categorical signals and generalized stochastic block models
Authors: Dmitriy Kunisky
Date: 2024-12-30 00:00:00
Tasks: Stochastic Block Model

We study when low coordinate degree functions (LCDF) -- linear combinations of functions depending on small subsets of entries of a vector -- can test for the presence of categorical structure, including community structure and generalizations thereof, in high-dimensional data. This complements the first paper of this series, which studied the power of LCDF in testing for continuous structure like real-valued signals perturbed by additive noise. We apply the tools developed there to a general form of stochastic block model (SBM), where a population is assigned random labels and every $p$-tuple of the population generates an observation according to an arbitrary probability measure associated to the $p$ labels of its members. We show that the performance of LCDF admits a unified analysis for this class of models. As applications, we prove tight lower bounds against LCDF (and therefore also against low degree polynomials) for nearly arbitrary graph and regular hypergraph SBMs, always matching suitable generalizations of the Kesten-Stigum threshold. We also prove tight lower bounds for group synchronization and abelian group sumset problems under the "truth-or-Haar" noise model, and use our technical results to give an improved analysis of Gaussian multi-frequency group synchronization. In most of these models, for some parameter settings our lower bounds give new evidence for conjectural statistical-to-computational gaps. Finally, interpreting some of our findings, we propose a precise analogy between categorical and continuous signals: a general SBM as above behaves, in terms of the tradeoff between subexponential runtime cost of testing algorithms and the signal strength needed for a testing algorithm to succeed, like a spiked $p_*$-tensor model of a certain order $p_*$ that may be computed from the parameters of the SBM.
LEARNER: A Transfer Learning Method for Low-Rank Matrix Estimation
Authors: Sean McGrath, Cenhao Zhu, Min Guo, Rui Duan
Date: 2024-12-29 00:00:00
Tasks: Transfer Learning

Low-rank matrix estimation is a fundamental problem in statistics and machine learning. In the context of heterogeneous data generated from diverse sources, a key challenge lies in leveraging data from a source population to enhance the estimation of a low-rank matrix in a target population of interest. One such example is estimating associations between genetic variants and diseases in non-European ancestry groups. We propose an approach that leverages similarity in the latent row and column spaces between the source and target populations to improve estimation in the target population, which we refer to as LatEnt spAce-based tRaNsfer lEaRning (LEARNER). LEARNER is based on performing a low-rank approximation of the target population data which penalizes differences between the latent row and column spaces between the source and target populations. We present a cross-validation approach that allows the method to adapt to the degree of heterogeneity across populations. We conducted extensive simulations which found that LEARNER often outperforms the benchmark approach that only uses the target population data, especially as the signal-to-noise ratio in the source population increases. We also performed an illustrative application and empirical comparison of LEARNER and benchmark approaches in a re-analysis of a genome-wide association study in the BioBank Japan cohort. LEARNER is implemented in the R package learner.
Stochastic gradient descent estimation of generalized matrix factorization models with application to single-cell RNA sequencing data
Authors: Cristian Castiglione, Alexandre Segers, Lieven Clement, Davide Risso
Date: 2024-12-29 00:00:00
Tasks: Dimensionality Reduction, Missing Values, Model Selection

Single-cell RNA sequencing allows the quantitation of gene expression at the individual cell level, enabling the study of cellular heterogeneity and gene expression dynamics. Dimensionality reduction is a common preprocessing step to simplify the visualization, clustering, and phenotypic characterization of samples. This step, often performed using principal component analysis or closely related methods, is challenging because of the size and complexity of the data. In this work, we present a generalized matrix factorization model assuming a general exponential dispersion family distribution and we show that many of the proposed approaches in the single-cell dimensionality reduction literature can be seen as special cases of this model. Furthermore, we propose a scalable adaptive stochastic gradient descent algorithm that allows us to estimate the model efficiently, enabling the analysis of millions of cells. Our contribution extends to introducing a novel warm start initialization method, designed to accelerate algorithm convergence and increase the precision of final estimates. Moreover, we discuss strategies for dealing with missing values and model selection. We benchmark the proposed algorithm through extensive numerical experiments against state-of-the-art methods and showcase its use in real-world biological applications. The proposed method systematically outperforms existing methods of both generalized and non-negative matrix factorization, demonstrating faster execution times while maintaining, or even enhancing, matrix reconstruction fidelity and accuracy in biological signal extraction. Finally, all the methods discussed here are implemented in an efficient open-source R package, sgdGMF, available at github/CristianCastiglione/sgdGMF
When the whole is greater than the sum of its parts: Scaling black-box inference to large data settings through divide-and-conquer
Authors: Emily C. Hector, Amanda Lenzi
Date: 2024-12-29 00:00:00
Tasks: Uncertainty Quantification

Black-box methods such as deep neural networks are exceptionally fast at obtaining point estimates of model parameters due to their amortisation of the loss function computation, but are currently restricted to settings for which simulating training data is inexpensive. When simulating data is computationally expensive, both the training and uncertainty quantification, which typically relies on a parametric bootstrap, become intractable. We propose a black-box divide-and-conquer estimation and inference framework when data simulation is computationally expensive that trains a black-box estimation method on a partition of the multivariate data domain, estimates and bootstraps on the partitioned data, and combines estimates and inferences across data partitions. Through the divide step, only small training data need be simulated, substantially accelerating the training. Further, the estimation and bootstrapping can be conducted in parallel across multiple computing nodes to further speed up the procedure. Finally, the conquer step accounts for any dependence between data partitions through a statistically and computationally efficient weighted average. We illustrate the implementation of our framework in high-dimensional spatial settings with Gaussian and max-stable processes. Applications to modeling extremal temperature data from both a climate model and observations from the National Oceanic and Atmospheric Administration highlight the feasibility of estimation and inference of max-stable process parameters with tens of thousands of locations.
Unsupervised dense retrieval with conterfactual contrastive learning
Authors: Haitian Chen, Qingyao Ai, Xiao Wang, Yiqun Liu, Fen Lin, Qin Liu
Date: 2024-12-30 00:00:00
Tasks: Contrastive Learning, counterfactual, Information Retrieval, Retrieval, Sensitivity

Efficiently retrieving a concise set of candidates from a large document corpus remains a pivotal challenge in Information Retrieval (IR). Neural retrieval models, particularly dense retrieval models built with transformers and pretrained language models, have been popular due to their superior performance. However, criticisms have also been raised on their lack of explainability and vulnerability to adversarial attacks. In response to these challenges, we propose to improve the robustness of dense retrieval models by enhancing their sensitivity of fine-graned relevance signals. A model achieving sensitivity in this context should exhibit high variances when documents' key passages determining their relevance to queries have been modified, while maintaining low variances for other changes in irrelevant passages. This sensitivity allows a dense retrieval model to produce robust results with respect to attacks that try to promote documents without actually increasing their relevance. It also makes it possible to analyze which part of a document is actually relevant to a query, and thus improve the explainability of the retrieval model. Motivated by causality and counterfactual analysis, we propose a series of counterfactual regularization methods based on game theory and unsupervised learning with counterfactual passages. Experiments show that, our method can extract key passages without reliance on the passage-level relevance annotations. Moreover, the regularized dense retrieval models exhibit heightened robustness against adversarial attacks, surpassing the state-of-the-art anti-attack methods.
AmalREC: A Dataset for Relation Extraction and Classification Leveraging Amalgamation of Large Language Models
Authors: Mansi, Pranshu Pandya, Mahek Bhavesh Vora, Soumya Bharadwaj, Ashish Anand
Date: 2024-12-29 00:00:00
Tasks: Relation, Relation Classification, Relation Extraction, Sentence

Existing datasets for relation classification and extraction often exhibit limitations such as restricted relation types and domain-specific biases. This work presents a generic framework to generate well-structured sentences from given tuples with the help of Large Language Models (LLMs). This study has focused on the following major questions: (i) how to generate sentences from relation tuples, (ii) how to compare and rank them, (iii) can we combine strengths of individual methods and amalgamate them to generate an even bette quality of sentences, and (iv) how to evaluate the final dataset? For the first question, we employ a multifaceted 5-stage pipeline approach, leveraging LLMs in conjunction with template-guided generation. We introduce Sentence Evaluation Index(SEI) that prioritizes factors like grammatical correctness, fluency, human-aligned sentiment, accuracy, and complexity to answer the first part of the second question. To answer the second part of the second question, this work introduces a SEI-Ranker module that leverages SEI to select top candidate generations. The top sentences are then strategically amalgamated to produce the final, high-quality sentence. Finally, we evaluate our dataset on LLM-based and SOTA baselines for relation classification. The proposed dataset features 255 relation types, with 15K sentences in the test set and around 150k in the train set organized in, significantly enhancing relational diversity and complexity. This work not only presents a new comprehensive benchmark dataset for RE/RC task, but also compare different LLMs for generation of quality sentences from relational tuples.
Introducing Semantic Capability in LinkedIn's Content Search Engine
Authors: Xin Yang, Rachel Zheng, Madhumitha Mohan, Sonali Bhadra, Pansul Bhatt, Lingyu, Zhang, Rupesh Gupta
Date: 2024-12-29 00:00:00
Tasks: Natural Language Queries

In the past, most search queries issued to a search engine were short and simple. A keyword based search engine was able to answer such queries quite well. However, members are now developing the habit of issuing long and complex natural language queries. Answering such queries requires evolution of a search engine to have semantic capability. In this paper we present the design of LinkedIn's new content search engine with semantic capability, and its impact on metrics.
Left-handed representation in top 100 male professional tennis players: Multi-disciplinary perspectives
Authors: Boris Bačić, Ali Ghazala
Date: 2024-12-29 00:00:00

A commonly held opinion is that left-handed tennis players are overrepresented compared to the percentage of left-handers within the general population. This study provides the domain insights supported by data analysis that could help inform the decision of parents and coaches considering whether a child should start playing tennis as left- or right-handed when there is no strong arm-handed dominance. Compared to the commonly cited figure of about 10% of left-handed male population, data analysis from the official ATP web site for the top 100 ranked tennis players over the past decades (1985-2016) shows evidence of overrepresentation of left-handed elite tennis players (about 15%). The insights and data analysis can inform the handedness decision, advance coaching and strategic game concepts, enhance media coverage/analytics, left-handed facts and statistics, and inform tennis equipment manufacturing.
A Contrastive Pretrain Model with Prompt Tuning for Multi-center Medication Recommendation
Authors: Qidong Liu, Zhaopeng Qiu, Xiangyu Zhao, Xian Wu, Zijian Zhang, Tong Xu, Feng Tian
Date: 2024-12-28 00:00:00

Medication recommendation is one of the most critical health-related applications, which has attracted extensive research interest recently. Most existing works focus on a single hospital with abundant medical data. However, many small hospitals only have a few records, which hinders applying existing medication recommendation works to the real world. Thus, we seek to explore a more practical setting, i.e., multi-center medication recommendation. In this setting, most hospitals have few records, but the total number of records is large. Though small hospitals may benefit from total affluent records, it is also faced with the challenge that the data distributions between various hospitals are much different. In this work, we introduce a novel conTrastive prEtrain Model with Prompt Tuning (TEMPT) for multi-center medication recommendation, which includes two stages of pretraining and finetuning. We first design two self-supervised tasks for the pretraining stage to learn general medical knowledge. They are mask prediction and contrastive tasks, which extract the intra- and inter-relationships of input diagnosis and procedures. Furthermore, we devise a novel prompt tuning method to capture the specific information of each hospital rather than adopting the common finetuning. On the one hand, the proposed prompt tuning can better learn the heterogeneity of each hospital to fit various distributions. On the other hand, it can also relieve the catastrophic forgetting problem of finetuning. To validate the proposed model, we conduct extensive experiments on the public eICU, a multi-center medical dataset. The experimental results illustrate the effectiveness of our model. The implementation code is available to ease the reproducibility https://github.com/Applied-Machine-Learning-Lab/TEMPT.
Invariant debiasing learning for recommendation via biased imputation
Authors: Ting Bai, WeiJie Chen, Cheng Yang, Chuan Shi
Date: 2024-12-28 00:00:00
Tasks: Imputation, Knowledge Distillation, Recommendation Systems

Previous debiasing studies utilize unbiased data to make supervision of model training. They suffer from the high trial risks and experimental costs to obtain unbiased data. Recent research attempts to use invariant learning to detach the invariant preference of users for unbiased recommendations in an unsupervised way. However, it faces the drawbacks of low model accuracy and unstable prediction performance due to the losing cooperation with variant preference. In this paper, we experimentally demonstrate that invariant learning causes information loss by directly discarding the variant information, which reduces the generalization ability and results in the degradation of model performance in unbiased recommendations. Based on this consideration, we propose a novel lightweight knowledge distillation framework (KDDebias) to automatically learn the unbiased preference of users from both invariant and variant information. Specifically, the variant information is imputed to the invariant user preference in the distance-aware knowledge distillation process. Extensive experiments on three public datasets, i.e., Yahoo!R3, Coat, and MIND, show that with the biased imputation from the variant preference of users, our proposed method achieves significant improvements with less than 50% learning parameters compared to the SOTA unsupervised debiasing model in recommender systems. Our code are publicly available at https://github.com/BAI-LAB/KD-Debias.
ERPA: Efficient RPA Model Integrating OCR and LLMs for Intelligent Document Processing
Authors: Osama Abdellaif, Abdelrahman Nader, Ali Hamdi
Date: 2024-12-24 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

This paper presents ERPA, an innovative Robotic Process Automation (RPA) model designed to enhance ID data extraction and optimize Optical Character Recognition (OCR) tasks within immigration workflows. Traditional RPA solutions often face performance limitations when processing large volumes of documents, leading to inefficiencies. ERPA addresses these challenges by incorporating Large Language Models (LLMs) to improve the accuracy and clarity of extracted text, effectively handling ambiguous characters and complex structures. Benchmark comparisons with leading platforms like UiPath and Automation Anywhere demonstrate that ERPA significantly reduces processing times by up to 94 percent, completing ID data extraction in just 9.94 seconds. These findings highlight ERPA's potential to revolutionize document automation, offering a faster and more reliable alternative to current RPA solutions.