pwc-archive/papers-with-abstracts huggingface dataset:
Revisiting Convergence of AdaGrad with Relaxed Assumptions
Authors: Yusu Hong, Junhong Lin
Date: 2024-02-21 00:00:00

In this study, we revisit the convergence of AdaGrad with momentum (covering AdaGrad as a special case) on non-convex smooth optimization problems. We consider a general noise model where the noise magnitude is controlled by the function value gap together with the gradient magnitude. This model encompasses a broad range of noises including bounded noise, sub-Gaussian noise, affine variance noise and the expected smoothness, and it has been shown to be more realistic in many practical applications. Our analysis yields a probabilistic convergence rate which, under the general noise, could reach at (\tilde{\mathcal{O}}(1/\sqrt{T})). This rate does not rely on prior knowledge of problem-parameters and could accelerate to (\tilde{\mathcal{O}}(1/T)) where (T) denotes the total number iterations, when the noise parameters related to the function value gap and noise level are sufficiently small. The convergence rate thus matches the lower rate for stochastic first-order methods over non-convex smooth landscape up to logarithm terms [Arjevani et al., 2023]. We further derive a convergence bound for AdaGrad with mometum, considering the generalized smoothness where the local smoothness is controlled by a first-order function of the gradient norm.
Full-Atom Peptide Design with Geometric Latent Diffusion
Authors: Xiangzhe Kong, Yinjun Jia, Wenbing Huang, Yang Liu
Date: 2024-02-21 00:00:00

Peptide design plays a pivotal role in therapeutics, allowing brand new possibility to leverage target binding sites that are previously undruggable. Most existing methods are either inefficient or only concerned with the target-agnostic design of 1D sequences. In this paper, we propose a generative model for full-atom \textbf{Pep}tide design with \textbf{G}eometric \textbf{LA}tent \textbf{D}iffusion (PepGLAD) given the binding site. We first establish a benchmark consisting of both 1D sequences and 3D structures from Protein Data Bank (PDB) and literature for systematic evaluation. We then identify two major challenges of leveraging current diffusion-based models for peptide design: the full-atom geometry and the variable binding geometry. To tackle the first challenge, PepGLAD derives a variational autoencoder that first encodes full-atom residues of variable size into fixed-dimensional latent representations, and then decodes back to the residue space after conducting the diffusion process in the latent space. For the second issue, PepGLAD explores a receptor-specific affine transformation to convert the 3D coordinates into a shared standard space, enabling better generalization ability across different binding shapes. Experimental Results show that our method not only improves diversity and binding affinity significantly in the task of sequence-structure co-design, but also excels at recovering reference structures for binding conformation generation.
Emergence and dynamics of delusions and hallucinations across stages in early psychosis
Authors: Catalina Mourgues-Codern, David Benrimoh, Jay Gandhi, Emily A. Farina, Raina Vin, Tihare Zamorano, Deven Parekh, Ashok Malla, Ridha Joober, Martin Lepage, Srividya N. Iyer, Jean Addington, Carrie E. Bearden, Kristin S. Cadenhead, Barbara Cornblatt, Matcheri Keshavan, William S. Stone, Daniel H. Mathalon, Diana O. Perkins, Elaine F. Walker, Tyrone D. Cannon, Scott W. Woods, Jai L. Shah, Albert R. Powers
Date: 2024-02-20 00:00:00
Tasks: Hallucination

Hallucinations and delusions are often grouped together within the positive symptoms of psychosis. However, recent evidence suggests they may be driven by distinct computational and neural mechanisms. Examining the time course of their emergence may provide insights into the relationship between these underlying mechanisms. Participants from the second (N = 719) and third (N = 699) iterations of the North American Prodrome Longitudinal Study (NAPLS 2 and 3) were assessed for timing of CHR-P-level delusion and hallucination onset. Pre-onset symptom patterns in first-episode psychosis patients (FEP) from the Prevention and Early Intervention Program for Psychosis (PEPP-Montreal; N = 694) were also assessed. Symptom onset was determined at baseline assessment and the evolution of symptom patterns examined over 24 months. In all three samples, participants were more likely to report the onset of delusion-spectrum symptoms prior to hallucination-spectrum symptoms (odds ratios (OR): NAPLS 2 = 4.09; NAPLS 3 = 4.14; PEPP, Z = 7.01, P < 0.001) and to present with only delusions compared to only hallucinations (OR: NAPLS 2 = 5.6; NAPLS 3 = 11.11; PEPP = 42.75). Re-emergence of delusions after remission was also more common than re-emergence of hallucinations (Ps < 0.05), and hallucinations more often resolved first (Ps < 0.001). In both CHR-P samples, ratings of delusional ideation fell with the onset of hallucinations (P = 0.007). Delusions tend to emerge before hallucinations and may play a role in their development. Further work should examine the relationship between the mechanisms driving these symptoms and its utility for diagnosis and treatment.
Implementation of a Model of the Cortex Basal Ganglia Loop
Authors: Naoya Arakawa
Date: 2024-02-17 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning

This article presents a simple model of the cortex-basal ganglia-thalamus loop, which is thought to serve for action selection and executions, and reports the results of its implementation. The model is based on the hypothesis that the cerebral cortex predicts actions, while the basal ganglia use reinforcement learning to decide whether to perform the actions predicted by the cortex. The implementation is intended to be used as a component of models of the brain consisting of cortical regions or brain-inspired cognitive architectures.
Network nestedness in primates: a structural constraint or a biological advantage of social complexity?
Authors: Maxime Herbrich, Eythan Cousin, Ivan Puga-Gonzalez, Barbara Tiddi, Claudia Fichtel, Meg Crofoot, Andrew Jj Macintosh, Erica van de Waal, Cédric Sueur
Date: 2024-02-21 00:00:00

This study investigates the prevalence and implications of nestedness within primate social networks, examining its relationship with cognitive and structural factors. We analysed data from 51 primate groups across 21 species, employing network analysis to evaluate nestedness and its correlation with modularity, neocortex ratio, and group size. We used Bayesian mixed effects modelling to investigate nestedness in primate social networks, controlling for phylogenetic dependencies and exploring various factors like neocortex ratio and group size. Our findings reveal a significant occurrence of nestedness in 66% of the species studied, exceeding chance expectations. This nestedness was more pronounced in groups with less steep dominance hierarchies, contrary to traditional assumptions linking it to hierarchical social structures. A notable inverse relationship between nestedness and modularity was observed, suggesting a structural trade-off in network formation. This pattern persisted even after controlling for species-specific social behaviours, indicating a general structural feature of primate networks. Surprisingly, our analysis showed no significant correlation between nestedness and neocortex ratio or group size, challenging the social brain hypothesis and suggesting a greater role for ecological factors in cognitive evolution. This study emphasises the importance of weak links in maintaining network resilience. Overall, our research provides new insights into primate social network structures, highlighting complex interplays between network characteristics and challenging existing paradigms in cognitive and evolutionary biology.
A new characterization of second-order stochastic dominance
Authors: Yuanying Guan, Muqiao Huang, Ruodu Wang
Date: 2024-02-20 00:00:00
Tasks: Management, Position

We provide a new characterization of second-order stochastic dominance, also known as increasing concave order. The result has an intuitive interpretation that adding a risk with negative expected value in adverse scenarios makes the resulting position generally less desirable for risk-averse agents. A similar characterization is also found for convex order and increasing convex order. The proof techniques for the main result are based on properties of Expected Shortfall, a family of risk measures that is popular in banking and insurance regulation. Applications in risk management and insurance are discussed.
KGroot: Enhancing Root Cause Analysis through Knowledge Graphs and Graph Convolutional Neural Networks
Authors: Tingting Wang, Guilin Qi, Tianxing Wu
Date: 2024-02-11 00:00:00
Tasks: Diagnostic, Fault Detection, Fault Diagnosis, Fault localization, Knowledge Graphs

Fault localization is challenging in online micro-service due to the wide variety of monitoring data volume, types, events and complex interdependencies in service and components. Faults events in services are propagative and can trigger a cascade of alerts in a short period of time. In the industry, fault localization is typically conducted manually by experienced personnel. This reliance on experience is unreliable and lacks automation. Different modules present information barriers during manual localization, making it difficult to quickly align during urgent faults. This inefficiency lags stability assurance to minimize fault detection and repair time. Though actionable methods aimed to automatic the process, the accuracy and efficiency are less than satisfactory. The precision of fault localization results is of paramount importance as it underpins engineers trust in the diagnostic conclusions, which are derived from multiple perspectives and offer comprehensive insights. Therefore, a more reliable method is required to automatically identify the associative relationships among fault events and propagation path. To achieve this, KGroot uses event knowledge and the correlation between events to perform root cause reasoning by integrating knowledge graphs and GCNs for RCA. FEKG is built based on historical data, an online graph is constructed in real-time when a failure event occurs, and the similarity between each knowledge graph and online graph is compared using GCNs to pinpoint the fault type through a ranking strategy. Comprehensive experiments demonstrate KGroot can locate the root cause with accuracy of 93.5% top 3 potential causes in second-level. This performance matches the level of real-time fault diagnosis in the industrial environment and significantly surpasses state-of-the-art baselines in RCA in terms of effectiveness and efficiency.
Contrastive Training of Complex-Valued Autoencoders for Object Discovery
Authors: Aleksandar Stanić, Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber
Date: 2023-05-24 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Contrastive Learning, Object, Object Discovery

Current state-of-the-art object-centric models use slots and attention-based routing for binding. However, this class of models has several conceptual limitations: the number of slots is hardwired; all slots have equal capacity; training has high computational cost; there are no object-level relational factors within slots. Synchrony-based models in principle can address these limitations by using complex-valued activations which store binding information in their phase components. However, working examples of such synchrony-based models have been developed only very recently, and are still limited to toy grayscale datasets and simultaneous storage of less than three objects in practice. Here we introduce architectural modifications and a novel contrastive learning method that greatly improve the state-of-the-art synchrony-based model. For the first time, we obtain a class of synchrony-based models capable of discovering objects in an unsupervised manner in multi-object color datasets and simultaneously representing more than three objects.
LexC-Gen: Generating Data for Extremely Low-Resource Languages with Large Language Models and Bilingual Lexicons
Authors: Zheng-Xin Yong, Cristina Menghini, Stephen H. Bach
Date: 2024-02-21 00:00:00
Tasks: Sentiment Analysis, Topic Classification, Translation, Word Translation

Data scarcity in low-resource languages can be addressed with word-to-word translations from labeled task data in high-resource languages using bilingual lexicons. However, bilingual lexicons often have limited lexical overlap with task data, which results in poor translation coverage and lexicon utilization. We propose lexicon-conditioned data generation LexC-Gen, a method that generates low-resource-language classification task data at scale. Specifically, LexC-Gen first uses high-resource-language words from bilingual lexicons to generate lexicon-compatible task data, and then it translates them into low-resource languages with bilingual lexicons via word translation. Across 17 extremely low-resource languages, LexC-Gen generated data is competitive with expert-translated gold data, and yields on average 5.6 and 8.9 points improvement over existing lexicon-based word translation methods on sentiment analysis and topic classification tasks respectively. Through ablation study, we show that conditioning on bilingual lexicons is the key component of LexC-Gen. LexC-Gen serves as a potential solution to close the performance gap between open-source multilingual models, such as BLOOMZ and Aya-101, and state-of-the-art commercial models like GPT-4o on low-resource-language tasks.
Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
Authors: Yujia Huang, Adishree Ghatare, Yuanzhe Liu, Ziniu Hu, Qinsheng Zhang, Chandramouli S Sastry, Siddharth Gururani, Sageev Oore, Yisong Yue
Date: 2024-02-22 00:00:00
Tasks: Music Generation

We study the problem of symbolic music generation (e.g., generating piano rolls), with a technical focus on non-differentiable rule guidance. Musical rules are often expressed in symbolic form on note characteristics, such as note density or chord progression, many of which are non-differentiable which pose a challenge when using them for guided diffusion. We propose Stochastic Control Guidance (SCG), a novel guidance method that only requires forward evaluation of rule functions that can work with pre-trained diffusion models in a plug-and-play way, thus achieving training-free guidance for non-differentiable rules for the first time. Additionally, we introduce a latent diffusion architecture for symbolic music generation with high time resolution, which can be composed with SCG in a plug-and-play fashion. Compared to standard strong baselines in symbolic music generation, this framework demonstrates marked advancements in music quality and rule-based controllability, outperforming current state-of-the-art generators in a variety of settings. For detailed demonstrations, code and model checkpoints, please visit our project website: https://scg-rule-guided-music.github.io/.
TIE-KD: Teacher-Independent and Explainable Knowledge Distillation for Monocular Depth Estimation
Authors: Sangwon Choi, Daejune Choi, Duksu Kim
Date: 2024-02-22 00:00:00
Tasks: Depth Estimation, Knowledge Distillation, Monocular Depth Estimation, Transfer Learning

Monocular depth estimation (MDE) is essential for numerous applications yet is impeded by the substantial computational demands of accurate deep learning models. To mitigate this, we introduce a novel Teacher-Independent Explainable Knowledge Distillation (TIE-KD) framework that streamlines the knowledge transfer from complex teacher models to compact student networks, eliminating the need for architectural similarity. The cornerstone of TIE-KD is the Depth Probability Map (DPM), an explainable feature map that interprets the teacher's output, enabling feature-based knowledge distillation solely from the teacher's response. This approach allows for efficient student learning, leveraging the strengths of feature-based distillation. Extensive evaluation of the KITTI dataset indicates that TIE-KD not only outperforms conventional response-based KD methods but also demonstrates consistent efficacy across diverse teacher and student architectures. The robustness and adaptability of TIE-KD underscore its potential for applications requiring efficient and interpretable models, affirming its practicality for real-world deployment.
Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition
Authors: Feng Lu, Lijun Zhang, Xiangyuan Lan, Shuting Dong, YaoWei Wang, Chun Yuan
Date: 2024-02-22 00:00:00
Tasks: Re-Ranking, Visual Place Recognition

Recent studies show that vision models pre-trained in generic visual learning tasks with large-scale data can provide useful feature representations for a wide range of visual perception problems. However, few attempts have been made to exploit pre-trained foundation models in visual place recognition (VPR). Due to the inherent difference in training objectives and data between the tasks of model pre-training and VPR, how to bridge the gap and fully unleash the capability of pre-trained models for VPR is still a key issue to address. To this end, we propose a novel method to realize seamless adaptation of pre-trained models for VPR. Specifically, to obtain both global and local features that focus on salient landmarks for discriminating places, we design a hybrid adaptation method to achieve both global and local adaptation efficiently, in which only lightweight adapters are tuned without adjusting the pre-trained model. Besides, to guide effective adaptation, we propose a mutual nearest neighbor local feature loss, which ensures proper dense local features are produced for local matching and avoids time-consuming spatial verification in re-ranking. Experimental results show that our method outperforms the state-of-the-art methods with less training data and training time, and uses about only 3% retrieval runtime of the two-stage VPR methods with RANSAC-based spatial verification. It ranks 1st on the MSLS challenge leaderboard (at the time of submission). The code is released at https://github.com/Lu-Feng/SelaVPR.
Can Large Language Models Detect Misinformation in Scientific News Reporting?
Authors: Yupeng Cao, Aishwarya Muralidharan Nair, Elyon Eyimife, Nastaran Jamalipour Soofi, K. P. Subbalakshmi, John R. Wullert II, Chumki Basu, David Shallcross
Date: 2024-02-22 00:00:00
Tasks: Articles, Claim Verification, Misinformation, Prompt Engineering

Scientific facts are often spun in the popular press with the intent to influence public opinion and action, as was evidenced during the COVID-19 pandemic. Automatic detection of misinformation in the scientific domain is challenging because of the distinct styles of writing in these two media types and is still in its nascence. Most research on the validity of scientific reporting treats this problem as a claim verification challenge. In doing so, significant expert human effort is required to generate appropriate claims. Our solution bypasses this step and addresses a more real-world scenario where such explicit, labeled claims may not be available. The central research question of this paper is whether it is possible to use large language models (LLMs) to detect misinformation in scientific reporting. To this end, we first present a new labeled dataset SciNews, containing 2.4k scientific news stories drawn from trusted and untrustworthy sources, paired with related abstracts from the CORD-19 database. Our dataset includes both human-written and LLM-generated news articles, making it more comprehensive in terms of capturing the growing trend of using LLMs to generate popular press articles. Then, we identify dimensions of scientific validity in science news articles and explore how this can be integrated into the automated detection of scientific misinformation. We propose several baseline architectures using LLMs to automatically detect false representations of scientific findings in the popular press. For each of these architectures, we use several prompt engineering strategies including zero-shot, few-shot, and chain-of-thought prompting. We also test these architectures and prompting strategies on GPT-3.5, GPT-4, and Llama2-7B, Llama2-13B.
EyeTrans: Merging Human and Machine Attention for Neural Code Summarization
Authors: Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, Yu Huang
Date: 2024-02-21 00:00:00
Tasks: Code Summarization

Neural code summarization leverages deep learning models to automatically generate brief natural language summaries of code snippets. The development of Transformer models has led to extensive use of attention during model design. While existing work has primarily and almost exclusively focused on static properties of source code and related structural representations like the Abstract Syntax Tree (AST), few studies have considered human attention, that is, where programmers focus while examining and comprehending code. In this paper, we develop a method for incorporating human attention into machine attention to enhance neural code summarization. To facilitate this incorporation and vindicate this hypothesis, we introduce EyeTrans, which consists of three steps: (1) we conduct an extensive eye-tracking human study to collect and pre-analyze data for model training, (2) we devise a data-centric approach to integrate human attention with machine attention in the Transformer architecture, and (3) we conduct comprehensive experiments on two code summarization tasks to demonstrate the effectiveness of incorporating human attention into Transformers. Integrating human attention leads to an improvement of up to 29.91% in Functional Summarization and up to 6.39% in General Code Summarization performance, demonstrating the substantial benefits of this combination. We further explore performance in terms of robustness and efficiency by creating challenging summarization scenarios in which EyeTrans exhibits interesting properties. We also visualize the attention map to depict the simplifying effect of machine attention in the Transformer by incorporating human attention. This work has the potential to propel AI research in software engineering by introducing more human-centered approaches and data.
CriticBench: Benchmarking LLMs for Critique-Correct Reasoning
Authors: Zicheng Lin, Zhibin Gou, Tian Liang, Ruilin Luo, Haowei Liu, Yujiu Yang
Date: 2024-02-22 00:00:00
Tasks: Benchmarking

The ability of Large Language Models (LLMs) to critique and refine their reasoning is crucial for their application in evaluation, feedback provision, and self-improvement. This paper introduces CriticBench, a comprehensive benchmark designed to assess LLMs' abilities to critique and rectify their reasoning across a variety of tasks. CriticBench encompasses five reasoning domains: mathematical, commonsense, symbolic, coding, and algorithmic. It compiles 15 datasets and incorporates responses from three LLM families. Utilizing CriticBench, we evaluate and dissect the performance of 17 LLMs in generation, critique, and correction reasoning, i.e., GQC reasoning. Our findings reveal: (1) a linear relationship in GQC capabilities, with critique-focused training markedly enhancing performance; (2) a task-dependent variation in correction effectiveness, with logic-oriented tasks being more amenable to correction; (3) GQC knowledge inconsistencies that decrease as model size increases; and (4) an intriguing inter-model critiquing dynamic, where stronger models are better at critiquing weaker ones, while weaker models can surprisingly surpass stronger ones in their self-critique. We hope these insights into the nuanced critique-correct reasoning of LLMs will foster further research in LLM critique and self-improvement.