pwc-archive/papers-with-abstracts huggingface dataset:
Is artificial consciousness achievable? Lessons from the human brain
Authors: Michele Farisco, Kathinka Evers, Jean-Pierre Changeux
Date: 2024-04-18 00:00:00

We here analyse the question of developing artificial consciousness from an evolutionary perspective, taking the evolution of the human brain and its relation with consciousness as a reference model. This kind of analysis reveals several structural and functional features of the human brain that appear to be key for reaching human-like complex conscious experience and that current research on Artificial Intelligence (AI) should take into account in its attempt to develop systems capable of conscious processing. We argue that, even if AI is limited in its ability to emulate human consciousness for both intrinsic (structural and architectural) and extrinsic (related to the current stage of scientific and technological knowledge) reasons, taking inspiration from those characteristics of the brain that make conscious processing possible and/or modulate it, is a potentially promising strategy towards developing conscious AI. Also, it is theoretically possible that AI research can develop partial or potentially alternative forms of consciousness that is qualitatively different from the human, and that may be either more or less sophisticated depending on the perspectives. Therefore, we recommend neuroscience-inspired caution in talking about artificial consciousness: since the use of the same word consciousness for humans and AI becomes ambiguous and potentially misleading, we propose to clearly specify what is common and what differs in AI conscious processing from full human conscious experience.
A design specification for Critical Illness Digital Twins to cure sepsis: responding to the National Academies of Sciences, Engineering and Medicine Report: Foundational Research Gaps and Future Directions for Digital Twins
Authors: Gary An, Chase Cockrell
Date: 2024-05-08 00:00:00
Tasks: Uncertainty Quantification

On December 15, 2023, The National Academies of Sciences, Engineering and Medicine (NASEM) released a report entitled: Foundational Research Gaps and Future Directions for Digital Twins. The ostensible purpose of this report was to bring some structure to the burgeoning field of digital twins by providing a working definition and a series of research challenges that need to be addressed to allow this technology to fulfill its full potential. In the work presented herein we focus on five specific findings from the NASEM Report: 1) definition of a Digital Twin, 2) using fit-for-purpose guidance, 3) developing novel approaches to Verification, Validation and Uncertainty Quantification (VVUQ) of Digital Twins, 4) incorporating control as an explicit purpose for a Digital Twin and 5) using a Digital Twin to guide data collection and sensor development, and describe how these findings are addressed through the design specifications for a Critical Illness Digital Twin (CIDT) aimed at curing sepsis.
Quantifying indirect and direct vaccination effects arising in the SIR model
Authors: Lixin Lin, Homayoun Hamedmoghadam, Robert Shorten, Lewi Stone
Date: 2024-05-03 00:00:00

Vaccination campaigns have both direct and indirect effects that act to control an infectious disease as it spreads through a population. Indirect effects arise when vaccinated individuals block disease transmission in any infection chains they are part of, and this in turn can benefit both vaccinated and unvaccinated individuals. Indirect effects are difficult to quantify in practice, but here, working with the Susceptible-Infected-Recovered (SIR) model, they are analytically calculated in important cases, through pivoting on the Final Size formula for epidemics. Their relationship to herd immunity is also clarified. Furthermore, we identify the important distinction between quantifying indirect effects of vaccination at the "population level" versus the "per capita" individual level, which often results in radically different conclusions. As an important example, the analysis unpacks why population-level indirect effect can appear significantly larger than its per capita analogue. In addition, we consider a recently proposed epidemiological non-pharamaceutical intervention used over COVID-19, referred to as "shielding", and study its impact in our mathematical analysis. The shielding scheme is extended by inclusion of limited vaccination.
Inflation Models with Correlation and Skew
Authors: Orcan Ogetbil, Bernhard Hientzsch
Date: 2024-05-08 00:00:00

We formulate a forward inflation index model with multi-factor volatility structure featuring a parametric form that allows calibration to correlations between indices of different tenors observed in the market. Assuming the nominal interest rate follows a single factor Gaussian short rate model, we present analytical prices for zero-coupon and year-on-year swaps, caps, and floors. The same method applies to any interest rate model for which one can compute the zero-coupon bond prices and measure shifts. We extend the multi-factor model with leverage functions to capture the entire market volatility skew with a single process. The time-consuming calibration step of this model can be avoided in the simplified model that we further propose. We demonstrate the leveraged and the simplified models with market data.
Complex network analysis of cryptocurrency market during crashes
Authors: Kundan Mukhia, Anish rai, SR Luwang, Md Nurujjaman, Sushovan Majhi, Chittaranjan Hens
Date: 2024-05-09 00:00:00

This paper identifies the cryptocurrency market crashes and analyses its dynamics using the complex network. We identify three distinct crashes during 2017-20, and the analysis is carried out by dividing the time series into pre-crash, crash, and post-crash periods. Partial correlation based complex network analysis is carried out to study the crashes. Degree density ($\rho_D$), average path length ($\bar{l}$), and average clustering coefficient ($\overline{cc}$) are estimated from these networks. We find that both $\rho_D$ and $\overline{cc}$ are smallest during the pre-crash period, and spike during the crash suggesting the network is dense during a crash. Although $\rho_D$ and $\overline{cc}$ decrease in the post-crash period, they remain higher than pre-crash levels for the 2017-18 and 2018-19 crashes suggesting a market attempt to return to normalcy. We get $\bar{l}$ is minimal during the crash period, suggesting a rapid flow of information. A dense network and rapid information flow suggest that during a crash uninformed synchronized panic sell-off happens. However, during the 2019-20 crash, the values of $\rho_D$, $\overline{cc}$, and $\bar{l}$ did not vary significantly, indicating minimal change in dynamics compared to other crashes. The findings of this study may guide investors in making decisions during market crashes.
High-Frequency Stock Market Order Transitions during the US-China Trade War 2018: A Discrete-Time Markov Chain Analysis
Authors: Salam Rabindrajit Luwang, Anish rai, Md. Nurujjaman, Om Prakash, Chittaranjan Hens
Date: 2024-05-09 00:00:00

Statistical analysis of high-frequency stock market order transaction data is conducted to understand order transition dynamics. We employ a first-order time-homogeneous discrete-time Markov chain model to the sequence of orders of stocks belonging to six different sectors during the USA-China trade war of 2018. The Markov property of the order sequence is validated by the Chi-square test. We estimate the transition probability matrix of the sequence using maximum likelihood estimation. From the heat-map of these matrices, we found the presence of active participation by different types of traders during high volatility days. On such days, these traders place limit orders primarily with the intention of deleting the majority of them to influence the market. These findings are supported by high stationary distribution and low mean recurrence values of add and delete orders. Further, we found similar spectral gap and entropy rate values, which indicates that similar trading strategies are employed on both high and low volatility days during the trade war. Among all the sectors considered in this study, we observe that there is a recurring pattern of full execution orders in Finance & Banking sector. This shows that the banking stocks are resilient during the trade war. Hence, this study may be useful in understanding stock market order dynamics and devise trading strategies accordingly on high and low volatility days during extreme macroeconomic events.
Tackling water table depth modeling via machine learning: From proxy observations to verifiability
Authors: Joseph Janssen, Ardalan Tootchi, Ali A. Ameli
Date: 2024-04-30 00:00:00
Tasks: Depth Estimation

Spatial patterns of water table depth (WTD) play a crucial role in shaping ecological resilience, hydrological connectivity, and human-centric systems. Generally, a large-scale (e.g., continental or global) continuous map of static WTD can be simulated using either physically-based (PB) or machine learning-based (ML) models. We construct three fine-resolution (500 m) ML simulations of WTD, using the XGBoost algorithm and more than 20 million real and proxy observations of WTD, across the United States and Canada. The three ML models were constrained using known physical relations between WTD's drivers and WTD and were trained by sequentially adding real and proxy observations of WTD. Through an extensive (pixel-by-pixel) evaluation across the study region and within ten major ecoregions of North America, we demonstrate that our models (corr=0.6-0.75) can more accurately predict unseen real and proxy observations of WTD compared to two available PB simulations of WTD (corr=0.21-0.40). However, we still argue that currently-available large-scale simulations of static WTD could be uncertain within data-scarce regions such as steep mountainous regions. We reason that biased observational data mainly collected from low-elevation floodplains and the over-flexibility of available models can negatively affect the verifiability of large-scale simulations of WTD. Ultimately, we thoroughly discuss future directions that may help hydrogeologists decide how to improve machine learning-based WTD estimations. In particular, we advocate for the use of proxy satellite data, the incorporation of physical laws, the implementation of better model verification standards, the development of novel globally-available emergent indices, and the collection of more reliable observations.
Financial Table Extraction in Image Documents
Authors: William Watson, Bo Liu
Date: 2024-03-18 00:00:00
Tasks: Image Segmentation, Optical Character Recognition (OCR), Semantic Segmentation, Table Extraction

Table extraction has long been a pervasive problem in financial services. This is more challenging in the image domain, where content is locked behind cumbersome pixel format. Luckily, advances in deep learning for image segmentation, OCR, and sequence modeling provides the necessary heavy lifting to achieve impressive results. This paper presents an end-to-end pipeline for identifying, extracting and transcribing tabular content in image documents, while retaining the original spatial relations with high fidelity.
3D Holistic OR Anonymization
Authors: Tony Danjun Wang
Date: 2024-03-18 00:00:00

We propose a novel method that leverages 3D information to automatically anonymize multi-view RGB-D video recordings of operating rooms (OR). Our anonymization method preserves the original data distribution by replacing the faces in each image with different faces so that the data remains suitable for further downstream tasks. In contrast to established anonymization methods, our approach localizes faces in 3D space first rather than in 2D space. Each face is then anonymized by reprojecting a different face back into each camera view, ultimately replacing the original faces in the resulting images. Furthermore, we introduce a multi-view RGB-D dataset, captured during a real operation of experienced surgeons performing laparoscopic surgery on an animal object (swine), which encapsulates typical characteristics of ORs. Finally, we present experimental results evaluated on that dataset, showing that leveraging 3D data can achieve better face localization in OR images and generate more realistic faces than the current state-of-the-art. There has been, to our knowledge, no prior work that addresses the anonymization of multi-view OR recordings, nor 2D face localization that leverages 3D information.
An Attention-Based Deep Generative Model for Anomaly Detection in Industrial Control Systems
Authors: Mayra Macas, Chunming Wu, Walter Fuertes
Date: 2024-05-03 00:00:00
Tasks: Anomaly Detection, Decoder

Anomaly detection is critical for the secure and reliable operation of industrial control systems. As our reliance on such complex cyber-physical systems grows, it becomes paramount to have automated methods for detecting anomalies, preventing attacks, and responding intelligently. {This paper presents a novel deep generative model to meet this need. The proposed model follows a variational autoencoder architecture with a convolutional encoder and decoder to extract features from both spatial and temporal dimensions. Additionally, we incorporate an attention mechanism that directs focus towards specific regions, enhancing the representation of relevant features and improving anomaly detection accuracy. We also employ a dynamic threshold approach leveraging the reconstruction probability and make our source code publicly available to promote reproducibility and facilitate further research. Comprehensive experimental analysis is conducted on data from all six stages of the Secure Water Treatment (SWaT) testbed, and the experimental results demonstrate the superior performance of our approach compared to several state-of-the-art baseline techniques.
Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R
Authors: Amirreza Esmaeili, Iman Saberi, Fatemeh H. Fard
Date: 2024-03-16 00:00:00
Tasks: Code Summarization, parameter-efficient fine-tuning, Transfer Learning

Parameter Efficient Fine-Tuning (PEFT) methods are proposed as an alternative fine-tuning approach for Large Language Models (LLM) to minimize high training costs. While prior research demonstrates the effectiveness of PEFT methods in knowledge transfer using smaller language models, their application to larger LLMs, particularly in low-resource and unseen programming languages such as R, remains under-explored. In this work, we evaluate PEFT methods, LoRA, Compacter, and IA^3 on LLMs for code summarization and generation, with a particular emphasis on knowledge transfer to R as an unseen under-explored target language. Our experiments reveal that LoRA consistently outperforms Compacter and IA^3 in all settings, while Compacter offers significant resource efficiency with minimal performance trade-offs. Additionally, we find that the number of trainable parameters has a greater influence on the functional accuracy of the generated code than PEFT architecture. Our study can direct future research in developing code intelligent tasks for unseen languages including R, as well as the choice of PEFT methods for knowledge transfer, especially when balancing the computational cost and performance.
Unsupervised multimodal modeling of cognitive and brain health trajectories for early dementia prediction
Authors: Michael C. Burkhart, Liz Y. Lee, Delshad Vaghari, An Qi Toh, Eddie Chong, Christopher Chen, Peter Tiňo, Zoe Kourtzi
Date: 2024-05-10 00:00:00
Proceeding: Scientific Reports 2024 5
Tasks: State Space Models, Trajectory Modeling

Predicting the course of neurodegenerative disorders early has potential to greatly improve clinical management and patient outcomes. A key challenge for early prediction in real-world clinical settings is the lack of labeled data (i.e., clinical diagnosis). In contrast to supervised classification approaches that require labeled data, we propose an unsupervised multimodal trajectory modeling (MTM) approach based on a mixture of state space models that captures changes in longitudinal data (i.e., trajectories) and stratifies individuals without using clinical diagnosis for model training. MTM learns the relationship between states comprising expensive, invasive biomarkers (β-amyloid, grey matter density) and readily obtainable cognitive observations. MTM training on trajectories stratifies individuals into clinically meaningful clusters more reliably than MTM training on baseline data alone and is robust to missing data (i.e., cognitive data alone or single assessments). Extracting an individualized cognitive health index (i.e., MTM-derived cluster membership index) allows us to predict progression to AD more precisely than standard clinical assessments (i.e., cognitive tests or MRI scans alone). Importantly, MTM generalizes successfully from research cohort to real-world clinical data from memory clinic patients with missing data, enhancing the clinical utility of our approach. Thus, our multimodal trajectory modeling approach provides a cost-effective and non- invasive tool for early dementia prediction without labeled data (i.e., clinical diagnosis) with strong potential for translation to clinical practice.
HMT: Hierarchical Memory Transformer for Long Context Language Processing
Authors: Zifan He, Zongyue Qin, Neha Prakriya, Yizhou Sun, Jason Cong
Date: 2024-05-09 00:00:00
Tasks: Language Modeling, Language Modelling, Memorization, Question Answering

Transformer-based large language models (LLM) have been widely used in language processing applications. However, most of them restrict the context window that permits the model to attend to every token in the inputs. Previous works in recurrent models can memorize past tokens to enable unlimited context and maintain effectiveness. However, they have "flat" memory architectures, which have limitations in selecting and filtering information. Since humans are good at learning and self-adjustment, we speculate that imitating brain memory hierarchy is beneficial for model memorization. We propose the Hierarchical Memory Transformer (HMT), a novel framework that enables and improves models' long-context processing ability by imitating human memorization behavior. Leveraging memory-augmented segment-level recurrence, we organize the memory hierarchy by preserving tokens from early input token segments, passing memory embeddings along the sequence, and recalling relevant information from history. Evaluating general language modeling (Wikitext-103, PG-19) and question-answering tasks (PubMedQA), we show that HMT steadily improves the long-context processing ability of context-constrained and long-context models. With an additional 0.5% - 2% of parameters, HMT can easily plug in and augment future LLMs to handle long context effectively. Our code is open-sourced on Github: https://github.com/OswaldHe/HMT-pytorch.
State-Free Inference of State-Space Models: The Transfer Function Approach
Authors: Rom N. Parnichkun, Stefano Massaroli, Alessandro Moro, Jimmy T. H. Smith, Ramin Hasani, Mathias Lechner, Qi An, Christopher Ré, Hajime Asama, Stefano Ermon, Taiji Suzuki, Atsushi Yamashita, Michael Poli
Date: 2024-05-10 00:00:00
Tasks: Language Modeling, Language Modelling, State Space Models

We approach designing a state-space model for deep learning applications through its dual representation, the transfer function, and uncover a highly efficient sequence parallel inference algorithm that is state-free: unlike other proposed algorithms, state-free inference does not incur any significant memory or computational cost with an increase in state size. We achieve this using properties of the proposed frequency domain transfer function parametrization, which enables direct computation of its corresponding convolutional kernel's spectrum via a single Fast Fourier Transform. Our experimental results across multiple sequence lengths and state sizes illustrates, on average, a 35% training speed improvement over S4 layers -- parametrized in time-domain -- on the Long Range Arena benchmark, while delivering state-of-the-art downstream performances over other attention-free approaches. Moreover, we report improved perplexity in language modeling over a long convolutional Hyena baseline, by simply introducing our transfer function parametrization. Our code is available at https://github.com/ruke1ire/RTF.
ODC-SA Net: Orthogonal Direction Enhancement and Scale Aware Network for Polyp Segmentation
Authors: Chenhao Xu, Yudian Zhang, Kaiye Xu, Haijiang Zhu
Date: 2024-05-10 00:00:00
Tasks: Segmentation

Accurate polyp segmentation is crucial for the early detection and prevention of colorectal cancer. However, the existing polyp detection methods sometimes ignore multi-directional features and drastic changes in scale. To address these challenges, we design an Orthogonal Direction Enhancement and Scale Aware Network (ODC-SA Net) for polyp segmentation. The Orthogonal Direction Convolutional (ODC) block can extract multi-directional features using transposed rectangular convolution kernels through forming an orthogonal feature vector basis, which solves the issue of random feature direction changes and reduces computational load. Additionally, the Multi-scale Fusion Attention (MSFA) mechanism is proposed to emphasize scale changes in both spatial and channel dimensions, enhancing the segmentation accuracy for polyps of varying sizes. Extraction with Re-attention Module (ERA) is used to re-combinane effective features, and Structures of Shallow Reverse Attention Mechanism (SRA) is used to enhance polyp edge with low level information. A large number of experiments conducted on public datasets have demonstrated that the performance of this model is superior to state-of-the-art methods.
Time Evidence Fusion Network: Multi-source View in Long-Term Time Series Forecasting
Authors: Tianxiang Zhan, Yuanpeng He, Yong Deng, Zhen Li, Wenjie Du, Qingsong Wen
Date: 2024-05-10 00:00:00
Tasks: Time Series, Time Series Forecasting

In practical scenarios, time series forecasting necessitates not only accuracy but also efficiency. Consequently, the exploration of model architectures remains a perennially trending topic in research. To address these challenges, we propose a novel backbone architecture named Time Evidence Fusion Network (TEFN) from the perspective of information fusion. Specifically, we introduce the Basic Probability Assignment (BPA) Module based on evidence theory to capture the uncertainty of multivariate time series data from both channel and time dimensions. Additionally, we develop a novel multi-source information fusion method to effectively integrate the two distinct dimensions from BPA output, leading to improved forecasting accuracy. Lastly, we conduct extensive experiments to demonstrate that TEFN achieves performance comparable to state-of-the-art methods while maintaining significantly lower complexity and reduced training time. Also, our experiments show that TEFN exhibits high robustness, with minimal error fluctuations during hyperparameter selection. Furthermore, due to the fact that BPA is derived from fuzzy theory, TEFN offers a high degree of interpretability. Therefore, the proposed TEFN balances accuracy, efficiency, stability, and interpretability, making it a desirable solution for time series forecasting.