pwc-archive/papers-with-abstracts huggingface dataset:
Towards deep learning sequence-structure co-generation for protein design
Authors: Chentong Wang, Sarah Alamdari, Carles Domingo-Enrich, Ava Amini, Kevin K. Yang
Date: 2024-10-02 00:00:00
Tasks: Deep Learning, Protein Design

Deep generative models that learn from the distribution of natural protein sequences and structures may enable the design of new proteins with valuable functions. While the majority of today's models focus on generating either sequences or structures, emerging co-generation methods promise more accurate and controllable protein design, ideally achieved by modeling both modalities simultaneously. Here we review recent advances in deep generative models for protein design, with a particular focus on sequence-structure co-generation methods. We describe the key methodological and evaluation principles underlying these methods, highlight recent advances from the literature, and discuss opportunities for continued development of sequence-structure co-generation approaches.
Inflation in Disaggregated Small Open Economies
Authors: Alvaro Silva
Date: 2024-10-01 00:00:00

This paper studies inflation in small open economies with production networks. I show that the production network alters the elasticity of the consumer price index (CPI) to changes in sectoral technology, factor prices, and import prices. Sectors can import and export directly but also indirectly through domestic intermediate inputs. Indirect exporting dampens the inflationary pressure from domestic forces, while indirect importing increases the inflation sensitivity to import price changes. Computing these CPI elasticities requires knowledge of the production network structure as these do not coincide with typical sufficient statistics used in the literature, such as sectoral sales-to-GDP ratios, factor shares, or imported consumption shares. Using input-output tables, I provide empirical evidence that adjusting CPI elasticities for indirect exports and imports matters quantitatively for small open economies. I use the model to illustrate the importance of production networks during the recent COVID-19 inflation in Chile and the United Kingdom.
A Computational Fluid Dynamics study of drug-releasing ocular implants for glaucoma treatment: Comparison of implant size and locations
Authors: Daniel Sebastia-Saez, Tao Chen, Cynthia Yu-Wai-Man
Date: 2024-10-01 00:00:00

Drug-releasing implants are gaining momentum in the treatment of glaucoma. Implants present however several limitations. Among these limitations, there is the inability to provide an even distribution of the drug in the trabecular meshwork. CFD simulations were used in this work to study the interplay between ocular fluid dynamics and drug diffusion to explore the options to achieve a homogeneous drug distribution. The analysis finds that the balance between convection and diffusion flux hinders mixing within the eye's anterior chamber. This results in highly localized drug delivery in the trabecular meshwork when using a gravity-driven location implant. The results also show that varying the size of the implant can help to solve the issue. Given the natural variability of the trabecular meshwork size amongst patients, implants with personalized size may become a potential solution. The location of the implant within the eye is also key to effective drug delivery. Natural laminar flow of aqueous humor within the eye's anterior chamber prevents achieving an even drug distribution at the target tissue for gravity-driven location implants. However, the iris-lens gap can be utilized as a natural mixer when placing the implant in the eye's posterior chamber, thus helping with effective delivery.
Handover_Management_in_UAV_Networks_with_Blockages
Authors: Neetu R R, Gourab Ghatak, Vivek Ashok Bohara
Date: 2024-09-30 00:00:00

We investigate the performance of unmanned aerial vehicle (UAV)-based networks in urban environments characterized by blockages, focusing on their capability to support the service demands of mobile users. The UAV-base stations (UAV-BSs) are modeled using a two-dimensional (2-D) marked- Poisson point process (MPPP), where the marks represent the altitude of each UAV-BS. Leveraging stochastic geometry, we analyze the impact of blockages on network reliability by studying the meta distribution (MD) of the signal-to-interference noise ratio (SINR) for a specific reliability threshold and the association probabilities for both line-of-sight (LoS) and non line-of-sight (NLoS) UAV-BSs. Furthermore, to enhance the performance of mobile users, we propose a novel cache-based handover management strategy that dynamically selects the cell search time and delays the received signal strength (RSS)-based base station (BS) associations. This strategy aims to minimize unnecessary handovers (HOs) experienced by users by leveraging caching capabilities at user equipment (UE), thus reducing latency, ensuring seamless connectivity, and maintaining the quality of service (QoS). This study provides valuable insights into optimizing UAV network deployments to support the stringent requirements in the network, ensuring reliable, low-latency, and high-throughput communication for next-generation smart cities.
Semi-Blind Receivers for Hybrid Reflecting and Sensing RIS
Authors: Amarilton L. Magalhães, André L. F. de Almeida
Date: 2024-09-29 00:00:00

Recent research has delved into advanced designs for reconfigurable intelligent surfaces (RIS) with integrated sensing functions. One promising concept is the hybrid RIS (HRIS), which blends sensing and reflecting meta-atoms. This enables HRIS to process signals, aiding in channel estimation (CE) and symbol detection tasks. This paper formulates semi-blind receivers for HRIS-aided wireless communications that enable joint symbol and CE at the HRIS and BS. The proposed receivers rely on a new tensor modeling approach for the signals received at both the HRIS and BS while exploiting a tensor signal coding scheme at the transmit side. Specifically, by capitalizing on the multilinear structures of the received signals, we develop iterative and closed-form receiver algorithms for joint estimation of the uplink channels and symbols at both the HRIS and the BS. Enabling joint channel and symbol estimation functionalities, the proposed receivers offer symbol decoding capabilities to the HRIS and ensure ambiguity-free separate CE without requiring an a priori training stage. We also study identifiability conditions ensuring a unique joint channel and symbol recovery and discuss the computational complexities and tradeoffs involved by the proposed semi-blind receivers. Our findings demonstrate the competitive performances of the proposed algorithms at the HRIS and the BS and uncover distinct performance trends based on the possible combinations of HRIS-BS receiver pairs. Finally, extensive numerical results elucidate the interplay between power splitting, symbol recovery, and CE accuracy in HRIS-assisted communications. Such insights are pivotal for optimizing receiver design and enhancing system performance in future HRIS deployments.
Generalizing Consistency Policy to Visual RL with Prioritized Proximal Experience Regularization
Authors: Haoran Li, Zhennan Jiang, Yuhui Chen, Dongbin Zhao
Date: 2024-09-28 00:00:00
Tasks: Reinforcement Learning (RL)

With high-dimensional state spaces, visual reinforcement learning (RL) faces significant challenges in exploitation and exploration, resulting in low sample efficiency and training stability. As a time-efficient diffusion model, although consistency models have been validated in online state-based RL, it is still an open question whether it can be extended to visual RL. In this paper, we investigate the impact of non-stationary distribution and the actor-critic framework on consistency policy in online RL, and find that consistency policy was unstable during the training, especially in visual RL with the high-dimensional state space. To this end, we suggest sample-based entropy regularization to stabilize the policy training, and propose a consistency policy with prioritized proximal experience regularization (CP3ER) to improve sample efficiency. CP3ER achieves new state-of-the-art (SOTA) performance in 21 tasks across DeepMind control suite and Meta-world. To our knowledge, CP3ER is the first method to apply diffusion/consistency models to visual RL and demonstrates the potential of consistency models in visual RL. More visualization results are available at https://jzndd.github.io/CP3ER-Page/.
Direct Multipath-Based SLAM
Authors: Mingchao Liang, Erik Leitinger, Florian Meyer
Date: 2024-09-30 00:00:00
Tasks: Position, Simultaneous Localization and Mapping

In future wireless networks, the availability of information on the position of mobile agents and the propagation environment can enable new services and increase the throughput and robustness of communications. Multipath-based simultaneous localization and mapping (SLAM) aims at estimating the position of agents and reflecting features in the environment by exploiting the relationship between the local geometry and multipath components (MPCs) in received radio signals. Existing multipath-based SLAM methods preprocess received radio signals using a channel estimator. The channel estimator lowers the data rate by extracting a set of dispersion parameters for each MPC. These parameters are then used as measurements for SLAM. Bayesian estimation for multipath-based SLAM is facilitated by the lower data rate. However, due to finite resolution capabilities limited by signal bandwidth, channel estimation is prone to errors and MPC parameters may be extracted incorrectly and lead to a reduced SLAM performance. We propose a multipath-based SLAM approach that directly uses received radio signals as inputs. A new statistical model that can effectively be represented by a factor graph is introduced. The factor graph is the starting point for the development of an efficient belief propagation (BP) method for multipath-based SLAM that avoids data preprocessing by a channel estimator. Numerical results based on synthetic and real data in challenging single-input, single-output (SISO) scenarios demonstrate that the proposed method outperforms conventional methods in terms of localization and mapping accuracy.
An Efficient Inference Frame for SMLM (Single-Molecule Localization Microscopy)
Authors: Tingdan Luo
Date: 2024-10-03 00:00:00
Tasks: Deep Learning, GPU, Image Reconstruction

Single-molecule localization microscopy (SMLM) surpasses the diffraction limit, achieving subcellular resolution. Traditional SMLM analysis methods often rely on point spread function (PSF) model fitting, limiting the application of complex PSF models. In recent years, deep learning approaches have significantly improved SMLM algorithms, yielding promising results. However, limitations in inference speed and model size have restricted the widespread adoption of deep learning in practical applications. To address these challenges, this paper proposes an efficient model deployment framework and introduces a lightweight neural network, DilatedLoc, aimed at enhancing both image reconstruction quality and inference speed. Compared to leading network models, DilatedLoc reduces network parameters to under 100 MB and achieves a 50% improvement in inference speed, with superior GPU utilization through a novel deployment architecture compatible with various network models.
CodeJudge: Evaluating Code Generation with Large Language Models
Authors: Weixi Tong, Tianyi Zhang
Date: 2024-10-03 00:00:00
Tasks: Code Generation

Large Language Models (LLMs) have shown promising performance in code generation. However, how to reliably evaluate code generated by LLMs remains an unresolved problem. This paper presents CodeJudge, a code evaluation framework that leverages LLMs to evaluate the semantic correctness of generated code without the need for test cases. We investigate different ways to guide the LLM in performing "slow thinking" to arrive at an in-depth and reliable evaluation. We experimented with four LLMs as evaluators on four code generation datasets and five programming languages. The results show that CodeJudge significantly outperformed existing methods in most settings. Furthermore, compared with a SOTA GPT-3.5-based code evaluation method, CodeJudge achieved better results even when using a much smaller model, Llama-3-8B-Instruct. Our code and datasets are available on GitHub https://github.com/VichyTong/CodeJudge.
Winning Solution For Meta KDD Cup' 24
Authors: Yikuan Xia, Jiazun Chen, Jun Gao
Date: 2024-09-13 00:00:00
Tasks: Hallucination, Knowledge Graphs, Language Modeling, Language Modelling, Large Language Model, RAG, Retrieval, Task 2

This paper describes the winning solutions of all tasks in Meta KDD Cup 24 from db3 team. The challenge is to build a RAG system from web sources and knowledge graphs. We are given multiple sources for each query to help us answer the question. The CRAG challenge involves three tasks: (1) condensing information from web pages into accurate answers, (2) integrating structured data from mock knowledge graphs, and (3) selecting and integrating critical data from extensive web pages and APIs to reflect real-world retrieval challenges. Our solution for Task #1 is a framework of web or open-data retrieval and answering. The large language model (LLM) is tuned for better RAG performance and less hallucination. Task #2 and Task #3 solutions are based on a regularized API set for domain questions and the API generation method using tuned LLM. Our knowledge graph API interface extracts directly relevant information to help LLMs answer correctly. Our solution achieves 1st place on all three tasks, achieving a score of 28.4%, 42.7%, and 47.8%, respectively.
Encryption-Friendly LLM Architecture
Authors: Donghwan Rho, Taeseong Kim, Minje Park, Jung Woo Kim, Hyunsik Chae, Ernest K. Ryu, Jung Hee Cheon
Date: 2024-10-03 00:00:00
Tasks: Privacy Preserving

Large language models (LLMs) offer personalized responses based on user interactions, but this use case raises serious privacy concerns. Homomorphic encryption (HE) is a cryptographic protocol supporting arithmetic computations in encrypted states and provides a potential solution for privacy-preserving machine learning (PPML). However, the computational intensity of transformers poses challenges for applying HE to LLMs. In this work, we propose a modified HE-friendly transformer architecture with an emphasis on inference following personalized (private) fine-tuning. Utilizing LoRA fine-tuning and Gaussian kernels, we achieve significant computational speedups -- 6.94x for fine-tuning and 2.3x for inference -- while maintaining performance comparable to plaintext models. Our findings provide a viable proof of concept for offering privacy-preserving LLM services in areas where data protection is crucial. Our code is available on GitHub.
Deep Learning-Based Prediction of Suspension Dynamics Performance in Multi-Axle Vehicles
Authors: Kai Chun Lin, Bo-Yi Lin
Date: 2024-10-03 00:00:00

This paper presents a deep learning-based framework for predicting the dynamic performance of suspension systems in multi-axle vehicles, emphasizing the integration of machine learning with traditional vehicle dynamics modeling. A Multi-Task Deep Belief Network Deep Neural Network (MTL-DBN-DNN) was developed to capture the relationships between key vehicle parameters and suspension performance metrics. The model was trained on data generated from numerical simulations and demonstrated superior prediction accuracy compared to conventional DNN models. A comprehensive sensitivity analysis was conducted to assess the impact of various vehicle and suspension parameters on dynamic suspension performance. Additionally, the Suspension Dynamic Performance Index (SDPI) was introduced as a holistic measure to quantify overall suspension performance, accounting for the combined effects of multiple parameters. The findings highlight the effectiveness of multitask learning in improving predictive models for complex vehicle systems.
Diffusion Models are Evolutionary Algorithms
Authors: Yanbo Zhang, Benedikt Hartl, Hananel Hazan, Michael Levin
Date: 2024-10-03 00:00:00
Tasks: Denoising, Evolutionary Algorithms

In a convergence of machine learning and biology, we reveal that diffusion models are evolutionary algorithms. By considering evolution as a denoising process and reversed evolution as diffusion, we mathematically demonstrate that diffusion models inherently perform evolutionary algorithms, naturally encompassing selection, mutation, and reproductive isolation. Building on this equivalence, we propose the Diffusion Evolution method: an evolutionary algorithm utilizing iterative denoising -- as originally introduced in the context of diffusion models -- to heuristically refine solutions in parameter spaces. Unlike traditional approaches, Diffusion Evolution efficiently identifies multiple optimal solutions and outperforms prominent mainstream evolutionary algorithms. Furthermore, leveraging advanced concepts from diffusion models, namely latent space diffusion and accelerated sampling, we introduce Latent Space Diffusion Evolution, which finds solutions for evolutionary tasks in high-dimensional complex parameter space while significantly reducing computational steps. This parallel between diffusion and evolution not only bridges two different fields but also opens new avenues for mutual enhancement, raising questions about open-ended evolution and potentially utilizing non-Gaussian or discrete diffusion models in the context of Diffusion Evolution.
Memory-Based Deep Neural Attention (mDNA) for Cognitive Multi-Turn Response Retrieval in Task-Oriented Chatbots
Authors: Jenhui Chen, Obinna Agbodike, Lei Wang
Date: 2020-08-22 00:00:00
Proceeding: MDPI Applied Sciences 2020 8
Tasks: Chatbot, Retrieval

One of the important criteria used in judging the performance of a chatbot is the ability to provide meaningful and informative responses that correspond with the context of a user’s utterance. Nowadays, the number of enterprises adopting and relying on task-oriented chatbots for profit is increasing. Dialog errors and inappropriate response to user queries by chatbots can result in huge cost implications. To achieve high performance, recent AI chatbot models are increasingly adopting the Transformer positional encoding and the attention-based architecture. While the transformer performs optimally in sequential generative chatbot models, recent studies has pointed out the occurrence of logical inconsistency and fuzzy error problems when the Transformer technique is adopted in retrieval-based chatbot models. Our investigation discovers that the encountered errors are caused by information losses. Therefore, in this paper, we address this problem by augmenting the Transformer-based retrieval chatbot architecture with a memory-based deep neural attention (mDNA) model by using an approach similar to late data fusion. The mDNA is a simple encoder-decoder neural architecture that comprises of bidirectional long short-term memory (Bi-LSTM), attention mechanism, and a memory for information retention in the encoder. In our experiments, we trained the model extensively on a large Ubuntu dialog corpus, and the results from recall evaluation scores show that the mDNA augmentation approach slightly outperforms selected state-of-the-art retrieval chatbot models. The results from the mDNA augmentation approach are quite impressive.
NestedMorph: Enhancing Deformable Medical Image Registration with Nested Attention Mechanisms
Authors: Gurucharan Marthi Krishna Kumar, Janine Mendola, Amir Shmuel
Date: 2024-10-03 00:00:00
Tasks: Decoder, Deformable Medical Image Registration, Diffusion  MRI, Image Registration, Medical Image Registration, SSIM

Deformable image registration is crucial for aligning medical images in a nonlinear fashion across different modalities, allowing for precise spatial correspondence between varying anatomical structures. This paper presents NestedMorph, a novel network utilizing a Nested Attention Fusion approach to improve intra-subject deformable registration between T1-weighted (T1w) MRI and diffusion MRI (dMRI) data. NestedMorph integrates high-resolution spatial details from an encoder with semantic information from a decoder using a multi-scale framework, enhancing both local and global feature extraction. Our model notably outperforms existing methods, including CNN-based approaches like VoxelMorph, MIDIR, and CycleMorph, as well as Transformer-based models such as TransMorph and ViT-V-Net, and traditional techniques like NiftyReg and SyN. Evaluations using the HCP dataset demonstrate that NestedMorph achieves superior performance across key metrics, including SSIM, HD95, and SDlogJ, with the highest SSIM of 0.89, the lowest HD95 of 2.5 and SDlogJ of 0.22. These results highlight NestedMorph's ability to capture both local and global image features effectively, leading to superior registration performance. The promising outcomes of this study underscore NestedMorph's potential to significantly advance deformable medical image registration, providing a robust framework for future research and clinical applications. The source code and our implementation are available at: https://github.com/AS-Lab/Marthi-et-al-2024-NestedMorph-Deformable-Medical-Image-Registration
PKET-GCN: Prior knowledge enhanced time-varying graph convolution network for traffic flow prediction
Authors: Yinxin Bao, Jiali Liu, Qinqin Shen, Yang Cao, Weiping Ding, Quan Shi
Date: 2023-04-01 00:00:00
Proceeding: 2023 2023 4

Due to prediction on the traffic flow is influenced by the real environment and historical data, the produced traffic graph may include significant uncertainty. The graph convolution operation is widely used in traffic flow prediction with its effective modeling ability on graph structures. However, in this method, it ignores the roles of external factors and historical data from fixed period is used that inevitably will lead to exclusion of detailed dynamic spatial–temporal correlation. To this end, we propose a novel method based on prior knowledge enhanced time-varying graph convolution network (PKET-GCN). First, we characterize factors affecting the traffic flow into dynamic and static features. The dynamic features include data correlation and external interference, while the static features consist of physical distances. Then we design a prior knowledge based module to extract the correlation of nodes and combine it with graph convolution to obtain dynamic spatial features. Next, a time-varying feature extraction module is designed to derive dynamic and long-term temporal features from periodic and adjacent sequences. Finally, the projection module is established to fuse the multiple modules and give the prediction value. The experimental results on five real-world datasets indicate that PKET-GCN is more effective than several existing methods.