pwc-archive/papers-with-abstracts huggingface dataset:
Understanding LLMs: A Comprehensive Overview from Training to Inference
Authors: Yiheng Liu, Hao He, Tianle Han, Xu Zhang, Mengyuan Liu, Jiaming Tian, Yutong Zhang, Jiaqi Wang, Xiaohui Gao, Tianyang Zhong, Yi Pan, Shaochen Xu, Zihao Wu, Zhengliang Liu, Xin Zhang, Shu Zhang, Xintao Hu, Tuo Zhang, Ning Qiang, Tianming Liu, Bao Ge
Date: 2024-01-04 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Model Compression, Scheduling

The introduction of ChatGPT has led to a significant increase in the utilization of Large Language Models (LLMs) for addressing downstream tasks. There's an increasing focus on cost-efficient training and deployment within this context. Low-cost training and deployment of LLMs represent the future development trend. This paper reviews the evolution of large language model training techniques and inference deployment technologies aligned with this emerging trend. The discussion on training includes various aspects, including data preprocessing, training architecture, pre-training tasks, parallel training, and relevant content related to model fine-tuning. On the inference side, the paper covers topics such as model compression, parallel computation, memory scheduling, and structural optimization. It also explores LLMs' utilization and provides insights into their future development.
Text2MDT: Extracting Medical Decision Trees from Medical Texts
Authors: Wei Zhu, Wenfeng Li, Xing Tian, Pengfei Wang, Xiaoling Wang, Jin Chen, Yuanbin Wu, Yuan Ni, Guotong Xie
Date: 2024-01-04 00:00:00

Knowledge of the medical decision process, which can be modeled as medical decision trees (MDTs), is critical to build clinical decision support systems. However, the current MDT construction methods rely heavily on time-consuming and laborious manual annotation. In this work, we propose a novel task, Text2MDT, to explore the automatic extraction of MDTs from medical texts such as medical guidelines and textbooks. We normalize the form of the MDT and create an annotated Text-to-MDT dataset in Chinese with the participation of medical experts. We investigate two different methods for the Text2MDT tasks: (a) an end-to-end framework which only relies on a GPT style large language models (LLM) instruction tuning to generate all the node information and tree structures. (b) The pipeline framework which decomposes the Text2MDT task to three subtasks. Experiments on our Text2MDT dataset demonstrate that: (a) the end-to-end method basd on LLMs (7B parameters or larger) show promising results, and successfully outperform the pipeline methods. (b) The chain-of-thought (COT) prompting method \cite{Wei2022ChainOT} can improve the performance of the fine-tuned LLMs on the Text2MDT test set. (c) the lightweight pipelined method based on encoder-based pretrained models can perform comparably with LLMs with model complexity two magnititudes smaller. Our Text2MDT dataset is open-sourced at \url{https://tianchi.aliyun.com/dataset/95414}, and the source codes are open-sourced at \url{https://github.com/michael-wzhu/text2dt}.
Human-in-the-Loop Policy Optimization for Preference-Based Multi-Objective Reinforcement Learning
Authors: Ke Li, Han Guo
Date: 2024-01-04 00:00:00
Tasks: Decision Making, Management, Multi-Objective Reinforcement Learning

Multi-objective reinforcement learning (MORL) aims to find a set of high-performing and diverse policies that address trade-offs between multiple conflicting objectives. However, in practice, decision makers (DMs) often deploy only one or a limited number of trade-off policies. Providing too many diversified trade-off policies to the DM not only significantly increases their workload but also introduces noise in multi-criterion decision-making. With this in mind, we propose a human-in-the-loop policy optimization framework for preference-based MORL that interactively identifies policies of interest. Our method proactively learns the DM's implicit preference information without requiring any a priori knowledge, which is often unavailable in real-world black-box decision scenarios. The learned preference information is used to progressively guide policy optimization towards policies of interest. We evaluate our approach against three conventional MORL algorithms that do not consider preference information and four state-of-the-art preference-based MORL algorithms on two MORL environments for robot control and smart grid management. Experimental results fully demonstrate the effectiveness of our proposed method in comparison to the other peer algorithms.
Hierarchical Clustering in $Λ$CDM Cosmologies via Persistence Energy
Authors: Michael Etienne Van Huffel, Leonardo Aldo Alejandro Barberi, Tobias Sagis
Date: 2024-01-03 00:00:00
Tasks: Clustering, Topological Data Analysis

In this research, we investigate the structural evolution of the cosmic web, employing advanced methodologies from Topological Data Analysis. Our approach involves leveraging LITE, an innovative method from recent literature that embeds persistence diagrams into elements of vector spaces. Utilizing this methodology, we analyze three quintessential cosmic structures: clusters, filaments, and voids. A central discovery is the correlation between \textit{Persistence Energy} and redshift values, linking persistent homology with cosmic evolution and providing insights into the dynamics of cosmic structures.
Spectral-Based Graph Neural Networks for Complementary Item Recommendation
Authors: Haitong Luo, Xuying Meng, Suhang Wang, Hanyun Cao, Weiyao Zhang, Yequan Wang, Yujun Zhang
Date: 2024-01-04 00:00:00
Tasks: Attribute, Recommendation Systems

Modeling complementary relationships greatly helps recommender systems to accurately and promptly recommend the subsequent items when one item is purchased. Unlike traditional similar relationships, items with complementary relationships may be purchased successively (such as iPhone and Airpods Pro), and they not only share relevance but also exhibit dissimilarity. Since the two attributes are opposites, modeling complementary relationships is challenging. Previous attempts to exploit these relationships have either ignored or oversimplified the dissimilarity attribute, resulting in ineffective modeling and an inability to balance the two attributes. Since Graph Neural Networks (GNNs) can capture the relevance and dissimilarity between nodes in the spectral domain, we can leverage spectral-based GNNs to effectively understand and model complementary relationships. In this study, we present a novel approach called Spectral-based Complementary Graph Neural Networks (SComGNN) that utilizes the spectral properties of complementary item graphs. We make the first observation that complementary relationships consist of low-frequency and mid-frequency components, corresponding to the relevance and dissimilarity attributes, respectively. Based on this spectral observation, we design spectral graph convolutional networks with low-pass and mid-pass filters to capture the low-frequency and mid-frequency components. Additionally, we propose a two-stage attention mechanism to adaptively integrate and balance the two attributes. Experimental results on four e-commerce datasets demonstrate the effectiveness of our model, with SComGNN significantly outperforming existing baseline models.
Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment
Authors: Mengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu, Zhangyang Peng, Xiangyu Ke, Yunjun Gao, Xiaoliang Xu, Rentong Guo, Charles Xie
Date: 2024-01-04 00:00:00

High-dimensional vector similarity search (HVSS) is gaining prominence as a powerful tool for various data science and AI applications. As vector data scales up, in-memory indexes pose a significant challenge due to the substantial increase in main memory requirements. A potential solution involves leveraging disk-based implementation, which stores and searches vector data on high-performance devices like NVMe SSDs. However, implementing HVSS for data segments proves to be intricate in vector databases where a single machine comprises multiple segments for system scalability. In this context, each segment operates with limited memory and disk space, necessitating a delicate balance between accuracy, efficiency, and space cost. Existing disk-based methods fall short as they do not holistically address all these requirements simultaneously. In this paper, we present Starling, an I/O-efficient disk-resident graph index framework that optimizes data layout and search strategy within the segment. It has two primary components: (1) a data layout incorporating an in-memory navigation graph and a reordered disk-based graph with enhanced locality, reducing the search path length and minimizing disk bandwidth wastage; and (2) a block search strategy designed to minimize costly disk I/O operations during vector query execution. Through extensive experiments, we validate the effectiveness, efficiency, and scalability of Starling. On a data segment with 2GB memory and 10GB disk capacity, Starling can accommodate up to 33 million vectors in 128 dimensions, offering HVSS with over 0.9 average precision and top-10 recall rate, and latency under 1 millisecond. The results showcase Starling's superior performance, exhibiting 43.9$\times$ higher throughput with 98% lower query latency compared to state-of-the-art methods while maintaining the same level of accuracy.
Coverage Control Algorithm for DSNs Based on Improved Gravitational Search
Authors: Yindi Yao, Huanmin Liao, Xiong Li, Student Member, IEEE, Feng Zhao, Xuan Yang, and Shanshan Hu
Date: 2022-04-01 00:00:00
Proceeding: IEEE Sensors Journal 2022 4
Tasks: Position

—In directional sensor networks (DSNs), coverage
control is an important way to ensure efficient communication
and reliable data transmission. Aiming at enhancing the DSNs
coverage performance, a coverage control algorithm based
on improved gravitational search (IGSCCA) was proposed.
Firstly, in the coverage enhancement phase, aiming at the
defects of gravitational search algorithm that it is easy to fall
into local optimum and slow convergencespeed when solving
the coverage enhancement problem, cosine decay function is used to improve the gravitational function, the global
optimum position is introduced into the velocity updating of gravitational search algorithm, and use the uncovered grid
points attractive force to disturb the worst particles. Secondly, in the node sleep phase, the node sleep strategy based on
security set is introduced to reduce the network redundancy rate. Finally, the simulation results showed that this paper
has great advantages in coverage rate, energy utilization rate and redundancy rate.
Trainable Self-Guided Filter for Multi-Focus Image Fusion
Authors: Levent Karacan
Date: 2023-11-21 00:00:00
Proceeding: IEEE Access 2023 11
Tasks: Decoder, Generative Adversarial Network, Multi Focus Image Fusion

Cameras are limited in their ability to capture all-in-focus images due to their limited depth of field. This results in blurriness for objects too far in front of or behind the focused point. To overcome this limitation, multi-focus image fusion (MFIF) approaches have been proposed. Although recent MFIF methods have shown promising results for this task, they still need to be improved in terms of artifacts and color degradation. Motivated by these observations, in this paper, we propose a new Generative Adversarial Network (GAN)–based MFIF model to improve fusion quality by predicting more accurate focus maps thanks to a trainable guided filter we incorporated. The proposed model comprises an encoder-decoder network, and a trainable self-guided filtering (TSGF) module that is specifically designed to enhance spatial consistency in the predicted focus map and to eliminate the requirements of post-processing in existing GAN-based methods. The encoder-decoder network first predicts raw focus maps, which are then passed to the TSGF to produce the final focus maps. To train the proposed model effectively, we define three objectives: L1 loss, GAN loss, and Focal Frequency Loss (FFL) in the frequency domain. L1 loss is defined on ground-truth and predicted focus maps, whereas GAN loss and FFL are defined on ground-truth all-in-focus images and fused images. Experimental results show that the proposed approach outperforms the existing GAN-based methods and achieves highly competitive performance with state-of-the-art methods in terms of standard quantitative image fusion metrics and visual quality on three MFIF benchmark datasets.
Comparison of artificial intelligence models for prognosis of breast cancer
Authors: Anish Samantaray, C. Saravanan, Meghana Bollam, R. Maheswari, P. Vijaya
Date: 2023-11-15 00:00:00
Proceeding: Explainable Artificial Intelligence (XAI): Concepts, enabling tools, technologies and applications 2023 11
Tasks: Prognosis

Breast cancer is the most common cancer not only affecting women but also men. Diagnosis and treatment are crucial stages in the cancer treatment process. However, even after treatment, individuals often experience ongoing challenges, including the regular need for painful procedures such as biopsies, MRIs, and scans as part of their journey towards recovery. We propose that in this case, machine learning (ML) and deep learning analyses may be used to perform longitudinal studies of women with breast cancer. We do a comparative analysis of three situations, in the first situation, we apply ML algorithms just after the primary preprocessing steps, in the second situation, we add balanced class weights hyperparameters, and in the third, we do principal component analysis (PCA). In the first situation, the light gradient boosting machine (LightGBM) gives the best accuracy of 87.87%, and the random forest (RF) gives an accuracy of 87.87% after the hyperparameter of balanced class weights is given. After PCA, logistic regression gives a maximum accuracy of 84.84%.
SciOps: Achieving Productivity and Reliability in Data-Intensive Research
Authors: Erik C. Johnson, Thinh T. Nguyen, Benjamin K. Dichter, Frank Zappulla, Montgomery Kosma, Kabilar Gunalan, Yaroslav O. Halchenko, Shay Q. Neufeld, Kristen Ratan, Nicholas J. Edwards, Susanne Ressl, Sarah R. Heilbronner, Michael Schirner, Petra Ritter, Brock Wester, Satrajit Ghosh, Maryann E. Martone, Franco Pestilli, Dimitri Yatsenko
Date: 2023-12-29 00:00:00
Tasks: Experimental Design

Scientists are increasingly leveraging advances in instruments, automation, and collaborative tools to scale up their experiments and research goals, leading to new bursts of discovery. Various scientific disciplines, including neuroscience, have adopted key technologies to enhance collaboration, reproducibility, and automation. Drawing inspiration from advancements in the software industry, we present a roadmap to enhance the reliability and scalability of scientific operations for diverse research teams tackling large and complex projects. We introduce a five-level Capability Maturity Model describing the principles of rigorous scientific operations in projects ranging from small-scale exploratory studies to large-scale, multi-disciplinary research endeavors. Achieving higher levels of operational maturity necessitates the adoption of new, technology-enabled methodologies, which we refer to as SciOps. This concept is derived from the DevOps methodologies that have revolutionized the software industry. SciOps involves digital research environments that seamlessly integrate computational, automation, and AI-driven efforts throughout the research cycle-from experimental design and data collection to analysis and dissemination, ultimately leading to closed-loop discovery. This maturity model offers a framework for assessing and improving operational practices in multidisciplinary research teams, guiding them towards greater efficiency and effectiveness in scientific inquiry.
Turing's Test, a Beautiful Thought Experiment
Authors: Bernardo Gonçalves
Date: 2023-12-18 00:00:00

In the wake of the latest trends of artificial intelligence (AI), there has been a resurgence of claims and questions about the Turing test and its value, which are reminiscent of decades of practical "Turing" tests. If AI were quantum physics, by now several "Schr\"odinger's" cats would have been killed. It is time for a historical reconstruction of Turing's beautiful thought experiment. This paper presents a wealth of evidence, including new archival sources, and gives original answers to several open questions about Turing's 1950 paper, including its relation with early AI.
Emissions Reporting Maturity Model: supporting cities to leverage emissions-related processes through performance indicators and artificial intelligence
Authors: Victor de A. Xavier, Felipe M. G. França, Priscila M. V. Lima
Date: 2023-12-08 00:00:00
Tasks: Clustering

Climate change and global warming have been trending topics worldwide since the Eco-92 conference. However, little progress has been made in reducing greenhouse gases (GHGs). The problems and challenges related to emissions are complex and require a concerted and comprehensive effort to address them. Emissions reporting is a critical component of GHG reduction policy and is therefore the focus of this work. The main goal of this work is two-fold: (i) to propose an emission reporting evaluation model to leverage emissions reporting overall quality and (ii) to use artificial intelligence (AI) to support the initiatives that improve emissions reporting. Thus, this work presents an Emissions Reporting Maturity Model (ERMM) for examining, clustering, and analysing data from emissions reporting initiatives to help the cities to deal with climate change and global warming challenges. The Performance Indicator Development Process (PIDP) proposed in this work provides ways to leverage the quality of the available data necessary for the execution of the evaluations identified by the ERMM. Hence, the PIDP supports the preparation of the data from emissions-related databases, the classification of the data according to similarities highlighted by different clustering techniques, and the identification of performance indicator candidates, which are strengthened by a qualitative analysis of selected data samples. Thus, the main goal of ERRM is to evaluate and classify the cities regarding the emission reporting processes, pointing out the drawbacks and challenges faced by other cities from different contexts, and at the end to help them to leverage the underlying emissions-related processes and emissions mitigation initiatives.
HITSnDIFFs: From Truth Discovery to Ability Discovery by Recovering Matrices with the Consecutive Ones Property
Authors: Zixuan Chen, Subhodeep Mitra, R Ravi, Wolfgang Gatterbauer
Date: 2023-12-21 00:00:00

We analyze a general problem in a crowd-sourced setting where one user asks a question (also called item) and other users return answers (also called labels) for this question. Different from existing crowd sourcing work which focuses on finding the most appropriate label for the question (the "truth"), our problem is to determine a ranking of the users based on their ability to answer questions. We call this problem "ability discovery" to emphasize the connection to and duality with the more well-studied problem of "truth discovery". To model items and their labels in a principled way, we draw upon Item Response Theory (IRT) which is the widely accepted theory behind standardized tests such as SAT and GRE. We start from an idealized setting where the relative performance of users is consistent across items and better users choose better fitting labels for each item. We posit that a principled algorithmic solution to our more general problem should solve this ideal setting correctly and observe that the response matrices in this setting obey the Consecutive Ones Property (C1P). While C1P is well understood algorithmically with various discrete algorithms, we devise a novel variant of the HITS algorithm which we call "HITSNDIFFS" (or HND), and prove that it can recover the ideal C1P-permutation in case it exists. Unlike fast combinatorial algorithms for finding the consecutive ones permutation (if it exists), HND also returns an ordering when such a permutation does not exist. Thus it provides a principled heuristic for our problem that is guaranteed to return the correct answer in the ideal setting. Our experiments show that HND produces user rankings with robustly high accuracy compared to state-of-the-art truth discovery methods. We also show that our novel variant of HITS scales better in the number of users than ABH, the only prior spectral C1P reconstruction algorithm.
Modeling of Memory Mechanisms in Cerebral Cortex and Simulation of Storage Performance
Authors: Hui Wei, Chenyue Feng, Jianning Zhang
Date: 2023-12-31 00:00:00
Tasks: Anatomy, Decision Making

At the intersection of computation and cognitive science, graph theory is utilized as a formalized description of complex relationships and structures. Traditional graph models are often static, lacking dynamic and autonomous behavioral patterns. They rely on algorithms with a global view, significantly differing from biological neural networks, in which, to simulate information storage and retrieval processes, the limitations of centralized algorithms must be overcome. This study introduces a directed graph model that equips each node with adaptive learning and decision-making capabilities, thereby facilitating decentralized dynamic information storage and modeling and simulation of the brain's memory process. We abstract different storage instances as directed graph paths, transforming the storage of information into the assignment, discrimination, and extraction of different paths. To address writing and reading challenges, each node has a personalized adaptive learning ability. A storage algorithm without a God's eye view is developed, where each node uses its limited neighborhood information to facilitate the extension, formation, solidification, and awakening of directed graph paths, achieving competitive, reciprocal, and sustainable utilization of limited resources. Storage behavior occurs in each node, with adaptive learning behaviors of nodes concretized in a microcircuit centered around a variable resistor, simulating the electrophysiological behavior of neurons. Under the constraints of neurobiology on the anatomy and electrophysiology of biological neural networks, this model offers a plausible explanation for the mechanism of memory realization, providing a comprehensive, system-level experimental validation of the memory trace theory.
Fast Quantum Convolutional Neural Networks for Low-Complexity Object Detection in Autonomous Driving Applications
Authors: Hankyul Baek, Donghyeon Kim, Joongheon Kim
Date: 2023-12-28 00:00:00
Tasks: Autonomous Driving, Object, object-detection, Object Detection

Spurred by consistent advances and innovation in deep learning, object detection applications have become prevalent, particularly in autonomous driving that leverages various visual data. As convolutional neural networks (CNNs) are being optimized, the performances and computation speeds of object detection in autonomous driving have been significantly improved. However, due to the exponentially rapid growth in the complexity and scale of data used in object detection, there are limitations in terms of computation speeds while conducting object detection solely with classical computing. Motivated by this, quantum convolution-based object detection (QCOD) is proposed to adopt quantum computing to perform object detection at high speed. The QCOD utilizes our proposed fast quantum convolution that uploads input channel information and re-constructs output channels for achieving reduced computational complexity and thus improving performances. Lastly, the extensive experiments with KITTI autonomous driving object detection dataset verify that the proposed fast quantum convolution and QCOD are successfully operated in real object detection applications.