pwc-archive/papers-with-abstracts huggingface dataset:
RAG-Fusion: a New Take on Retrieval-Augmented Generation
Authors: Zackary Rackauckas
Date: 2024-01-31 00:00:00
Tasks: RAG, Reranking, Retrieval, Retrieval-augmented Generation

Infineon has identified a need for engineers, account managers, and customers to rapidly obtain product information. This problem is traditionally addressed with retrieval-augmented generation (RAG) chatbots, but in this study, I evaluated the use of the newly popularized RAG-Fusion method. RAG-Fusion combines RAG and reciprocal rank fusion (RRF) by generating multiple queries, reranking them with reciprocal scores and fusing the documents and scores. Through manually evaluating answers on accuracy, relevance, and comprehensiveness, I found that RAG-Fusion was able to provide accurate and comprehensive answers due to the generated queries contextualizing the original query from various perspectives. However, some answers strayed off topic when the generated queries' relevance to the original query is insufficient. This research marks significant progress in artificial intelligence (AI) and natural language processing (NLP) applications and demonstrates transformations in a global and multi-industry context.
Uncertainty-Aware Explainable Recommendation with Large Language Models
Authors: Yicui Peng, Hao Chen, ChingSheng Lin, Guo Huang, Jinrong Hu, Hui Guo, Bin Kong, Shu Hu, Xi Wu, Xin Wang
Date: 2024-01-31 00:00:00
Tasks: Explainable Recommendation, Multi-Task Learning

Providing explanations within the recommendation system would boost user satisfaction and foster trust, especially by elaborating on the reasons for selecting recommended items tailored to the user. The predominant approach in this domain revolves around generating text-based explanations, with a notable emphasis on applying large language models (LLMs). However, refining LLMs for explainable recommendations proves impractical due to time constraints and computing resource limitations. As an alternative, the current approach involves training the prompt rather than the LLM. In this study, we developed a model that utilizes the ID vectors of user and item inputs as prompts for GPT-2. We employed a joint training mechanism within a multi-task learning framework to optimize both the recommendation task and explanation task. This strategy enables a more effective exploration of users' interests, improving recommendation effectiveness and user satisfaction. Through the experiments, our method achieving 1.59 DIV, 0.57 USR and 0.41 FCR on the Yelp, TripAdvisor and Amazon dataset respectively, demonstrates superior performance over four SOTA methods in terms of explainability evaluation metric. In addition, we identified that the proposed model is able to ensure stable textual quality on the three public datasets.
Heterophily-Aware Fair Recommendation using Graph Convolutional Networks
Authors: Nemat Gholinejad, Mostafa Haghir Chehreghani
Date: 2024-01-31 00:00:00
Tasks: Fairness, Recommendation Systems

In recent years, graph neural networks (GNNs) have become a popular tool to improve the accuracy and performance of recommender systems. Modern recommender systems are not only designed to serve end users, but also to benefit other participants, such as items and item providers. These participants may have different or conflicting goals and interests, which raises the need for fairness and popularity bias considerations. GNN-based recommendation methods also face the challenges of unfairness and popularity bias, and their normalization and aggregation processes suffer from these challenges. In this paper, we propose a fair GNN-based recommender system, called HetroFair, to improve item-side fairness. HetroFair uses two separate components to generate fairness-aware embeddings: i) Fairness-aware attention, which incorporates the dot product in the normalization process of GNNs to decrease the effect of nodes' degrees. ii) Heterophily feature weighting, to assign distinct weights to different features during the aggregation process. To evaluate the effectiveness of HetroFair, we conduct extensive experiments over six real-world datasets. Our experimental results reveal that HetroFair not only alleviates unfairness and popularity bias on the item side but also achieves superior accuracy on the user side. Our implementation is publicly available at https://github.com/NematGH/HetroFair.
NanoNER: Named Entity Recognition for nanobiology using experts' knowledge and distant supervision
Authors: Martin Lentschat, Cyril Labbé, Ran Cheng
Date: 2024-01-30 00:00:00
Tasks: Articles, Experimental Design, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Here we present the training and evaluation of NanoNER, a Named Entity Recognition (NER) model for Nanobiology. NER consists in the identification of specific entities in spans of unstructured texts and is often a primary task in Natural Language Processing (NLP) and Information Extraction. The aim of our model is to recognise entities previously identified by domain experts as constituting the essential knowledge of the domain. Relying on ontologies, which provide us with a domain vocabulary and taxonomy, we implemented an iterative process enabling experts to determine the entities relevant to the domain at hand. We then delve into the potential of distant supervision learning in NER, supporting how this method can increase the quantity of annotated data with minimal additional manpower. On our full corpus of 728 full-text nanobiology articles, containing more than 120k entity occurrences, NanoNER obtained a F1-score of 0.98 on the recognition of previously known entities. Our model also demonstrated its ability to discover new entities in the text, with precision scores ranging from 0.77 to 0.81. Ablation experiments further confirmed this and allowed us to assess the dependency of our approach on the external resources. It highlighted the dependency of the approach to the resource, while also confirming its ability to rediscover up to 30% of the ablated terms. This paper details the methodology employed, experimental design, and key findings, providing valuable insights and directions for future related researches on NER in specialized domain. Furthermore, since our approach require minimal manpower , we believe that it can be generalized to other specialized fields.
Transfer Learning With Densenet201 Architecture Model For Potato Leaf Disease Classification
Authors: Rifqi Alfinnur Charisma, Faisal Dharma Adhinata
Date: 2024-01-25 00:00:00
Tasks: Classification, Transfer Learning

Potato plants are plants that are beneficial to humans. Like other plants in general, potato plants also have diseases; if this disease is not treated immediately, there will be a significant decrease in food production. Therefore, it is necessary to detect diseases quickly and precisely so that disease control can be carried out effectively and efficiently. Classification of potato leaf disease can be done directly. Still, the symptoms cannot always explain the type of disease that attacks potato leaves because there are many types of diseases with symptoms that look the same. Humans also have deficiencies in determining the results of identification of potato leaf disease, so sometimes the results of identification between individuals can be different. Therefore, the use of Deep Learning for the classification process of potato leaf disease is expected to shorten the time and have a high classification accuracy. This study uses a deep learning method with the DenseNet201 architecture. The choice to use the DenseNet201 algorithm in this study is because the model can identify important features of potato leaves and recognize early signs of emerging diseases. This study aimed to evaluate the effectiveness of the transfer learning method with the DenseNet201 architecture in increasing the classification accuracy of potato leaf disease compared to traditional classification methods. This study uses two types of scenarios, namely, comparing the number of dropouts and comparing the three optimizers. This test produces the best model using dropout 0.1 and Adam optimizer with an accuracy of 99.5% for training, 95.2% for validation, and 96% for the confusion matrix. In this study, using data testing, as many as 40 images were tested into the model that has been built. The test results on this model resulted in a new accuracy for classifying potato leaf disease, namely 92.5%.
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
Authors: Amirhossein Nourian; Majid Sadedel
Date: 2023-12-19 00:00:00
Proceeding: 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM) 2023 12
Tasks: OpenAI Gym, Pathfinder, Q-Learning, reinforcement-learning, Reinforcement Learning

Self-reinforcement algorithms, especially Q-learning and value iteration have been popular and effective for navigating mobile robots. To provide a deeper insight through their performance and usage, this paper investigates solving a simple Q-Learning problem for various working environments for mobile robots. No matter how quick or reliable they are, all temporal difference algorithms employ iterative methods to produce desired results. For this reason, in this paper, we looked at the relationship between episode steps, mesh sizes, and most importantly the impact of the environment on both convergence rate and solution reliability. As previously indicated, the majority of the conclusions of this study about the relationship between computation cost and environment and also dependability can be transferred to more sophisticated temporal difference-based algorithms because all methods are iterative. As many robotic applications for reinforcement learning have two or three-dimensional state space for path planning, knowing the approximate convergence episode count would be a useful tool for researchers to tackle non-optimal episodes, especially on embedded systems with computational constraints. The content also discussed the relationship between three mesh sizes and tried to guide researchers to implement their reinforcement learning pipeline more effectively. As shown by this work, the performance and reliability of iterative reinforcement learning algorithms depend deeply on the environment. In certain cases, increasing the number of episodes would not alter the results entirely. Therefore, when solving these kinds of maps, researchers would refrain from using Q-learning techniques.
Artificial Intelligence for EEG Prediction: Applied Chaos Theory
Authors: Soul Syrup
Date: 2023-10-03 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Prediction

In the present research, we delve into the intricate realm of electroencephalogram (EEG) data analysis, focusing on sequence-to-sequence prediction of data across 32 EEG channels. The study harmoniously fuses the principles of applied chaos theory and dynamical systems theory to engender a novel feature set, enriching the representational capacity of our deep learning model. The endeavour's cornerstone is a transformer-based sequence-to-sequence architecture, calibrated meticulously to capture the non-linear and high-dimensional temporal dependencies inherent in EEG sequences. Through judicious architecture design, parameter initialisation strategies, and optimisation techniques, we have navigated the intricate balance between computational expediency and predictive performance. Our model stands as a vanguard in EEG data sequence prediction, demonstrating remarkable generalisability and robustness. The findings not only extend our understanding of EEG data dynamics but also unveil a potent analytical framework that can be adapted to diverse temporal sequence prediction tasks in neuroscience and beyond.
Design and implementation of a real-time onboard system for a stratospheric balloon mission using commercial off-the-self components and a model-based approach
Authors: Angel-Grover Perez-Munoz, Jose-Carlos Gamazo-Real, David Gonzalez-Barcena, Juan Zamorano
Date: 2024-02-06 00:00:00
Tasks: Code Generation

Stratospheric balloons have emerged as an affordable and flexible alternative to traditional spacecrafts as they are implemented using commercial off-the-shelf (COTS) equipment without following strict methodologies. HERCCULES is a stratospheric balloon mission that aims to characterize the convective heat and radiative environment in the stratosphere. The purpose of this article is to present the HERCCULES onboard software (OBSW) whose design and complexity is comparable to that of satellite systems, since it must control about sixty COTS equipment using a single Raspberry Pi 4B as onboard computer and ensure the real-time requirements. Compared to similar systems, novel contributions are presented as the OBSW is developed following modelbased and component-based approaches using the TASTE toolchain from the European Space Agency (ESA) for automatic code generation. Besides, the OBSW is verified and validated following the ESA standards and the results obtained demonstrate the suitability and efficiency of the solution and the selected methodologies.
Variable regularization affine projection sign algorithm in impulsive noisy environment
Authors: Ying-Ren Chien
Date: 2019-05-01 00:00:00
Proceeding: IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences 2019 5

Affine projection sign algorithm (APSA) is an important adaptive filtering method to combat the impulsive noisy environment.
However, the performance of APSA is poor, if its regularization parameter is not well chosen. We propose a variable regularization APSA (VR-APSA) approach, which adopts a gradient-based method to recursively reduce the norm of the \textsl{a priori} error vector. The resulting VR-APSA leverages the time correlation of both the input signal matrix and error vector to adjust the value of the regularization parameter. Simulation results confirm that our algorithm exhibits both fast convergence and small misadjustment properties.
The Role of Child Gender in the Formation of Parents' Social Networks
Authors: Aristide Houndetoungan, Asad Islam, Michael Vlassopoulos, Yves Zenou
Date: 2024-02-06 00:00:00
Tasks: counterfactual

Social networks play an important role in various aspects of life. While extensive research has explored factors such as gender, race, and education in network formation, one dimension that has received less attention is the gender of one's child. Children tend to form friendships with same-gender peers, potentially leading their parents to interact based on their child's gender. Focusing on households with children aged 3-5, we leverage a rich dataset from rural Bangladesh to investigate the role of children's gender in parental network formation. We estimate an equilibrium model of network formation that considers a child's gender alongside other socioeconomic factors. Counterfactual analyses reveal that children's gender significantly shapes parents' network structure. Specifically, if all children share the same gender, households would have approximately 15% more links, with a stronger effect for families having girls. Importantly, the impact of children's gender on network structure is on par with or even surpasses that of factors such as income distribution, parental occupation, education, and age. These findings carry implications for debates surrounding coed versus single-sex schools, as well as policies that foster inter-gender social interactions among children.
A generalized decision tree ensemble based on the NeuralNetworks architecture: Distributed Gradient Boosting Forest (DGBF)
Authors: Ángel Delgado-Panadero, José Alberto Benítez-Andrades, María Teresa García-Ordás
Date: 2024-02-04 00:00:00
Tasks: Representation Learning

Tree ensemble algorithms as RandomForest and GradientBoosting are currently the dominant methods for modeling discrete or tabular data, however, they are unable to perform a hierarchical representation learning from raw data as NeuralNetworks does thanks to its multi-layered structure, which is a key feature for DeepLearning problems and modeling unstructured data. This limitation is due to the fact that tree algorithms can not be trained with back-propagation because of their mathematical nature. However, in this work, we demonstrate that the mathematical formulation of bagging and boosting can be combined together to define a graph-structured-tree-ensemble algorithm with a distributed representation learning process between trees naturally (without using back-propagation). We call this novel approach Distributed Gradient Boosting Forest (DGBF) and we demonstrate that both RandomForest and GradientBoosting can be expressed as particular graph architectures of DGBT. Finally, we see that the distributed learning outperforms both RandomForest and GradientBoosting in 7 out of 9 datasets.
BetterV: Controlled Verilog Generation with Discriminative Guidance
Authors: Zehua Pei, Hui-Ling Zhen, Mingxuan Yuan, Yu Huang, Bei Yu
Date: 2024-02-03 00:00:00
Tasks: Text Generation

Due to the growing complexity of modern Integrated Circuits (ICs), there is a need for automated circuit design methods. Recent years have seen rising research in hardware design language generation to facilitate the design process. In this work, we propose a Verilog generation framework, BetterV, which fine-tunes the large language models (LLMs) on processed domain-specific datasets and incorporates generative discriminators for guidance on particular design demands. The Verilog modules are collected, filtered and processed from internet to form a clean and abundant dataset. Instruct-tuning methods are specially designed to fine-tune the LLMs to understand the knowledge about Verilog. Furthermore, data are augmented to enrich the training set and also used to train a generative discriminator on particular downstream task, which leads a guidance for the LLMs to optimize the Verilog implementation. BetterV has the ability to generate syntactically and functionally correct Verilog, which can outperform GPT-4 on the VerilogEval benchmark. With the help of task-specific generative discriminator, BetterV can achieve remarkable improvement on various electronic design automation (EDA) downstream tasks, including the netlist node reduction for synthesis and verification runtime reduction with Boolean Satisfiability (SAT) solving.
Gaussian Plane-Wave Neural Operator for Electron Density Estimation
Authors: Seongsu Kim, Sungsoo Ahn
Date: 2024-02-05 00:00:00
Tasks: Density Estimation

This work studies machine learning for electron density prediction, which is fundamental for understanding chemical systems and density functional theory (DFT) simulations. To this end, we introduce the Gaussian plane-wave neural operator (GPWNO), which operates in the infinite-dimensional functional space using the plane-wave and Gaussian-type orbital bases, widely recognized in the context of DFT. In particular, both high- and low-frequency components of the density can be effectively represented due to the complementary nature of the two bases. Extensive experiments on QM9, MD, and material project datasets demonstrate GPWNO's superior performance over ten baselines.
StableMask: Refining Causal Masking in Decoder-only Transformer
Authors: Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao, Jianhua Yao, Xiaoyu Shen, Qiang Zhang
Date: 2024-02-07 00:00:00
Tasks: Decoder, Language Modeling, Language Modelling, Position

The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional performance across various tasks, we have identified two limitations: First, it requires all attention scores to be non-zero and sum up to 1, even if the current embedding has sufficient self-contained information. This compels the model to assign disproportional excessive attention to specific tokens. Second, RPE-based Transformers are not universal approximators due to their limited capacity at encoding absolute positional information, which limits their application in position-critical tasks. In this work, we propose StableMask: a parameter-free method to address both limitations by refining the causal mask. It introduces pseudo-attention values to balance attention distributions and encodes absolute positional information via a progressively decreasing mask ratio. StableMask's effectiveness is validated both theoretically and empirically, showing significant enhancements in language models with parameter sizes ranging from 71M to 1.4B across diverse datasets and encoding methods. We further show that it naturally supports (1) efficient extrapolation without special tricks such as StreamingLLM and (2) easy integration with existing attention optimization techniques.
Navigating Complexity: Toward Lossless Graph Condensation via Expanding Window Matching
Authors: Yuchen Zhang, Tianle Zhang, Kai Wang, Ziyao Guo, Yuxuan Liang, Xavier Bresson, Wei Jin, Yang You
Date: 2024-02-07 00:00:00

Graph condensation aims to reduce the size of a large-scale graph dataset by synthesizing a compact counterpart without sacrificing the performance of Graph Neural Networks (GNNs) trained on it, which has shed light on reducing the computational cost for training GNNs. Nevertheless, existing methods often fall short of accurately replicating the original graph for certain datasets, thereby failing to achieve the objective of lossless condensation. To understand this phenomenon, we investigate the potential reasons and reveal that the previous state-of-the-art trajectory matching method provides biased and restricted supervision signals from the original graph when optimizing the condensed one. This significantly limits both the scale and efficacy of the condensed graph. In this paper, we make the first attempt toward \textit{lossless graph condensation} by bridging the previously neglected supervision signals. Specifically, we employ a curriculum learning strategy to train expert trajectories with more diverse supervision signals from the original graph, and then effectively transfer the information into the condensed graph with expanding window matching. Moreover, we design a loss function to further extract knowledge from the expert trajectories. Theoretical analysis justifies the design of our method and extensive experiments verify its superiority across different datasets. Code is released at https://github.com/NUS-HPC-AI-Lab/GEOM.
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
Authors: Haoyi Niu, Jianming Hu, Guyue Zhou, Xianyuan Zhan
Date: 2024-02-07 00:00:00

The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, collecting sufficient unbiased data from the target domain remains a challenge due to costly data collection processes and stringent safety requirements. Consequently, researchers often resort to data from easily accessible source domains, such as simulation and laboratory environments, for cost-effective data acquisition and rapid model iteration. Nevertheless, the environments and embodiments of these source domains can be quite different from their target domain counterparts, underscoring the need for effective cross-domain policy transfer approaches. In this paper, we conduct a systematic review of existing cross-domain policy transfer methods. Through a nuanced categorization of domain gaps, we encapsulate the overarching insights and design considerations of each problem setting. We also provide a high-level discussion about the key methodologies used in cross-domain policy transfer problems. Lastly, we summarize the open challenges that lie beyond the capabilities of current paradigms and discuss potential future directions in this field.