pwc-archive/papers-with-abstracts huggingface dataset:
In-Context Code-Text Learning for Bimodal Software Engineering
Authors: Xunzhu Tang, Liran Wang, Yonghui Liu, Linzheng Chai, Jian Yang, Zhoujun Li, Haoye Tian, Jacques Klein, Tegawende F. Bissyande
Date: 2024-10-08 00:00:00
Tasks: Clone Detection, In-Context Learning, Program Repair, Prompt Learning, Retrieval-augmented Generation

Bimodal software analysis initially appeared to be within reach with the advent of large language models. Unfortunately, the complex interplay of natural language text and code in software engineering, presents unique challenges that prevent pretrained models to generalize to a variety of tasks. We postulate that in-context learning for the code-text bimodality is a promising avenue. This paper thus introduces a comprehensive study of in-context code-text learning, focusing on leveraging pretrained CodeLLAMA models. We consider a diverse dataset encompassing 23 software engineering tasks, which we transform in an in-context learning format. To effectively extract informative features, we propose a configurable prompt template. Our proposed pipeline, InCTRL, then unifies prompt learning across various software engineering tasks. Extensive evaluation on the study datasets demonstrates the superiority of INCTRL-models in few-shot performance, surpassing state-of-the-art models including the support model, CodeLLAMA. Typically, we observe that applied to the CodeLLAMA model, INCTRL brings improvements in terms of precision (at least about 12\%) and recall (up to 93.88\%) on various tasks. For example, on the task of program repair, INCTRL improves the BLEU score of CodeLLAMA by 85 points, while for clone detection, INCTRL achieves an improvement of 69 percentage points. Moreover, INCTRL-models offer state-of-the-art performance when using retrieval-augmented generation on individual downstream tasks. Finally, we qualitatively analyze the benefits of INCTRL over CodeLLAMA and open-source all models for broader impact. We make our code and dataset publicly available at: \begin{center} {\url{https://anonymous.4open.science/r/inctrl-B65B}} \end{center}
Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond
Authors: Shanshan Han
Date: 2024-10-09 00:00:00

The advancements in generative AI inevitably raise concerns about their risks and safety implications, which, in return, catalyzes significant progress in AI safety. However, as this field continues to evolve, a critical question arises: are our current efforts on AI safety aligned with the advancements of AI as well as the long-term goal of human civilization? This paper presents a blueprint for an advanced human society and leverages this vision to guide current AI safety efforts. It outlines a future where the Internet of Everything becomes reality, and creates a roadmap of significant technological advancements towards this envisioned future. For each stage of the advancements, this paper forecasts potential AI safety issues that humanity may face. By projecting current efforts against this blueprint, this paper examines the alignment between the current efforts and the long-term needs, and highlights unique challenges and missions that demand increasing attention from AI safety practitioners in the 2020s. This vision paper aims to offer a broader perspective on AI safety, emphasizing that our current efforts should not only address immediate concerns but also anticipate potential risks in the expanding AI landscape, thereby promoting a safe and sustainable future of AI and human civilization.
The Impact of Generative Artificial Intelligence on Ideation and the performance of Innovation Teams (Preprint)
Authors: Michael Gindert, Marvin Lutz Müller
Date: 2024-09-23 00:00:00
Tasks: Diversity, Management

This study investigates the impact of Generative Artificial Intelligence (GenAI) on the dynamics and performance of innovation teams during the idea generation phase of the innovation process. Utilizing a custom AI-augmented ideation tool, the study applies the Knowledge Spillover Theory of Entrepreneurship to understand the effects of AI on knowledge spillover, generation and application. Through a framed field experiment with participants divided into experimental and control groups, findings indicate that AI-augmented teams generated higher quality ideas in less time. GenAI application led to improved efficiency, knowledge exchange, increased satisfaction and engagement as well as enhanced idea diversity. These results highlight the transformative role of the field of AI within the innovation management domain and shows that GenAI has a positive impact on important elements of the Knowledge Spillover Theory of Entrepreneurship, emphasizing its potential impact on innovation, entrepreneurship, and economic growth. Future research should further explore the dynamic interaction between GenAI and creative processes.
Graph Contrastive Learning via Cluster-refined Negative Sampling for Semi-supervised Text Classification
Authors: Wei Ai, Jianbin Li, Ze Wang, JiaYi Du, Tao Meng, Yuntao Shou, Keqin Li
Date: 2024-10-18 00:00:00
Tasks: Classification, Clustering, Contrastive Learning, Semi-Supervised Text Classification, text-classification, Text Classification

Graph contrastive learning (GCL) has been widely applied to text classification tasks due to its ability to generate self-supervised signals from unlabeled data, thus facilitating model training. However, existing GCL-based text classification methods often suffer from negative sampling bias, where similar nodes are incorrectly paired as negative pairs. This can lead to over-clustering, where instances of the same class are divided into different clusters. To address the over-clustering issue, we propose an innovative GCL-based method of graph contrastive learning via cluster-refined negative sampling for semi-supervised text classification, namely ClusterText. Firstly, we combine the pre-trained model Bert with graph neural networks to learn text representations. Secondly, we introduce a clustering refinement strategy, which clusters the learned text representations to obtain pseudo labels. For each text node, its negative sample set is drawn from different clusters. Additionally, we propose a self-correction mechanism to mitigate the loss of true negative samples caused by clustering inconsistency. By calculating the Euclidean distance between each text node and other nodes within the same cluster, distant nodes are still selected as negative samples. Our proposed ClusterText demonstrates good scalable computing, as it can effectively extract important information from from a large amount of data. Experimental results demonstrate the superiority of ClusterText in text classification tasks.
CLOUDSPAM: Contrastive Learning On Unlabeled Data for Segmentation and Pre-Training Using Aggregated Point Clouds and MoCo
Authors: Reza Mahmoudi Kouhi, Olivier Stocker, Philippe Giguère, and Sylvie Daniel
Date: 2024-10-26 00:00:00
Proceeding: Remote Sensing 2024 10
Tasks: 3D Semantic Segmentation, Autonomous Driving, Contrastive Learning, Data Augmentation, Diversity, LIDAR Semantic Segmentation, object-detection, Object Detection, Semantic Segmentation

SegContrast paved the way for contrastive learning on outdoor point clouds. Its original formulation targeted individual scans in applications like autonomous driving and object detection. However, mobile mapping purposes such as digital twin cities and urban planning require large-scale dense datasets to capture the full complexity and diversity present in outdoor environments. In this paper, the SegContrast method is revisited and adapted to overcome its limitations associated with mobile mapping datasets, namely the scarcity of contrastive pairs and memory constraints. To overcome the scarcity of contrastive pairs, we propose the merging of heterogeneous datasets. However, this merging is not a straightforward procedure due to the variety of size and number of points in the point clouds of these datasets. Therefore, a data augmentation approach is designed to create a vast number of segments while optimizing the size of the point cloud samples to the allocated memory. This methodology, called CLOUDSPAM, guarantees the performance of the self-supervised model for both small- and large-scale mobile mapping point clouds. Overall, the results demonstrate the benefits of utilizing datasets with a wide range of densities and class diversity. CLOUDSPAM matched the state of the art on the KITTI-360 dataset, with a 63.6% mIoU, and came in second place on the Toronto-3D dataset. Finally, CLOUDSPAM achieved competitive results against its fully supervised counterpart with only 10% of labeled data.
Rapid Shear Capacity Prediction of TRM-Strengthened Unreinforced Masonry Walls through Interpretable Machine Learning using a Web App
Authors: Petros Lazaridis, Athanasia Thomoglou
Date: 2024-04-30 00:00:00
Tasks: Interpretable Machine Learning

The presented study aims to provide an efficient and reliable tool for rapid estimation of the shear capacity of a TRM-strengthened masonry wall. For this purpose, a data-driven methodology based on a machine learning system is proposed using a dataset constituted of experimental results selected from the bibliography. The outlier points were detected using Cook's distance methodology and removed from the raw dataset, which consisted of 113 examples and 11 input variables. In the processed dataset, 17 Machine Learning methods were trained, optimized through hyperparameter tuning, and compared on the test set. The most effective models are combined into a voting model to leverage the predictive capacity of more than a single regressor. The final blended model shows remarkable predicting capacity with the determination factor ($R^2$) equal to 0.95 and the mean absolute percentage error equal to 8.03\%. In sequence, machine learning interpretation methods are applied to find how the predictors influence the target output. $A_m$, $f_t$, and $n\cdot t_f$ were identified as the most significant predictors with a mainly positive influence on the shear capacity. Finally, the built ML system is employed in a user-friendly web app for easy access and usage by professionals and researchers.
Enhancing TinyBERT for Financial Sentiment Analysis Using GPT-Augmented FinBERT Distillation
Authors: Graison Jos Thomas
Date: 2024-09-19 00:00:00
Tasks: Data Augmentation, Edge-computing, Knowledge Distillation, Sentiment Analysis

In the rapidly evolving field of financial sentiment analysis, the efficiency and accuracy of predictive models are critical due to their significant impact on financial markets. Transformer based models like BERT and large language models (LLMs) like GPT-4, have advanced NLP tasks considerably. Despite their advantages, BERT-based models face challenges with computational intensity in edge computing environments, and the substantial size and compute requirements of LLMs limit their practical deployment. This study proposes leveraging the generative capabilities of LLMs, such as GPT-4 Omni, to create synthetic, domain-specific training data. This approach addresses the challenge of data scarcity and enhances the performance of smaller models by making them competitive with their larger counterparts. The research specifically aims to enhance FinBERT, a BERT model fine-tuned for financial sentiment analysis, and develop TinyFinBERT, a compact transformer model, through a structured, two-tiered knowledge distillation strategy. Using data augmented by GPT-4 Omni, which involves generating new training examples and transforming existing data, we significantly improved the accuracy of FinBERT, preparing it to serve as a teacher model. This enhanced FinBERT then distilled knowledge to TinyFinBERT, employing both GPT-4 Omni and GPT-3.5 Turbo augmented data. The distillation strategy incorporated both logit and intermediate layer distillation. The training and evaluation of TinyFinBERT utilized the PhraseBank dataset and the FiQA 2018 Task1 dataset, achieving performance comparable to FinBERT while being substantially smaller and more efficient. This research demonstrates how LLMs can effectively contribute to the advancement of financial sentiment analysis by enhancing the capabilities of smaller, more efficient models through innovative data augmentation and distillation techniques.
A Deep-Based Approach for Multi-Descriptor Feature Extraction: Applications on SAR Image Registration
Authors: Javid Norouzi, Mohammad Sadegh Helfroush ∗, Alireza Liaghat, Habibollah Danyali
Date: 2024-11-05 00:00:00
Proceeding: Expert Systems with Applications 2024 11
Tasks: Image Registration

Synthetic aperture radar (SAR) images have found increasing attention in various applications, such as remote sensing, surveillance, and disaster management. Unlike optical sensors, SAR can capture images in any weather conditions at any time, making it particularly valuable. However, it poses unique challenges for image processing tasks, like complex speckle noise, texture, and geometric distortions. For many image processing tasks such as image registration; feature detection and extraction are critical initial steps. In this work, we propose a deep learning-based method for detecting and describing keypoints in SAR images, which can be specifically applied to SAR image registration tasks. To address the challenges of SAR images, we utilize the R2D2 network and propose a training scheme specifically tailored for these images. Furthermore, our approach involves multiple descriptors utilization for outlier removal between image keypoints and a two-stage pipeline consisting of coarse and fine stages for robust feature point matching of SAR images. By leveraging the coarse registration stage, which maintains robustness, and also a fine stage that employs multiple descriptors along with a local search technique, our approach enhances the keypoint matching accuracy. Evaluation results show that the proposed method outperforms most of the traditional and deep learning-based methods in terms of the percentage of correct matches, while achieving comparable or even better root mean squared error (RMSE) results.
Suspension Dynamic Performance Analysis of Multi-axle Vehicles
Authors: Bo-Yi, Lin
Date: 2023-08-01 00:00:00
Proceeding: Master Thesis 2023 8

The purpose of this study is to analyze the suspension dynamic performance of multi-axle vehicles, and to investigate the effects of each vehicle parameter on the vehicle dynamics. A generalized numerical model of multi-axle vehicles is developed using MATLAB to simulate the vehicle dynamics of multi-axle vehicles and to observe the time and frequency response. First, the basic properties of the vehicle suspension system are analyzed. Then, the effect of axle numbers on suspension dynamic performance is analyzed. It is found that when the number of axles increases, the wheelbase filtering effect is more significant and the suspension dynamic performance is improved. Moreover, the effects of vehicle parameters on the suspension dynamic performance of multi-axle vehicles are analyzed and a generalized conclusion on the effects of suspension parameter configurations and center of gravity position on vehicle dynamics is also proposed. Finally, a method to optimize the suspension dynamic performance is proposed to provide a basis for the design of the chassis of the multi-axle vehicle and to verify the conclusions presented. According to the analysis of this study, the generalized properties of multi-axle vehicle suspension dynamic performance is understood.
Sentiment-Driven Community Detection in a Network of Perfume Preferences
Authors: Kamand Kalashi, Sajjad Saed, Babak Teimourpour
Date: 2024-10-24 00:00:00
Tasks: Community Detection, Graph Clustering, graph construction, Marketing, Network Community Partition, Persian Sentiment Analysis, Recommendation Systems, Sentiment Analysis, Sentiment Classification

Network analysis is increasingly important across various fields, including the fragrance industry, where perfumes are represented as nodes and shared user preferences as edges in perfume networks. Community detection can uncover clusters of similar perfumes, providing insights into consumer preferences, enhancing recommendation systems, and informing targeted marketing strategies. This study aims to apply community detection techniques to group perfumes favored by users into relevant clusters for better recommendations. We constructed a bipartite network from user reviews on the Persian retail platform "Atrafshan," with nodes representing users and perfumes, and edges formed by positive comments. This network was transformed into a Perfume Co-Preference Network, connecting perfumes liked by the same users. By applying community detection algorithms, we identified clusters based on shared preferences, enhancing our understanding of user sentiment in the fragrance market. To improve sentiment analysis, we integrated emojis and a user voting system for greater accuracy. Emojis, aligned with their Persian counterparts, captured the emotional tone of reviews, while user ratings for scent, longevity, and sillage refined sentiment classification. Edge weights were adjusted by combining adjacency values with user ratings in a 60:40 ratio, reflecting both connection strength and user preferences. These enhancements led to improved modularity of detected communities, resulting in more accurate perfume groupings. This research pioneers the use of community detection in perfume networks, offering new insights into consumer preferences. Our advancements in sentiment analysis and edge weight refinement provide actionable insights for optimizing product recommendations and marketing strategies in the fragrance industry.
MonoDGP: Monocular 3D Object Detection with Decoupled-Query and Geometry-Error Priors
Authors: Fanqi Pu, Yifan Wang, Jiru Deng, Wenming Yang
Date: 2024-10-25 00:00:00
Proceeding: CVPR 2025 1
Tasks: 3D Object Detection, Depth Estimation, Depth Prediction, Monocular 3D Object Detection, Object, object-detection, Object Detection

Perspective projection has been extensively utilized in monocular 3D object detection methods. It introduces geometric priors from 2D bounding boxes and 3D object dimensions to reduce the uncertainty of depth estimation. However, due to depth errors originating from the object's visual surface, the height of the bounding box often fails to represent the actual projected central height, which undermines the effectiveness of geometric depth. Direct prediction for the projected height unavoidably results in a loss of 2D priors, while multi-depth prediction with complex branches does not fully leverage geometric depth. This paper presents a Transformer-based monocular 3D object detection method called MonoDGP, which adopts perspective-invariant geometry errors to modify the projection formula. We also try to systematically discuss and explain the mechanisms and efficacy behind geometry errors, which serve as a simple but effective alternative to multi-depth prediction. Additionally, MonoDGP decouples the depth-guided decoder and constructs a 2D decoder only dependent on visual features, providing 2D priors and initializing object queries without the disturbance of 3D detection. To further optimize and fine-tune input tokens of the transformer decoder, we also introduce a Region Segment Head (RSH) that generates enhanced features and segment embeddings. Our monocular method demonstrates state-of-the-art performance on the KITTI benchmark without extra data. Code is available at https://github.com/PuFanqi23/MonoDGP.
Visual Text Matters: Improving Text-KVQA with Visual Text Entity Knowledge-aware Large Multimodal Assistant
Authors: Abhirama Subramanyam Penamakuri, Anand Mishra
Date: 2024-10-24 00:00:00
Tasks: Entity Linking, Question Answering, Visual Question Answering

We revisit knowledge-aware text-based visual question answering, also known as Text-KVQA, in the light of modern advancements in large multimodal models (LMMs), and make the following contributions: (i) We propose VisTEL - a principled approach to perform visual text entity linking. The proposed VisTEL module harnesses a state-of-the-art visual text recognition engine and the power of a large multimodal model to jointly reason using textual and visual context obtained using surrounding cues in the image to link the visual text entity to the correct knowledge base entity. (ii) We present KaLMA - a knowledge-aware large multimodal assistant that augments an LMM with knowledge associated with visual text entity in the image to arrive at an accurate answer. Further, we provide a comprehensive experimental analysis and comparison of our approach with traditional visual question answering, pre-large multimodal models, and large multimodal models, as well as prior top-performing approaches. Averaging over three splits of Text-KVQA, our proposed approach surpasses the previous best approach by a substantial 23.3% on an absolute scale and establishes a new state of the art. We make our implementation publicly available.
CHESTNUT: A QoS Dataset for Mobile Edge Environments
Authors: Guobing Zou, Fei Zhao, Shengxiang Hu
Date: 2024-10-25 00:00:00

Quality of Service (QoS) is an important metric to measure the performance of network services. Nowadays, it is widely used in mobile edge environments to evaluate the quality of service when mobile devices request services from edge servers. QoS usually involves multiple dimensions, such as bandwidth, latency, jitter, and data packet loss rate. However, most existing QoS datasets, such as the common WS-Dream dataset, focus mainly on static QoS metrics of network services and ignore dynamic attributes such as time and geographic location. This means they should have detailed the mobile device's location at the time of the service request or the chronological order in which the request was made. However, these dynamic attributes are crucial for understanding and predicting the actual performance of network services, as QoS performance typically fluctuates with time and geographic location. To this end, we propose a novel dataset that accurately records temporal and geographic location information on quality of service during the collection process, aiming to provide more accurate and reliable data to support future QoS prediction in mobile edge environments.