pwc-archive/papers-with-abstracts huggingface dataset:
Anomaly Detection and Inlet Pressure Prediction in Water Distribution Systems Using Machine Learning
Authors: Tran Dang Khoa
Date: 2024-10-12 00:00:00
Tasks: Anomaly Detection, Management

This study presents two models to optimize pressure management in water distribution networks. The first model forecasts pressure at distribution points and compares predictions with actual data to detect anomalies such as leaks and blockages. Early detection allows for timely interventions, minimizing economic losses and ensuring system sustainability. The second model estimates the necessary inlet pressure based on the influence of various distribution points, ensuring consistent water supply while reducing waste and optimizing resource management. Both models utilize modern machine learning algorithms to enhance the prediction process. The methodology includes the CNN-EMD model, which analyzes historical data collected every 15 minutes over two months to predict future pressures. The Empirical Mode Decomposition (EMD) method identifies fluctuations and anomalies, improving prediction accuracy. The second model combines CNN, EMD, and LSTM techniques to forecast required inlet pressure, emphasizing the impact of distribution points. Results show that the CNN-EMD and CNN-EMD-LSTM models enhance pressure management capabilities, with the first model achieving an anomaly detection accuracy of 85% to 95% and the second model predicting inlet pressure with an average accuracy of 93%. This enables flexible system adjustments and identifies critical factors affecting inlet pressure. In conclusion, advanced machine learning models like CNN-EMD and LSTM significantly improve pressure management in water distribution networks, facilitating early issue identification, ensuring efficient water supply, and optimizing resource management for future generations.
Investigating the Impact of Text Summarization on Topic Modeling
Authors: Trishia Khandelwal
Date: 2024-09-28 00:00:00
Tasks: Diversity, Language Modeling, Language Modelling, Large Language Model, Text Summarization, Topic Models

Topic models are used to identify and group similar themes in a set of documents. Recent advancements in deep learning based neural topic models has received significant research interest. In this paper, an approach is proposed that further enhances topic modeling performance by utilizing a pre-trained large language model (LLM) to generate summaries of documents before inputting them into the topic model. Few shot prompting is used to generate summaries of different lengths to compare their impact on topic modeling. This approach is particularly effective for larger documents because it helps capture the most essential information while reducing noise and irrelevant details that could obscure the overall theme. Additionally, it is observed that datasets exhibit an optimal summary length that leads to improved topic modeling performance. The proposed method yields better topic diversity and comparable coherence values compared to previous models.
Fish-TViT: A novel fish species classification method in multi water areas based on transfer learning and vision transformer
Authors: Bo Gong, Kanyuan Dai, Ji Shao, Ling Jing, Yingyi Chen
Date: 2023-05-31 00:00:00
Proceeding: HELIYON 2023 5
Tasks: Data Augmentation, Transfer Learning

The classification of fish species has important practical significance for both the aquaculture industry and ordinary people. However, existing methods for classifying marine and freshwater fishes have poor feature extraction ability and do not meet actual needs. To address this issue, we propose a novel method for multi-water fish classification (Fish-TViT) based on transfer learning and visual transformers. Fish-TViT uses a label smoothing loss function to solve the problem of overfitting and overconfidence of the classifier. We also employ Gradient-weighted Category Activation Mapping (Grad-CAM) technology to visualize and understand the features of the model and the areas on which the decision depends, which guides the optimization of the model architecture. We first crop and clean fish images, and then use data augmentation to expand the number of training datasets. A pre-trained visual transformer model is used to extract enhanced features of fish images, which are subsequently cropped into a series of flat patches. Finally, a multi-layer perceptron is used to predict fish species. Experimental results show that Fish-TViT achieves high classification accuracy on both low-resolution marine fish data (94.33%) and high-resolution freshwater fish data (98.34%). Compared with traditional convolutional neural networks, Fish-TViT has better performance.
Efficacy of Large Language Models in Systematic Reviews
Authors: Aaditya Shah, Shridhar Mehendale, Siddha Kanthi
Date: 2024-08-03 00:00:00

This study investigates the effectiveness of Large Language Models (LLMs) in interpreting existing literature through a systematic review of the relationship between Environmental, Social, and Governance (ESG) factors and financial performance. The primary objective is to assess how LLMs can replicate a systematic review on a corpus of ESG-focused papers. We compiled and hand-coded a database of 88 relevant papers published from March 2020 to May 2024. Additionally, we used a set of 238 papers from a previous systematic review of ESG literature from January 2015 to February 2020. We evaluated two current state-of-the-art LLMs, Meta AI's Llama 3 8B and OpenAI's GPT-4o, on the accuracy of their interpretations relative to human-made classifications on both sets of papers. We then compared these results to a "Custom GPT" and a fine-tuned GPT-4o Mini model using the corpus of 238 papers as training data. The fine-tuned GPT-4o Mini model outperformed the base LLMs by 28.3% on average in overall accuracy on prompt 1. At the same time, the "Custom GPT" showed a 3.0% and 15.7% improvement on average in overall accuracy on prompts 2 and 3, respectively. Our findings reveal promising results for investors and agencies to leverage LLMs to summarize complex evidence related to ESG investing, thereby enabling quicker decision-making and a more efficient market.
‘Unexpected item in the bagging area’: Anomaly Detection in X-ray Security Images
Authors: Lewis D. Griffin, Matthew Caldwell, Jerone T. A. Andrews, Helene Bohler
Date: 2018-11-16 00:00:00
Proceeding: IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 2018 11
Tasks: Anomaly Detection

The role of Anomaly Detection in X-ray security
imaging, as a supplement to targeted threat detection, is described;
and a taxonomy of anomalies types in this domain is presented.
Algorithms are described for detecting appearance anomalies, of
shape, texture and density; and semantic anomalies of object
category presence. The anomalies are detected on the basis of
representations extracted from a convolutional neural network
pre-trained to identify object categories in photographs: from the
final pooling layer for appearance anomalies, and from the logit
layer for semantic anomalies. The distribution of representations
in normal data are modelled using high-dimensional, full-
covariance, Gaussians; and anomalies are scored according to
their likelihood relative to those models. The algorithms are tested
on X-ray parcel images using stream-of-commerce data as the
normal class, and parcels with firearms present as examples of
anomalies to be detected. Despite the representations being learnt
for photographic images, and the varied contents of stream-of-
commerce parcels; the system, trained on stream-of-commerce
images only, is able to detect 90% of firearms as anomalies, while
raising false alarms on 18% of stream-of-commerce.
Are High-Degree Representations Really Unnecessary in Equivariant Graph Neural Networks?
Authors: Jiacheng Cen, Anyi Li, Ning Lin, Yuxiang Ren, Zihe Wang, Wenbing Huang
Date: 2024-10-15 00:00:00

Equivariant Graph Neural Networks (GNNs) that incorporate E(3) symmetry have achieved significant success in various scientific applications. As one of the most successful models, EGNN leverages a simple scalarization technique to perform equivariant message passing over only Cartesian vectors (i.e., 1st-degree steerable vectors), enjoying greater efficiency and efficacy compared to equivariant GNNs using higher-degree steerable vectors. This success suggests that higher-degree representations might be unnecessary. In this paper, we disprove this hypothesis by exploring the expressivity of equivariant GNNs on symmetric structures, including $k$-fold rotations and regular polyhedra. We theoretically demonstrate that equivariant GNNs will always degenerate to a zero function if the degree of the output representations is fixed to 1 or other specific values. Based on this theoretical insight, we propose HEGNN, a high-degree version of EGNN to increase the expressivity by incorporating high-degree steerable vectors while maintaining EGNN's efficiency through the scalarization trick. Our extensive experiments demonstrate that HEGNN not only aligns with our theoretical analyses on toy datasets consisting of symmetric structures, but also shows substantial improvements on more complicated datasets such as $N$-body and MD17. Our theoretical findings and empirical results potentially open up new possibilities for the research of equivariant GNNs.
Biologically Inspired Swarm Dynamic Target Tracking and Obstacle Avoidance
Authors: Lucas Page
Date: 2024-10-15 00:00:00
Tasks: Prediction, Trajectory Prediction

This study proposes a novel artificial intelligence (AI) driven flight computer, integrating an online free-retraining-prediction model, a swarm control, and an obstacle avoidance strategy, to track dynamic targets using a distributed drone swarm for military applications. To enable dynamic target tracking the swarm requires a trajectory prediction capability to achieve intercept allowing for the tracking of rapid maneuvers and movements while maintaining efficient path planning. Traditional predicative methods such as curve fitting or Long ShortTerm Memory (LSTM) have low robustness and struggle with dynamic target tracking in the short term due to slow convergence of single agent-based trajectory prediction and often require extensive offline training or tuning to be effective. Consequently, this paper introduces a novel robust adaptive bidirectional fuzzy brain emotional learning prediction (BFBEL-P) methodology to address these challenges. The controller integrates a fuzzy interface, a neural network enabling rapid adaption, predictive capability and multi-agent solving enabling multiple solutions to be aggregated to achieve rapid convergence times and high accuracy in both the short and long term. This was verified through the use of numerical simulations seeing complex trajectory being predicted and tracked by a swarm of drones. These simulations show improved adaptability and accuracy to state of the art methods in the short term and strong results over long time domains, enabling accurate swarm target tracking and predictive capability.
IntGrad MT: Eliciting LLMs' Machine Translation Capabilities with Sentence Interpolation and Gradual MT
Authors: Seung-Woo Choi, Ga-Hyun Yoo, Jay-Yoon Lee
Date: 2024-10-15 00:00:00
Tasks: Machine Translation, Sentence, Translation

Recent Large Language Models (LLMs) have demonstrated strong performance in translation without needing to be finetuned on additional parallel corpora. However, they still underperform for low-resource language pairs. Previous works have focused on mitigating this issue by leveraging relevant few-shot examples or external resources such as dictionaries or grammar books, making models heavily reliant on these nonparametric sources of information. In this paper, we propose a novel method named IntGrad MT that focuses on fully exploiting an LLM's inherent translation capability. IntGrad MT achieves this by constructing a chain of few-shot examples, each consisting of a source sentence and the model's own translation, that rise incrementally in difficulty. IntGrad MT employs two techniques: Sentence Interpolation, which generates a sequence of sentences that gradually change from an easy sentence to translate to a difficult one, and Gradual MT, which sequentially translates this chain using translations of earlier sentences as few-shot examples for the translation of subsequent ones. With this approach, we observe a substantial enhancement in the xCOMET scores of various LLMs for multiple languages, especially in low-resource languages such as Hindi(8.26), Swahili(7.10), Bengali(6.97) and Marathi(13.03). Our approach presents a practical way of enhancing LLMs' performance without extra training.
Test Case-Informed Knowledge Tracing for Open-ended Coding Tasks
Authors: Zhangqi Duan, Nigel Fernandez, Alexander Hicks, Andrew Lan
Date: 2024-09-28 00:00:00
Tasks: Knowledge Tracing, Language Modeling, Language Modelling, Large Language Model, Multi-Task Learning

Open-ended coding tasks, which ask students to construct programs according to certain specifications, are common in computer science education. Student modeling can be challenging since their open-ended nature means that student code can be diverse. Traditional knowledge tracing (KT) models that only analyze response correctness may not fully capture nuances in student knowledge from student code. In this paper, we introduce Test case-Informed Knowledge Tracing for Open-ended Coding (TIKTOC), a framework to simultaneously analyze and predict both open-ended student code and whether the code passes each test case. We augment the existing CodeWorkout dataset with the test cases used for a subset of the open-ended coding questions, and propose a multi-task learning KT method to simultaneously analyze and predict 1) whether a student's code submission passes each test case and 2) the student's open-ended code, using a large language model as the backbone. We quantitatively show that these methods outperform existing KT methods for coding that only use the overall score a code submission receives. We also qualitatively demonstrate how test case information, combined with open-ended code, helps us gain fine-grained insights into student knowledge.
SlideChat: A Large Vision-Language Assistant for Whole-Slide Pathology Image Understanding
Authors: Ying Chen, Guoan Wang, Yuanfeng Ji, Yanjun Li, Jin Ye, Tianbin Li, Bin Zhang, Nana Pei, Rongshan Yu, Yu Qiao, Junjun He
Date: 2024-10-15 00:00:00
Proceeding: CVPR 2025 1
Tasks: Instruction Following, Visual Question Answering (VQA), whole slide images

Despite the progress made by multimodal large language models (MLLMs) in computational pathology, they remain limited by a predominant focus on patch-level analysis, missing essential contextual information at the whole-slide level. The lack of large-scale instruction datasets and the gigapixel scale of whole slide images (WSIs) pose significant developmental challenges. In this paper, we present SlideChat, the first vision-language assistant capable of understanding gigapixel whole-slide images, exhibiting excellent multimodal conversational capability and response complex instruction across diverse pathology scenarios. To support its development, we created SlideInstruction, the largest instruction-following dataset for WSIs consisting of 4.2K WSI captions and 176K VQA pairs with multiple categories. Furthermore, we propose SlideBench, a multimodal benchmark that incorporates captioning and VQA tasks to assess SlideChat's capabilities in varied clinical settings such as microscopy, diagnosis. Compared to both general and specialized MLLMs, SlideChat exhibits exceptional capabilities achieving state-of-the-art performance on 18 of 22 tasks. For example, it achieved an overall accuracy of 81.17% on SlideBench-VQA (TCGA), and 54.15% on SlideBench-VQA (BCNB). We will fully release SlideChat, SlideInstruction and SlideBench as open-source resources to facilitate research and development in computational pathology.
Deep Hybrid Architecture for Very Low-Resolution Image Classification Using Capsule Attention
Authors: Hasindu Dewasurendra, Taejoon Kim
Date: 2024-09-27 00:00:00
Proceeding: IEEE Access 2024 9
Tasks: image-classification, Image Classification, Transfer Learning

Despite extensive applications in surveillance and remote sensing, research on very low-resolution (VLR) image classification remains relatively unexplored in comparison to high-resolution (HR) image classification. We introduce a deep hybrid network that integrates capsule routing networks with a two-layer attention module. In the proposed architecture, the attention mechanism captures the more salient features, and the capsule network encodes these features to be robust to resolution changes. To enhance the network’s performance, a transfer learning on a custom image dataset, which is well-aligned to CIFAR-10, is utilized. The proposed model (Codes for the models are available at: https://github.com/kdhasi/Deep-CapsuleAttention.git ) is evaluated on two VLR classification tasks of ‘VLR complex image’ and ‘VLR real-world digit’. Experimental results demonstrate the superiority of the proposed model, achieving state-of-the-art (SOTA) results in both VLR complex image and VLR real-world digit domains while using fewer parameters compared to previous SOTA networks. Specifically, on the VLR CIFAR-10 dataset, the proposed model attains a 3.17% improvement in detection accuracy over the current benchmarks, and, on the VLR SVHN dataset, it achieves a 3.85% improvement by using 80% fewer parameters.
Sorted Weight Sectioning for Energy-Efficient Unstructured Sparse DNNs on Compute-in-Memory Crossbars
Authors: Matheus Farias, H. T. Kung
Date: 2024-10-15 00:00:00

We introduce $\textit{sorted weight sectioning}$ (SWS): a weight allocation algorithm that places sorted deep neural network (DNN) weight sections on bit-sliced compute-in-memory (CIM) crossbars to reduce analog-to-digital converter (ADC) energy consumption. Data conversions are the most energy-intensive process in crossbar operation. SWS effectively reduces this cost leveraging (1) small weights and (2) zero weights (weight sparsity). DNN weights follow bell-shaped distributions, with most weights near zero. Using SWS, we only need low-order crossbar columns for sections with low-magnitude weights. This reduces the quantity and resolution of ADCs used, exponentially decreasing ADC energy costs without significantly degrading DNN accuracy. Unstructured sparsification further sharpens the weight distribution with small accuracy loss. However, it presents challenges in hardware tracking of zeros: we cannot switch zero rows to other layer weights in unsorted crossbars without index matching. SWS efficiently addresses unstructured sparse models using offline remapping of zeros into earlier sections, which reveals full sparsity potential and maximizes energy efficiency. Our method reduces ADC energy use by 89.5% on unstructured sparse BERT models. Overall, this paper introduces a novel algorithm to promote energy-efficient CIM crossbars for unstructured sparse DNN workloads.
Multi-Objective Multidisciplinary Optimization of Wave Energy Converter Array Layout and Controls
Authors: Kapil Khanal, Nate DeGoede, Olivia Vitale, Maha N. Haji
Date: 2024-10-14 00:00:00
Tasks: Maximum Separation

This study utilizes multidisciplinary design optimization (MDO) to design an array of heaving wave energy converters (WECs) for grid-scale energy production with decision variables and parameters chosen from the coupled disciplines of geometry, hydrodynamics, layout, motor-actuated reactive controls (with a force maximum constraint) and economics. We vary a WEC's dimensions, array layout, and control gain to minimize two objectives: the levelized cost of energy (LCOE) and the maximum separation distance. This multi-objective optimization approach results in a set of optimal design configurations that stakeholders can choose from for their specific application and needs. The framework yields a range of optimal (minimum) LCOE values from 0.21 to 0.23 \$/kWh and a separation distance ranging from 97 to 62 meters. The WEC radius of 4m is found to be optimal, and the q-factor for optimal designs are greater than 1 up to 1.06 for a rhombus-like layout. Additionally, a post-optimality global sensitivity analysis of a design shows that wave heading, wave frequency, WEC lifetime, amplitude and interest rate accounts for most of the variance. Different designs in the Pareto set may be appealing for different decision makers based on their trade-off analysis. To that end, regression model is developed for design heuristics.
CVCP-Fusion: On Implicit Depth Estimation for 3D Bounding Box Prediction
Authors: Pranav Gupta, Rishabh Rengarajan, Viren Bankapur, Vedansh Mannem, Lakshit Ahuja, Surya Vijay, Kevin Wang
Date: 2024-10-15 00:00:00
Tasks: 3D Object Detection, Depth Estimation, object-detection, Object Detection

Combining LiDAR and Camera-view data has become a common approach for 3D Object Detection. However, previous approaches combine the two input streams at a point-level, throwing away semantic information derived from camera features. In this paper we propose Cross-View Center Point-Fusion, a state-of-the-art model to perform 3D object detection by combining camera and LiDAR-derived features in the BEV space to preserve semantic density from the camera stream while incorporating spacial data from the LiDAR stream. Our architecture utilizes aspects from previously established algorithms, Cross-View Transformers and CenterPoint, and runs their backbones in parallel, allowing efficient computation for real-time processing and application. In this paper we find that while an implicitly calculated depth-estimate may be sufficiently accurate in a 2D map-view representation, explicitly calculated geometric and spacial information is needed for precise bounding box prediction in the 3D world-view space.
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
Authors: Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, Huajun Chen
Date: 2024-10-15 00:00:00
Tasks: Hallucination, Language Modeling, Language Modelling

Multimodal Large Language Models (MLLMs) frequently exhibit hallucination phenomena, but the underlying reasons remain poorly understood. In this paper, we present an empirical analysis and find that, although MLLMs incorrectly generate the objects in the final output, they are actually able to recognize visual objects in the preceding layers. We speculate that this may be due to the strong knowledge priors of the language model suppressing the visual information, leading to hallucinations. Motivated by this, we propose a novel dynamic correction decoding method for MLLMs DeCo, which adaptively selects the appropriate preceding layers and proportionally integrates knowledge into the final layer to adjust the output logits. Note that DeCo is model agnostic and can be seamlessly incorporated with various classic decoding strategies and applied to different MLLMs. We evaluate DeCo on widely-used benchmarks, demonstrating that it can reduce hallucination rates by a large margin compared to baselines, highlighting its potential to mitigate hallucinations. Code is available at https://github.com/zjunlp/DeCo.