pwc-archive/papers-with-abstracts huggingface dataset:
Optimal Transport for Domain Adaptation through Gaussian Mixture Models
Authors: Eduardo Fernandes Montesuma, Fred Maurice Ngolè Mboula, Antoine Souloumiac
Date: 2024-03-18 00:00:00
Tasks: Domain Adaptation

Machine learning systems operate under the assumption that training and test data are sampled from a fixed probability distribution. However, this assumptions is rarely verified in practice, as the conditions upon which data was acquired are likely to change. In this context, the adaptation of the unsupervised domain requires minimal access to the data of the new conditions for learning models robust to changes in the data distribution. Optimal transport is a theoretically grounded tool for analyzing changes in distribution, especially as it allows the mapping between domains. However, these methods are usually computationally expensive as their complexity scales cubically with the number of samples. In this work, we explore optimal transport between Gaussian Mixture Models (GMMs), which is conveniently written in terms of the components of source and target GMMs. We experiment with 9 benchmarks, with a total of $85$ adaptation tasks, showing that our methods are more efficient than previous shallow domain adaptation methods, and they scale well with number of samples $n$ and dimensions $d$.
Graphs Unveiled: Graph Neural Networks and Graph Generation
Authors: László Kovács, Ali Jlidi
Date: 2024-03-18 00:00:00
Tasks: Graph Generation, Survey

One of the hot topics in machine learning is the field of GNN. The complexity of graph data has imposed significant challenges on existing machine learning algorithms. Recently, many studies on extending deep learning approaches for graph data have emerged. This paper represents a survey, providing a comprehensive overview of Graph Neural Networks (GNNs). We discuss the applications of graph neural networks across various domains. Finally, we present an advanced field in GNNs: graph generation.
Autonomous microARPES
Authors: Steinn Ymir Agustsson, Alfred J. H. Jones, Davide Curcio, Søren Ulstrup, Jill Miwa, Davide Mottin, Panagiotis Karras, Philip Hofmann
Date: 2024-02-16 00:00:00

Angle-resolved photoemission spectroscopy (ARPES) is a technique used to map the occupied electronic structure of solids. Recent progress in X-ray focusing optics has led to the development of ARPES into a microscopic tool, permitting the electronic structure to be spatially mapped across the surface of a sample. This comes at the expense of a time-consuming scanning process to cover not only a three-dimensional energy-momentum ($E, k_z, k_y$) space but also the two-dimensional surface area. Here, we implement a protocol to autonomously search both $\mathbf{k}$- and real space in order to find positions of particular interest, either because of their high photoemission intensity or because of sharp spectral features. The search is based on the use of Gaussian process regression and can easily be expanded to include additional parameters or optimisation criteria. This autonomous experimental control is implemented on the SGM4 micro-focus beamline of the synchrotron radiation source ASTRID2.
Self-Supervised Path Planning in UAV-aided Wireless Networks based on Active Inference
Authors: Ali Krayani, Khalid Khan, Lucio Marcenaro, Mario Marchese, Carlo Regazzoni
Date: 2024-03-06 00:00:00

This paper presents a novel self-supervised path-planning method for UAV-aided networks. First, we employed an optimizer to solve training examples offline and then used the resulting solutions as demonstrations from which the UAV can learn the world model to understand the environment and implicitly discover the optimizer's policy. UAV equipped with the world model can make real-time autonomous decisions and engage in online planning using active inference. During planning, UAV can score different policies based on the expected surprise, allowing it to choose among alternative futures. Additionally, UAV can anticipate the outcomes of its actions using the world model and assess the expected surprise in a self-supervised manner. Our method enables quicker adaptation to new situations and better performance than traditional RL, leading to broader generalizability.
Predicting Confinement Effect of Carbon Fiber Reinforced Polymers on Strength of Concrete using Metaheuristics-based Artificial Neural Networks
Authors: Sarmed Wahab, Mohamed Suleiman, Faisal Shabbir, Nasim Shakouri Mahmoudabadi, Sarmad Waqas, Nouman Herl, Afaq Ahmad
Date: 2023-12-22 00:00:00

This article deals with the study of predicting the confinement effect of carbon fiber reinforced polymers (CFRPs) on concrete cylinder strength using metaheuristics-based artificial neural networks. A detailed database of 708 CFRP confined concrete cylinders is developed from previously published research with information on 8 parameters including geometrical parameters like the diameter (d) and height (h) of a cylinder, unconfined compressive strength of concrete (fco'), thickness (nt), the elastic modulus of CFRP (Ef), unconfined concrete strain confined concrete strain and the ultimate compressive strength of confined concrete fcc'. Three metaheuristic models are implemented including particle swarm optimization (PSO), grey wolf optimizer (GWO), and bat algorithm (BA). These algorithms are trained on the data using an objective function of mean square error and their predicted results are validated against the experimental studies and finite element analysis. The study shows that the hybrid model of PSO predicted the strength of CFRP-confined concrete cylinders with maximum accuracy of 99.13% and GWO predicted the results with an accuracy of 98.17%. The high accuracy of axial compressive strength predictions demonstrated that these prediction models are a reliable solution to the empirical methods. The prediction models are especially suitable for avoiding full-scale time-consuming experimental tests that make the process quick and economical.
A Model Ensemble Approach with LLM for Chinese Text Classification
Authors: Chengyan Wu, Wenlong Fang, Feipeng Dai, Hailong Yin
Date: 2024-03-22 00:00:00
Proceeding: China Health Information Processing Conference, 2023 2024 3
Tasks: Diagnostic, Few-shot NER, Text Categorization, text-classification, Text Classification

Automatic medical text categorization can assist doctors in efficiently managing patient information. By categorizing textual information such as patients’ descriptions of symptoms, doctors can easily find key information, accelerate the diagnostic process, provide superior medical advice, and successfully promote smart diagnosis and medical automated QA services. In this paper, an approach to medical text categorization is presented in the open-share task of the 9th China Conference on Health Information Processing (CHIP 2023), where complex textual relations are the two main challenges of this task. A model integration approach is proposed for this task, which can effectively solve medical text categorization through the complementary relationship of three different submodels. In addition, the solution provides external tools for targeted data enhancement for difficult samples that are hard to classify to reduce misclassification. Final results are obtained by the models through a voting mechanism. Experimental results show that the proposed method can achieve 92% accuracy and also prove the effectiveness of the model.
Beyond Inference: Performance Analysis of DNN Server Overheads for Computer Vision
Authors: Ahmed F. AbouElhamayed, Susanne Balle, Deshanand Singh, Mohamed S. Abdelfattah
Date: 2024-03-02 00:00:00
Tasks: Deep Learning, Depth Estimation, image-classification, Image Classification

Deep neural network (DNN) inference has become an important part of many data-center workloads. This has prompted focused efforts to design ever-faster deep learning accelerators such as GPUs and TPUs. However, an end-to-end DNN-based vision application contains more than just DNN inference, including input decompression, resizing, sampling, normalization, and data transfer. In this paper, we perform a thorough evaluation of computer vision inference requests performed on a throughput-optimized serving system. We quantify the performance impact of server overheads such as data movement, preprocessing, and message brokers between two DNNs producing outputs at different rates. Our empirical analysis encompasses many computer vision tasks including image classification, segmentation, detection, depth-estimation, and more complex processing pipelines with multiple DNNs. Our results consistently demonstrate that end-to-end application performance can easily be dominated by data processing and data movement functions (up to 56% of end-to-end latency in a medium-sized image, and $\sim$ 80% impact on system throughput in a large image), even though these functions have been conventionally overlooked in deep learning system design. Our work identifies important performance bottlenecks in different application scenarios, achieves 2.25$\times$ better throughput compared to prior work, and paves the way for more holistic deep learning system design.
MapSeg: Segmentation guided structured model for online HD map construction
Authors: Mingchao Jiang, Yin Cheng, Linghai Liu
Date: 2023-06-13 00:00:00
Tasks: Autonomous Driving, BEV Segmentation, Segmentation

The development of online high-definition maps is significant since they provide real-time, accurate, and updatable geographic information for location-based applications, such as autonomous driving and intelligent transportation, thus improving the performance and reliability of these applications. Previous works, such as VectorMapNet and MapTR, show that direct model generation of vectorized HD maps is a promising solution. However, these methods did not take into account the usage of global semantic information to improve map construction accuracy. To address this limitation, we propose a segmentation-guided structured model (MapSeg) for online HD map construction. Specifically, we added a UV segmentation module (USM) and a BEV segmentation module (BSM) based on the MapTR structure, enabling the model to better capture the semantic information.
Advancing Multilingual Handwritten Numeral Recognition with Attention-driven Transfer Learning
Authors: Amirreza Fateh, Reza Tahmasbi Birgani, Mansoor Fateh, Vahid Abolghasemi
Date: 2024-03-18 00:00:00
Proceeding: IEEEXplore 2024 3
Tasks: Handwritten Digit Recognition, Optical Character Recognition, Optical Character Recognition (OCR), Transfer Learning

As deep learning continues to evolve, we have observed huge breakthroughs in the fields of medical imaging, video and frame generation, optical character recognition (OCR), and other domains. In the field of data analysis and document processing, the recognition of handwritten numerals plays a crucial role. This work has led to remarkable changes in OCR, historical handwritten document analysis, and postal automation. In this study, we present a novel framework to overcome this challenge, going beyond digit recognition in only one language. Unlike common methods that focus on a limited set of languages, our method provides a comprehensive solution for recognition of handwritten digit images in 12 different languages. These specific languages are chosen because most of them have fairly distant representations in latent space. We utilize transfer learning, as it reduces the computational cost and maintains the quality of enhanced images and the models’ recognition accuracy. Another strength of our approach is the innovative attention-based module called the MRA module. Our experiments confirm that by applying this module, major progress is made in both image quality and the accuracy of handwritten digit recognition. Notably, we reached high precisions, surpassing nearly 2% improvement in specific languages compared to earlier techniques. In this work, we present a robust and cost-effective approach that handles multilingual handwritten numeral recognition across a wide range of languages. The code and further implementation details are available at https://github.com/CVLab-SHUT/HandWrittenDigitRecognition.
FABSA: An aspect-based sentiment analysis dataset of user reviews
Authors: Georgios Kontonatsios, Jordan Clive, Georgia Harrison, Thomas Metcalfe, Patrycja Sliwiak, Hassan Tahir, Aji Ghose
Date: 2023-12-28 00:00:00
Proceeding: Neurocomputing 2023 12
Tasks: Aspect-Based Sentiment Analysis, Aspect-Based Sentiment Analysis (ABSA), Aspect Category Detection, Aspect Category Polarity, Aspect Category Sentiment Analysis, Aspect Category Sentiment Classification, Sentiment Analysis

Aspect-based sentiment analysis (ABSA) aims at automatically extracting aspects of entities and classifying the polarity of each extracted aspect. The majority of available ABSA systems heavily rely on manually annotated datasets to train supervised machine learning models. However, the development of such manually curated datasets is a labour-intensive process and therefore existing ABSA datasets cover only a few domains and they are limited in size. In response, we present FABSA (Feedback ABSA), a new large-scale and multi-domain ABSA dataset of feedback reviews. FABSA consists of approximately 10,500 reviews which span across 10 domains. We conduct a number of experiments to evaluate the performance of state-of-the-art deep learning models when applied to the FABSA dataset. Our results demonstrate that ABSA models can generalise across different domains when trained on our FABSA dataset while the performance of the models is enhanced when using a larger training dataset. Our FABSA dataset is publicly available.
Alleviating Exposure Bias via Multi-level Contrastive Learning and Deviation Simulation in Abstractive Summarization
Authors: Jiawen Xie, Qi Su, Shaoting Zhang, and Xiaofan Zhang
Date: 2023-05-03 00:00:00
Proceeding: ACL 2023 5
Tasks: Abstractive Text Summarization, Contrastive Learning, Decoder, Few-Shot Learning

Most Transformer based abstractive summarization systems have a severe mismatch between training and inference, i.e., exposure bias. From diverse perspectives, we introduce a simple multi-level contrastive learning framework for abstractive summarization (SimMCS) and a tailored sparse decoder self-attention pattern (SDSA) to bridge the gap between training and inference to improve model performance. Compared with previous contrastive objectives focusing only on the relative order of probability mass assigned to non-gold summaries, SimMCS additionally takes their absolute positions into account, which guarantees that the relatively high-quality (positive) summaries among them could be properly assigned high probability mass, and further enhances the capability of discriminating summary quality beyond exploiting potential artifacts of specific metrics. SDSA simulates the possible inference scenarios of deviation in the training phase to get closer to the ideal paradigm. Our approaches outperform the previous state-of-the-art results on two summarization datasets while just adding fairly low overhead. Further empirical analysis shows our model preserves the advantages of prior contrastive methods and possesses strong few-shot learning ability.
Investigation of Genomic Effect of Zirconium Oxide Nanoparticles in Escherichia coli Bacteria
Authors: Simin Rashidi, Bahram Golestani Eimani
Date: 2024-03-21 00:00:00

Due to the concerns of the society about the increase of antibiotic resistant infections, many studies and research have been done on nanoparticles and applications of nano-biotechnology. Zirconium Oxide ($\text{ZrO}_{2}$) in which called zirconia, is a white oxide of zirconium metal that its diameter is 20 nm. The colloidal size of these particles is often smaller than bacterial and eukaryotic cells. The main intention of this paper is to investigate the effect of different doses of $\text{ZrO}_{2}$ NPs on the sequences changes for the $\textit{Escherichia coli}$ ($\textit{E. coli}$) genome. At the first step, $\textit{E. coli}$ was cultured in eosin methylene blue agar and brain heart broth (BHB) mediums, respectively. Then, bacteria were treated with $\text{ZrO}_{2}$ NPs at concentrations of 100, 250, and 350 $\mu$g/ml. After treatment, the growth of bacteria was evaluated by utilizing spectrophotometry at 600 nm after incubation times including 2, 4, 6, 8, and 24 hours at 37 $^{\circ}$C. At the second step, the extraction of DNA was performed by using control and treated samples. Then, the changes in the sequence of bacterial genome were investigated using RAPD markers. Finally, NTSYS-PC platform was employed in order to analyze of the results extracted by electrophoresis of products on agarose gel. In this paper, it was observed that $\text{ZrO}_{2}$ NPs can inhibit the growth of bacteria at concentrations of 250 and 350 $\mu$g/ml after 8 hours of treatment. It was also found that the $\text{ZrO}_{2}$ NPs at different concentrations have not changed the genome sequence of $\textit{E. coli}$. Furthermore, it was concluded that the $\text{ZrO}_{2}$ NPs with the concentration of 350 $\mu$g/ml had the highest inhibitory properties without significant changing in the genomic sequence of $\textit{E. coli}$.
AVT2-DWF: Improving Deepfake Detection with Audio-Visual Fusion and Dynamic Weighting Strategies
Authors: Rui Wang, Dengpan Ye, Long Tang, Yunming Zhang, Jiacheng Deng
Date: 2024-03-22 00:00:00
Tasks: DeepFake Detection, Face Swapping

With the continuous improvements of deepfake methods, forgery messages have transitioned from single-modality to multi-modal fusion, posing new challenges for existing forgery detection algorithms. In this paper, we propose AVT2-DWF, the Audio-Visual dual Transformers grounded in Dynamic Weight Fusion, which aims to amplify both intra- and cross-modal forgery cues, thereby enhancing detection capabilities. AVT2-DWF adopts a dual-stage approach to capture both spatial characteristics and temporal dynamics of facial expressions. This is achieved through a face transformer with an n-frame-wise tokenization strategy encoder and an audio transformer encoder. Subsequently, it uses multi-modal conversion with dynamic weight fusion to address the challenge of heterogeneous information fusion between audio and visual modalities. Experiments on DeepfakeTIMIT, FakeAVCeleb, and DFDC datasets indicate that AVT2-DWF achieves state-of-the-art performance intra- and cross-dataset Deepfake detection. Code is available at https://github.com/raining-dev/AVT2-DWF.
Towards a Framework for Deep Learning Certification in Safety-Critical Applications Using Inherently Safe Design and Run-Time Error Detection
Authors: Romeo Valentin
Date: 2024-03-12 00:00:00
Tasks: Deep Learning, Out-of-Distribution Detection, Representation Learning, State Estimation, Uncertainty Quantification

Although an ever-growing number of applications employ deep learning based systems for prediction, decision-making, or state estimation, almost no certification processes have been established that would allow such systems to be deployed in safety-critical applications. In this work we consider real-world problems arising in aviation and other safety-critical areas, and investigate their requirements for a certified model. To this end, we investigate methodologies from the machine learning research community aimed towards verifying robustness and reliability of deep learning systems, and evaluate these methodologies with regard to their applicability to real-world problems. Then, we establish a new framework towards deep learning certification based on (i) inherently safe design, and (ii) run-time error detection. Using a concrete use case from aviation, we show how deep learning models can recover disentangled variables through the use of weakly-supervised representation learning. We argue that such a system design is inherently less prone to common model failures, and can be verified to encode underlying mechanisms governing the data. Then, we investigate four techniques related to the run-time safety of a model, namely (i) uncertainty quantification, (ii) out-of-distribution detection, (iii) feature collapse, and (iv) adversarial attacks. We evaluate each for their applicability and formulate a set of desiderata that a certified model should fulfill. Finally, we propose a novel model structure that exhibits all desired properties discussed in this work, and is able to make regression and uncertainty predictions, as well as detect out-of-distribution inputs, while requiring no regression labels to train. We conclude with a discussion of the current state and expected future progress of deep learning certification, and its industrial and social implications.
Continual Learning by Three-Phase Consolidation
Authors: Davide Maltoni, Lorenzo Pellegrini
Date: 2024-03-12 00:00:00
Tasks: Continual Learning

TPC (Three-Phase Consolidation) is here introduced as a simple but effective approach to continually learn new classes (and/or instances of known classes) while controlling forgetting of previous knowledge. Each experience (a.k.a. task) is learned in three phases characterized by different rules and learning dynamics, aimed at removing the class-bias problem (due to class unbalancing) and limiting gradient-based corrections to prevent forgetting of underrepresented classes. Several experiments on complex datasets demonstrate its accuracy and efficiency advantages over competitive existing approaches. The algorithm and all the results presented in this paper are fully reproducible thanks to its publication on the Avalanche open framework for continual learning.
Cyclical Log Annealing as a Learning Rate Scheduler
Authors: Philip Naveen
Date: 2024-03-13 00:00:00

A learning rate scheduler is a predefined set of instructions for varying search stepsizes during model training processes. This paper introduces a new logarithmic method using harsh restarting of step sizes through stochastic gradient descent. Cyclical log annealing implements the restart pattern more aggressively to maybe allow the usage of more greedy algorithms on the online convex optimization framework. The algorithm was tested on the CIFAR-10 image datasets, and seemed to perform analogously with cosine annealing on large transformer-enhanced residual neural networks. Future experiments would involve testing the scheduler in generative adversarial networks and finding the best parameters for the scheduler with more experiments.
On the Performance of Imputation Techniques for Missing Values on Healthcare Datasets
Authors: Luke Oluwaseye Joel, Wesley Doorsamy, Babu Sena Paul
Date: 2024-03-13 00:00:00
Tasks: feature selection, Imputation, Missing Values

Missing values or data is one popular characteristic of real-world datasets, especially healthcare data. This could be frustrating when using machine learning algorithms on such datasets, simply because most machine learning models perform poorly in the presence of missing values. The aim of this study is to compare the performance of seven imputation techniques, namely Mean imputation, Median Imputation, Last Observation carried Forward (LOCF) imputation, K-Nearest Neighbor (KNN) imputation, Interpolation imputation, Missforest imputation, and Multiple imputation by Chained Equations (MICE), on three healthcare datasets. Some percentage of missing values - 10\%, 15\%, 20\% and 25\% - were introduced into the dataset, and the imputation techniques were employed to impute these missing values. The comparison of their performance was evaluated by using root mean squared error (RMSE) and mean absolute error (MAE). The results show that Missforest imputation performs the best followed by MICE imputation. Additionally, we try to determine whether it is better to perform feature selection before imputation or vice versa by using the following metrics - the recall, precision, f1-score and accuracy. Due to the fact that there are few literature on this and some debate on the subject among researchers, we hope that the results from this experiment will encourage data scientists and researchers to perform imputation first before feature selection when dealing with data containing missing values.