pwc-archive/papers-with-abstracts huggingface dataset:
Is computational creativity flourishing on the dead internet?
Authors: Terence Broad
Date: 2024-06-03 00:00:00

The dead internet theory is a conspiracy theory that states that all interactions and posts on social media are no longer being made by real people, but rather by autonomous bots. While the theory is obviously not true, an increasing amount of posts on social media have been made by bots optimised to gain followers and drive engagement on social media platforms. This paper looks at the recent phenomenon of these bots, analysing their behaviour through the lens of computational creativity to investigate the question: is computational creativity flourishing on the dead internet?
Unified Prediction Model for Employability in Indian Higher Education System
Authors: Pooja Thakar, Anil Mehta, Manisha
Date: 2024-06-05 00:00:00
Tasks: Prediction

Educational Data Mining has become extremely popular among researchers in last decade. Prior effort in this area was only directed towards prediction of academic performance of a student. Very less number of researches are directed towards predicting employability of a student i.e. prediction of students performance in campus placements at an early stage of enrollment. Furthermore, existing researches on students employability prediction are not universal in approach and is either based upon only one type of course or University/Institute. Henceforth, is not scalable from one context to another. With the necessity of unification, data of professional technical courses namely Bachelor in Engineering/Technology and Masters in Computer Applications students have been collected from 17 states of India. To deal with such a data, a unified predictive model has been developed and applied on 17 states datasets. The research done in this paper proves that model has universal application and can be applied to various states and institutes pan India with different cultural background and course structure. This paper also explores and proves statistically that there is no significant difference in Indian Education System with respect to states as far as prediction of employability of students is concerned. Model provides a generalized solution for student employability prediction in Indian Scenario.
Long Input Sequence Network for Long Time Series Forecasting
Authors: Chao Ma, Yikai Hou, Xiang Li, Yinggang Sun, Haining Yu
Date: 2024-07-18 00:00:00
Tasks: Time Series, Time Series Forecasting

Short fixed-length inputs are the main bottleneck of deep learning methods in long time-series forecasting tasks. Prolonging input length causes overfitting, rapidly deteriorating accuracy. Our research indicates that the overfitting is a combination reaction of the multi-scale pattern coupling in time series and the fixed focusing scale of current models. First, we find that the patterns exhibited by a time series across various scales are reflective of its multi-periodic nature, where each scale corresponds to specific period length. Second, We find that the token size predominantly dictates model behavior, as it determines the scale at which the model focuses and the context size it can accommodate. Our idea is to decouple the multi-scale temporal patterns of time series and to model each pattern with its corresponding period length as token size. We introduced a novel series-decomposition module(MPSD), and a Multi-Token Pattern Recognition neural network(MTPR), enabling the model to handle \textit{inputs up to $10\times$ longer}. Sufficient context enhances performance(\textit{38% maximum precision improvement}), and the decoupling approach offers \textit{Low complexity($0.22\times$ cost)} and \textit{high interpretability}.
Customising General Large Language Models for Specialised Emotion Recognition Tasks
Authors: Liyizhe Peng; Zixing Zhang; Tao Pang; Jing Han; Huan Zhao; Hao Chen; Björn W. Schuller
Date: 2024-04-14 00:00:00
Proceeding: Conference 2024 4
Tasks: Emotion Recognition, Language Modeling, Language Modelling, Zero-Shot Learning

The advent of large language models (LLMs) has gained tremendous attention over the past year. Previous studies have shown the astonishing performance of LLMs not only in other tasks but also in emotion recognition in terms of accuracy, universality, explanation, robustness, few/zero-shot learning, and others. Leveraging the capability of LLMs inevitably becomes an essential solution for emotion recognition. To this end, we further comprehensively investigate how LLMs perform in linguistic emotion recognition if we concentrate on this specific task. Specifically, we exemplify a publicly available and widely used LLM – Chat General Language Model, and customise it for our target by using two different model adaptation techniques, i.e., deep prompt tuning and low-rank adaptation. The experimental results obtained on six widely used datasets present that the adapted LLM can easily outperform other state-of-the-art but specialised deep models. This indicates the strong transferability and feasibility of LLMs in the field of emotion recognition.
Congruency of hydrocarbon patterns in heterospecific groups of ants: transfer and/or biosynthesis?
Authors: C. Vienne 1, 2, V. Soroker 1 and A. Hefetz 1
Date: 1995-09-01 00:00:00
Proceeding: September 1995 1995 9

In homospecific groups of ants, each species has its own hydrocarbon profile, on the epicuticle and
in the postpharyngeal gland (PPG). When reared together in bispecific groups, workers of both
species possess each other's hydrocarbons in both locations. The present study investigated two
alternative mechanisms by which a mixed "odour" in artificial groups of Formica selysi/Manica
rubida can be created. Using [1-14C] sodium acetate as a precursor, de novo biosynthesis of hydrocarbons was demonstrated for both species whether reared in homospecific or mixed-species
groups. The newly synthesized hydrocarbons occurred on the epicuticle, internally, and in particularly large amounts in the PPG. As expected from their PPG and epicuticular hydrocarbons
composition, workers E selysi synthesized alkanes and alkenes in comparable amounts irrespective
of their rearing scheme. Likewise, M. rubida reared in bispecific groups synthesized mostly
alkanes with only negligible amounts of alkenes, according to a ratio characteristic to M. rubida
workers from homospecific groups and not to E selysi workers. During dyadic encounters, a
transfer of labeled hydrocarbons between nestmates (conspecific in homospecific groups and
allospecific in mixed groups) was observed. These results suggest that the formation of the mixed
hydrocarbon profile in artificial groups of ants is the result of a transfer of these chemicals between
nestmates rather than de novo biosynthesis of the allospecific hydrocarbons. Behaviours like
trophallaxis, grooming and body contact that occurred during the encounters mediated such a
transfer.
Boosted multilayer feedforward neural network with multiple output layers
Authors: Hussein Aly, Abdulaziz K Al-Ali, Ponnuthurai Nagaratnam Suganthan
Date: 2024-07-16 00:00:00
Proceeding: Pattern Recognition 2024 7

This research introduces the Boosted Ensemble deep Multi-Layer Layer Perceptron (EdMLP) architecture with multiple output layers, a novel enhancement for the traditional Multi-Layer Perceptron (MLP). By adopting a layer-wise training approach, EdMLP enables the integration of boosting techniques within a single model, treating each layer as a weak learner, resulting in substantial performance gains. Additionally, the inclusion of layer-wise hyperparameter tuning allows optimization of individual layers thereby reducing the tuning time. Furthermore, the ensemble deep architecture’s versatility can be extended to other neural network-based models, such as the Self Normalized Network (SNN) where experiments demonstrate substantial performance enhancements yielded by the EdSNN compared to the standard original SNN model. This research underscores the potential of the EdMLP, and the Ed architecture in general as a powerful tool for improving the performance of various multilayer feedforward neural network models. The source code of this work is publicly accessible from the authors GitHub.
CrossLocate: Cross-modal Large-scale Visual Geo-Localization in Natural Environments using Rendered Modalities
Authors: Jan Tomešek, Martin Čadík, Jan Brejcha
Date: 2022-01-01 00:00:00
Proceeding: WACV 2022 1
Tasks: Camera Localization, Camera Pose Estimation, geo-localization, Image-Based Localization, Image Retrieval, Outdoor Localization, Photo geolocation estimation, Visual Localization

We propose a novel approach to visual geo-localization in natural environments. This is a challenging problem due to vast localization areas, the variable appearance of outdoor environments and the scarcity of available data. In order to make the research of new approaches possible, we first create two databases containing ''synthetic'' images of various modalities. These image modalities are rendered from a 3D terrain model and include semantic segmentations, silhouette maps and depth maps. By combining the rendered database views with existing datasets of photographs (used as ''queries'' to be localized), we create a unique benchmark for visual geo-localization in natural environments, which contains correspondences between query photographs and rendered database imagery. The distinct ability to match photographs to synthetically rendered databases defines our task as ''cross-modal''. On top of this benchmark, we provide thorough ablation studies analysing the localization potential of the database image modalities. We reveal the depth information as the best choice for outdoor localization. Finally, based on our observations, we carefully develop a fully-automatic method for large-scale cross-modal localization using image retrieval. We demonstrate its localization performance outdoors in the entire state of Switzerland. Our method reveals a large gap between operating within a single image domain (e.g. photographs) and working across domains (e.g. photographs matched to rendered images), as gained knowledge is not transferable between the two. Moreover, we show that modern localization methods fail when applied to such a cross-modal task and that our method achieves significantly better results than state-of-the-art approaches. The datasets, code and trained models are available on this website.
Toward Open-Set Human Object Interaction Detection
Authors: Mingrui Wu, Yuqi Liu, Jiayi Ji, Xiaoshuai Sun, Rongrong Ji
Date: 2024-03-24 00:00:00
Proceeding: Proceedings of the AAAI Conference on Artificial Intelligence 2024 3
Tasks: Contrastive Learning, Human-Object Interaction Detection, Language Modeling, Language Modelling, Object

This work is oriented toward the task of open-set Human Object Interaction (HOI) detection. The challenge lies in identifying completely new, out-of-domain relationships, as opposed to in-domain ones which have seen improvements in zero-shot HOI detection. To address this challenge, we introduce a simple Disentangled HOI Detection (DHD) model for detecting novel relationships by integrating an open-set object detector with a Visual Language Model (VLM). We utilize a disentangled image-text contrastive learning metric for training and connect the bottom-up visual features to text embeddings through lightweight unary and pair-wise adapters. Our model can benefit from the open-set object detector and the VLM to detect novel action categories and combine actions with novel object categories. We further present the VG-HOI dataset, a comprehensive benchmark with over 17k HOI relationships for open-set scenarios. Experimental results show that our model can detect unknown action classes and combine unknown object classes. Furthermore, it can generalize to over 17k HOI classes while being trained on just 600 HOI classes.
Profit Allocation for Federated Learning
Authors: Tianshu Song, Yongxin Tong*, Shuyue Wei
Date: 2019-01-01 00:00:00
Proceeding: IEEE International Conference on Big Data (Big Data) 2019 1
Tasks: Data Valuation, Federated Learning

Due to stricter data management regulations such as General Data Protection Regulation (GDPR), traditional production mode of machine learning services is shifting to federated learning, a paradigm that allows multiple data providers to train a joint model collaboratively with their data kept locally. A key enabler for practical adoption of federated learning is how to allocate the profit earned by the joint model to each data provider. For fair profit allocation, a metric to quantify the contribution of each data provider to the joint model is essential. Shapley value is a classical concept in cooperative game theory which assigns a unique distribution (among the players) of a total surplus generated by the coalition of all players and has been used for data valuation in machine learning services. However, prior Shapley value based data valuation schemes either do not apply to federated learning or involve extra model training which leads to high cost. In this paper, given n data providers with data sets D1, D2, · · · , Dn, a federated learning algorithm A and a standard test set T , we propose the contribution index, a new Shapley value based metric fit for assessing the contribution of each data provider for the joint model trained by federated learning. The contribution index shares the same properties as Shapley value. However, direct calculation of the contribution index is time consuming, since a large number of joint models with different combinations of data sets are required to be trained and evaluated. To solve this problem, we propose two gradient based methods. The idea is to reconstruct approximately the models on different combinations of the data sets through the intermediate results of the training process of federated learning so as to avoid extra training. The first method reconstructs models by updating the initial global model in federated learning with the gradients in different rounds. Then it calculates the contribution index by the performance of these reconstructed models. The second method calculates contribution index in each round by updating the global model in the previous round with the gradients in the current round. Contribution indexes of multiple rounds are then added with elaborated weights to get the final result. We conduct extensive experiments on the MNIST data set in different settings. The results demonstrate that the proposed methods can approximate the exact contribution index effectively and achieve a time speed up of up to 2x-100x compared with the exact calculation and other baselines extended from existing work.
A Scalable Quantum Non-local Neural Network for Image Classification
Authors: Sparsh Gupta, Debanjan Konar, Vaneet Aggarwal
Date: 2024-07-26 00:00:00
Tasks: Binary Classification, image-classification, Image Classification

Non-local operations play a crucial role in computer vision enabling the capture of long-range dependencies through weighted sums of features across the input, surpassing the constraints of traditional convolution operations that focus solely on local neighborhoods. Non-local operations typically require computing pairwise relationships between all elements in a set, leading to quadratic complexity in terms of time and memory. Due to the high computational and memory demands, scaling non-local neural networks to large-scale problems can be challenging. This article introduces a hybrid quantum-classical scalable non-local neural network, referred to as Quantum Non-Local Neural Network (QNL-Net), to enhance pattern recognition. The proposed QNL-Net relies on inherent quantum parallelism to allow the simultaneous processing of a large number of input features enabling more efficient computations in quantum-enhanced feature space and involving pairwise relationships through quantum entanglement. We benchmark our proposed QNL-Net with other quantum counterparts to binary classification with datasets MNIST and CIFAR-10. The simulation findings showcase our QNL-Net achieves cutting-edge accuracy levels in binary image classification among quantum classifiers while utilizing fewer qubits.
PP-TIL: Personalized Planning for Autonomous Driving with Instance-based Transfer Imitation Learning
Authors: Fangze Lin, Ying He, Fei Yu
Date: 2024-07-26 00:00:00
Tasks: Autonomous Driving, Imitation Learning, Motion Planning, Transfer Learning

Personalized motion planning holds significant importance within urban automated driving, catering to the unique requirements of individual users. Nevertheless, prior endeavors have frequently encountered difficulties in simultaneously addressing two crucial aspects: personalized planning within intricate urban settings and enhancing planning performance through data utilization. The challenge arises from the expensive and limited nature of user data, coupled with the scene state space tending towards infinity. These factors contribute to overfitting and poor generalization problems during model training. Henceforth, we propose an instance-based transfer imitation learning approach. This method facilitates knowledge transfer from extensive expert domain data to the user domain, presenting a fundamental resolution to these issues. We initially train a pre-trained model using large-scale expert data. Subsequently, during the fine-tuning phase, we feed the batch data, which comprises expert and user data. Employing the inverse reinforcement learning technique, we extract the style feature distribution from user demonstrations, constructing the regularization term for the approximation of user style. In our experiments, we conducted extensive evaluations of the proposed method. Compared to the baseline methods, our approach mitigates the overfitting issue caused by sparse user data. Furthermore, we discovered that integrating the driving model with a differentiable nonlinear optimizer as a safety protection layer for end-to-end personalized fine-tuning results in superior planning performance.
Text-Region Matching for Multi-Label Image Recognition with Missing Labels
Authors: Leilei Ma, Hongxing Xie, Lei Wang, Yanping Fu, Dengdi Sun, Haifeng Zhao
Date: 2024-07-26 00:00:00
Tasks: Contrastive Learning, Missing Labels, Multi-Label Image Recognition, Pseudo Label

Recently, large-scale visual language pre-trained (VLP) models have demonstrated impressive performance across various downstream tasks. Motivated by these advancements, pioneering efforts have emerged in multi-label image recognition with missing labels, leveraging VLP prompt-tuning technology. However, they usually cannot match text and vision features well, due to complicated semantics gaps and missing labels in a multi-label image. To tackle this challenge, we propose $\textbf{T}$ext-$\textbf{R}$egion $\textbf{M}$atching for optimizing $\textbf{M}$ulti-$\textbf{L}$abel prompt tuning, namely TRM-ML, a novel method for enhancing meaningful cross-modal matching. Compared to existing methods, we advocate exploring the information of category-aware regions rather than the entire image or pixels, which contributes to bridging the semantic gap between textual and visual representations in a one-to-one matching manner. Concurrently, we further introduce multimodal contrastive learning to narrow the semantic gap between textual and visual modalities and establish intra-class and inter-class relationships. Additionally, to deal with missing labels, we propose a multimodal category prototype that leverages intra- and inter-category semantic relationships to estimate unknown labels, facilitating pseudo-label generation. Extensive experiments on the MS-COCO, PASCAL VOC, Visual Genome, NUS-WIDE, and CUB-200-211 benchmark datasets demonstrate that our proposed framework outperforms the state-of-the-art methods by a significant margin. Our code is available here: https://github.com/yu-gi-oh-leilei/TRM-ML.
Unifying Visual and Semantic Feature Spaces with Diffusion Models for Enhanced Cross-Modal Alignment
Authors: Yuze Zheng, Zixuan Li, Xiangxian Li, JinXing Liu, Yuqing Wang, Xiangxu Meng, Lei Meng
Date: 2024-07-26 00:00:00
Tasks: cross-modal alignment, image-classification, Image Classification

Image classification models often demonstrate unstable performance in real-world applications due to variations in image information, driven by differing visual perspectives of subject objects and lighting discrepancies. To mitigate these challenges, existing studies commonly incorporate additional modal information matching the visual data to regularize the model's learning process, enabling the extraction of high-quality visual features from complex image regions. Specifically, in the realm of multimodal learning, cross-modal alignment is recognized as an effective strategy, harmonizing different modal information by learning a domain-consistent latent feature space for visual and semantic features. However, this approach may face limitations due to the heterogeneity between multimodal information, such as differences in feature distribution and structure. To address this issue, we introduce a Multimodal Alignment and Reconstruction Network (MARNet), designed to enhance the model's resistance to visual noise. Importantly, MARNet includes a cross-modal diffusion reconstruction module for smoothly and stably blending information across different domains. Experiments conducted on two benchmark datasets, Vireo-Food172 and Ingredient-101, demonstrate that MARNet effectively improves the quality of image information extracted by the model. It is a plug-and-play framework that can be rapidly integrated into various image classification frameworks, boosting model performance.
ChatSchema: A pipeline of extracting structured information with Large Multimodal Models based on schema
Authors: Fei Wang, Yuewen Zheng, Qin Li, Jingyi Wu, Pengfei Li, Luxia Zhang
Date: 2024-07-26 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

Objective: This study introduces ChatSchema, an effective method for extracting and structuring information from unstructured data in medical paper reports using a combination of Large Multimodal Models (LMMs) and Optical Character Recognition (OCR) based on the schema. By integrating predefined schema, we intend to enable LMMs to directly extract and standardize information according to the schema specifications, facilitating further data entry. Method: Our approach involves a two-stage process, including classification and extraction for categorizing report scenarios and structuring information. We established and annotated a dataset to verify the effectiveness of ChatSchema, and evaluated key extraction using precision, recall, F1-score, and accuracy metrics. Based on key extraction, we further assessed value extraction. We conducted ablation studies on two LMMs to illustrate the improvement of structured information extraction with different input modals and methods. Result: We analyzed 100 medical reports from Peking University First Hospital and established a ground truth dataset with 2,945 key-value pairs. We evaluated ChatSchema using GPT-4o and Gemini 1.5 Pro and found a higher overall performance of GPT-4o. The results are as follows: For the result of key extraction, key-precision was 98.6%, key-recall was 98.5%, key-F1-score was 98.6%. For the result of value extraction based on correct key extraction, the overall accuracy was 97.2%, precision was 95.8%, recall was 95.8%, and F1-score was 95.8%. An ablation study demonstrated that ChatSchema achieved significantly higher overall accuracy and overall F1-score of key-value extraction, compared to the Baseline, with increases of 26.9% overall accuracy and 27.4% overall F1-score, respectively.