pwc-archive/papers-with-abstracts huggingface dataset:
Amazon rainforest photosynthesis increases in response to atmospheric dryness
Authors: J. K. Green, J. Berry, P. Ciais, Y. Zhang, P. Gentine
Date: 2020-11-20 00:00:00
Proceeding: Science Advances 2020 11

Earth system models predict that increases in atmospheric and soil dryness will reduce photosynthesis in the
Amazon rainforest, with large implications for the global carbon cycle. Using in situ observations, solar-induced
fluorescence, and nonlinear machine learning techniques, we show that, in reality, this is not necessarily the case:
In many of the wettest parts of this region, photosynthesis and biomass tend to increase with increased atmo-
spheric dryness, despite the associated reductions in canopy conductance to CO2. These results can be largely
explained by changes in canopy properties, specifically, new leaves flushed during the dry season have higher
photosynthetic capacity than the leaves they replace, compensating for the negative stomatal response to in-
creased dryness. As atmospheric dryness will increase with climate change, our study highlights the importance
of reframing how we represent the response of ecosystem photosynthesis to atmospheric dryness in very wet
regions, to accurately quantify the land carbon sink.
Enhancing Data Provenance and Model Transparency in Federated Learning Systems - A Database Approach
Authors: Michael Gu, Ramasoumya Naraparaju, Dongfang Zhao
Date: 2024-03-03 00:00:00
Tasks: Federated Learning

Federated Learning (FL) presents a promising paradigm for training machine learning models across decentralized edge devices while preserving data privacy. Ensuring the integrity and traceability of data across these distributed environments, however, remains a critical challenge. The ability to create transparent artificial intelligence, such as detailing the training process of a machine learning model, has become an increasingly prominent concern due to the large number of sensitive (hyper)parameters it utilizes; thus, it is imperative to strike a reasonable balance between openness and the need to protect sensitive information. In this paper, we propose one of the first approaches to enhance data provenance and model transparency in federated learning systems. Our methodology leverages a combination of cryptographic techniques and efficient model management to track the transformation of data throughout the FL process, and seeks to increase the reproducibility and trustworthiness of a trained FL model. We demonstrate the effectiveness of our approach through experimental evaluations on diverse FL scenarios, showcasing its ability to tackle accountability and explainability across the board. Our findings show that our system can greatly enhance data transparency in various FL environments by storing chained cryptographic hashes and client model snapshots in our proposed design for data decoupled FL. This is made possible by also employing multiple optimization techniques which enables comprehensive data provenance without imposing substantial computational loads. Extensive experimental results suggest that integrating a database subsystem into federated learning systems can improve data provenance in an efficient manner, encouraging secure FL adoption in privacy-sensitive applications and paving the way for future advancements in FL transparency and security features.
NusaBERT: Teaching IndoBERT to be Multilingual and Multicultural
Authors: Wilson Wongso, David Samuel Setiawan, Steven Limcorn, Ananto Joyoadikusumo
Date: 2024-03-04 00:00:00
Tasks: Diversity, Natural Language Understanding

Indonesia's linguistic landscape is remarkably diverse, encompassing over 700 languages and dialects, making it one of the world's most linguistically rich nations. This diversity, coupled with the widespread practice of code-switching and the presence of low-resource regional languages, presents unique challenges for modern pre-trained language models. In response to these challenges, we developed NusaBERT, building upon IndoBERT by incorporating vocabulary expansion and leveraging a diverse multilingual corpus that includes regional languages and dialects. Through rigorous evaluation across a range of benchmarks, NusaBERT demonstrates state-of-the-art performance in tasks involving multiple languages of Indonesia, paving the way for future natural language understanding research for under-represented languages.
Can Poverty Be Reduced by Acting on Discrimination? An Agent-based Model for Policy Making
Authors: Alba Aguilera, Nieves Montes, Georgina Curto, Carles Sierra, Nardine Osman
Date: 2024-03-03 00:00:00

In the last decades, there has been a deceleration in the rates of poverty reduction, suggesting that traditional redistributive approaches to poverty mitigation could be losing effectiveness, and alternative insights to advance the number one UN Sustainable Development Goal are required. The criminalization of poor people has been denounced by several NGOs, and an increasing number of voices suggest that discrimination against the poor (a phenomenon known as \emph{aporophobia}) could be an impediment to mitigating poverty. In this paper, we present the novel Aporophobia Agent-Based Model (AABM) to provide evidence of the correlation between aporophobia and poverty computationally. We present our use case built with real-world demographic data and poverty-mitigation public policies (either enforced or under parliamentary discussion) for the city of Barcelona. We classify policies as discriminatory or non-discriminatory against the poor, with the support of specialized NGOs, and we observe the results in the AABM in terms of the impact on wealth inequality. The simulation provides evidence of the relationship between aporophobia and the increase of wealth inequality levels, paving the way for a new generation of poverty reduction policies that act on discrimination and tackle poverty as a societal problem (not only a problem of the poor).
Recency-Weighted Temporally-Segmented Ensemble for Time-Series Modeling
Authors: Pål V. Johnsen, Eivind Bøhn, Sølve Eidnes, Filippo Remonato, Signe Riemer-Sørensen
Date: 2024-03-04 00:00:00
Tasks: Time Series

Time-series modeling in process industries faces the challenge of dealing with complex, multi-faceted, and evolving data characteristics. Conventional single model approaches often struggle to capture the interplay of diverse dynamics, resulting in suboptimal forecasts. Addressing this, we introduce the Recency-Weighted Temporally-Segmented (ReWTS, pronounced `roots') ensemble model, a novel chunk-based approach for multi-step forecasting. The key characteristics of the ReWTS model are twofold: 1) It facilitates specialization of models into different dynamics by segmenting the training data into `chunks' of data and training one model per chunk. 2) During inference, an optimization procedure assesses each model on the recent past and selects the active models, such that the appropriate mixture of previously learned dynamics can be recalled to forecast the future. This method not only captures the nuances of each period, but also adapts more effectively to changes over time compared to conventional `global' models trained on all data in one go. We present a comparative analysis, utilizing two years of data from a wastewater treatment plant and a drinking water treatment plant in Norway, demonstrating the ReWTS ensemble's superiority. It consistently outperforms the global model in terms of mean squared forecasting error across various model architectures by 10-70\% on both datasets, notably exhibiting greater resilience to outliers. This approach shows promise in developing automatic, adaptable forecasting models for decision-making and control systems in process industries and other complex systems.
An Ultralightweight Hybrid CNN Based on Redundancy Removal for Hyperspectral Image Classification
Authors: Xiaohu Ma, Wuli Wang, Member, IEEE
Date: 2024-01-24 00:00:00
Proceeding: IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 2024 1
Tasks: Hyperspectral Image Classification, image-classification, Image Classification

Convolutional neural network (CNN)-based hyper-
spectral image (HSI) classification models often exhibit high
volume and complexity. This not only poses challenges in
deploying them on mobile and embedded devices due to storage
and power constraints, but also introduces a dilemma between
the growing demand for labeled samples and the high cost
associated with manual labeling. To address these challenges,
we propose an ultralightweight hybrid CNN based on redundancy
removal (ULite-R2HCN), specifically designed for HSI classifica-
tion in scenarios with limited samples. To reduce computational
costs and enhance feature extraction effectiveness, we focus on
optimizing the widely used depthwise convolution (DW-Conv)
and pointwise convolution (PW-Conv) in the lightweight HSI
classification model. For DW-Conv, we design a spatial convo-
lution with redundancy removal (R2Spatial-Conv). This involves
the design of multiscale 3-D convolution kernels with specific
structures instead of 2-D convolution kernels, aiming to reduce
redundant convolution kernels and extract multiscale spatial
features. Simultaneously, for PW-Conv, we design a spectral
convolution with redundancy removal (R2Spectral-Conv). This
utilizes a “copy-splicing-grouping” structure to extract spectral
features within arbitrary range intervals, effectively reducing
redundant spectral extractions and capturing long-range spec-
tral relationships. Numerous experiments have shown that the
proposed ULite-R2HCN achieves higher classification accuracy
with an ultralight volume for a few training samples. In addition,
sufficient ablation experiments also verified the advanced perfor-
mance of the designed R2Spatial-Conv and R2Spectral-Conv.
VTG-GPT: Tuning-Free Zero-Shot Video Temporal Grounding with GPT
Authors: Yifang Xu, Yunzhuo Sun, Zien Xie, Benxiang Zhai, Sidan Du
Date: 2024-03-04 00:00:00
Proceeding: Applied Sciences 2024 2
Tasks: Image Captioning, Zero-shot Moment Retrieval

Video temporal grounding (VTG) aims to locate specific temporal segments from an untrimmed video based on a linguistic query. Most existing VTG models are trained on extensive annotated video-text pairs, a process that not only introduces human biases from the queries but also incurs significant computational costs. To tackle these challenges, we propose VTG-GPT, a GPT-based method for zero-shot VTG without training or fine-tuning. To reduce prejudice in the original query, we employ Baichuan2 to generate debiased queries. To lessen redundant information in videos, we apply MiniGPT-v2 to transform visual content into more precise captions. Finally, we devise the proposal generator and post-processing to produce accurate segments from debiased queries and image captions. Extensive experiments demonstrate that VTG-GPT significantly outperforms SOTA methods in zero-shot settings and surpasses unsupervised approaches. More notably, it achieves competitive performance comparable to supervised methods. The code is available on https://github.com/YoucanBaby/VTG-GPT
Tree Counting by Bridging 3D Point Clouds with Imagery
Authors: Lei LI, Tianfang Zhang, Zhongyu Jiang, Cheng-Yen Yang, Jenq-Neng Hwang, Stefan Oehmcke, Dimitri Pierre Johannes Gominski, Fabian Gieseke, Christian Igel
Date: 2024-03-04 00:00:00
Tasks: Management

Accurate and consistent methods for counting trees based on remote sensing data are needed to support sustainable forest management, assess climate change mitigation strategies, and build trust in tree carbon credits. Two-dimensional remote sensing imagery primarily shows overstory canopy, and it does not facilitate easy differentiation of individual trees in areas with a dense canopy and does not allow for easy separation of trees when the canopy is dense. We leverage the fusion of three-dimensional LiDAR measurements and 2D imagery to facilitate the accurate counting of trees. We compare a deep learning approach to counting trees in forests using 3D airborne LiDAR data and 2D imagery. The approach is compared with state-of-the-art algorithms, like operating on 3D point cloud and 2D imagery. We empirically evaluate the different methods on the NeonTreeCount data set, which we use to define a tree-counting benchmark. The experiments show that FuseCountNet yields more accurate tree counts.
EEE-QA: Exploring Effective and Efficient Question-Answer Representations
Authors: Zhanghao Hu, Yijun Yang, Junjie Xu, Yifu Qiu, Pinzhen Chen
Date: 2024-03-04 00:00:00
Tasks: Knowledge Graphs, Question Answering, Sentence

Current approaches to question answering rely on pre-trained language models (PLMs) like RoBERTa. This work challenges the existing question-answer encoding convention and explores finer representations. We begin with testing various pooling methods compared to using the begin-of-sentence token as a question representation for better quality. Next, we explore opportunities to simultaneously embed all answer candidates with the question. This enables cross-reference between answer choices and improves inference throughput via reduced memory usage. Despite their simplicity and effectiveness, these methods have yet to be widely studied in current frameworks. We experiment with different PLMs, and with and without the integration of knowledge graphs. Results prove that the memory efficacy of the proposed techniques with little sacrifice in performance. Practically, our work enhances 38-100% throughput with 26-65% speedups on consumer-grade GPUs by allowing for considerably larger batch sizes. Our work sends a message to the community with promising directions in both representation quality and efficiency for the question-answering task in natural language processing.
Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures
Authors: Yuchen Duan, Weiyun Wang, Zhe Chen, Xizhou Zhu, Lewei Lu, Tong Lu, Yu Qiao, Hongsheng Li, Jifeng Dai, Wenhai Wang
Date: 2024-03-04 00:00:00
Tasks: image-classification, Image Classification

Transformers have revolutionized computer vision and natural language processing, but their high computational complexity limits their application in high-resolution image processing and long-context analysis. This paper introduces Vision-RWKV (VRWKV), a model adapted from the RWKV model used in the NLP field with necessary modifications for vision tasks. Similar to the Vision Transformer (ViT), our model is designed to efficiently handle sparse inputs and demonstrate robust global processing capabilities, while also scaling up effectively, accommodating both large-scale parameters and extensive datasets. Its distinctive advantage lies in its reduced spatial aggregation complexity, which renders it exceptionally adept at processing high-resolution images seamlessly, eliminating the necessity for windowing operations. Our evaluations demonstrate that VRWKV surpasses ViT's performance in image classification and has significantly faster speeds and lower memory usage processing high-resolution inputs. In dense prediction tasks, it outperforms window-based models, maintaining comparable speeds. These results highlight VRWKV's potential as a more efficient alternative for visual perception tasks. Code is released at https://github.com/OpenGVLab/Vision-RWKV.
Hybrid Quantum Neural Network Advantage for Radar-Based Drone Detection and Classification in Low Signal-to-Noise Ratio
Authors: Aiswariya Sweety Malarvanan
Date: 2024-03-04 00:00:00
Tasks: Time Series

In this paper, we investigate the performance of a Hybrid Quantum Neural Network (HQNN) and a comparable classical Convolution Neural Network (CNN) for detection and classification problem using a radar. Specifically, we take a fairly complex radar time-series model derived from electromagnetic theory, namely the Martin-Mulgrew model, that is used to simulate radar returns of objects with rotating blades, such as drones. We find that when that signal-to-noise ratio (SNR) is high, CNN outperforms the HQNN for detection and classification. However, in the low SNR regime (which is of greatest interest in practice) the performance of HQNN is found to be superior to that of the CNN of a similar architecture.
Randomized Approach to Matrix Completion: Applications in Collaborative Filtering and Image Inpainting
Authors: Antonina Krajewska, Ewa Niewiadomska-Szynkiewicz
Date: 2024-03-04 00:00:00
Tasks: Collaborative Filtering, Image Inpainting, Low-Rank Matrix Completion, Matrix Completion, Missing Elements, Recommendation Systems

We present a novel method for matrix completion, specifically designed for matrices where one dimension significantly exceeds the other. Our Columns Selected Matrix Completion (CSMC) method combines Column Subset Selection and Low-Rank Matrix Completion to efficiently reconstruct incomplete datasets. In each step, CSMC solves a convex optimization problem. We introduce two algorithms to implement CSMC, each tailored to problems of different sizes. A formal analysis is provided, outlining the necessary assumptions and the probability of obtaining a correct solution. To assess the impact of matrix size, rank, and the ratio of missing entries on solution quality and computation time, we conducted experiments on synthetic data. The method was also applied to two real-world problems: recommendation systems and image inpainting. Our results show that CSMC provides solutions of the same quality as state-of-the-art matrix completion algorithms based on convex optimization, while achieving significant reductions in computational runtime.
Beyond Specialization: Assessing the Capabilities of MLLMs in Age and Gender Estimation
Authors: Maksim Kuprashevich, Grigorii Alekseenko, Irina Tolstykh
Date: 2024-03-04 00:00:00
Tasks: Age And Gender Classification, Age and Gender Estimation, Age Estimation, Facial Attribute Classification, Gender Prediction, General Knowledge

Multimodal Large Language Models (MLLMs) have recently gained immense popularity. Powerful commercial models like ChatGPT-4V and Gemini, as well as open-source ones such as LLaVA, are essentially general-purpose models and are applied to solve a wide variety of tasks, including those in computer vision. These neural networks possess such strong general knowledge and reasoning abilities that they have proven capable of working even on tasks for which they were not specifically trained. We compared the capabilities of the most powerful MLLMs to date: ShareGPT4V, ChatGPT, LLaVA-Next in a specialized task of age and gender estimation with our state-of-the-art specialized model, MiVOLO. We also updated MiVOLO and provide details and new metrics in this article. This comparison has yielded some interesting results and insights about the strengths and weaknesses of the participating models. Furthermore, we attempted various ways to fine-tune the ShareGPT4V model for this specific task, aiming to achieve state-of-the-art results in this particular challenge. Although such a model would not be practical in production, as it is incredibly expensive compared to a specialized model like MiVOLO, it could be very useful in some tasks, like data annotation.
What has LeBenchmark Learnt about French Syntax?
Authors: Zdravko Dugonjić, Adrien Pupier, Benjamin Lecouteux, Maximin Coavoux
Date: 2024-03-04 00:00:00
Tasks: Automatic Speech Recognition, speech-recognition, Speech Recognition, Spoken Language Understanding

The paper reports on a series of experiments aiming at probing LeBenchmark, a pretrained acoustic model trained on 7k hours of spoken French, for syntactic information. Pretrained acoustic models are increasingly used for downstream speech tasks such as automatic speech recognition, speech translation, spoken language understanding or speech parsing. They are trained on very low level information (the raw speech signal), and do not have explicit lexical knowledge. Despite that, they obtained reasonable results on tasks that requires higher level linguistic knowledge. As a result, an emerging question is whether these models encode syntactic information. We probe each representation layer of LeBenchmark for syntax, using the Orf\'eo treebank, and observe that it has learnt some syntactic information. Our results show that syntactic information is more easily extractable from the middle layers of the network, after which a very sharp decrease is observed.
Optimal Sizing of Hybrid Renewable Energy Based Microgrid System
Authors: Irfan Rahman, Farheen Suha, Ashik Ahmed
Date: 2024-03-03 00:00:00

With the decline of fossil fuel reserves and the escalating global average temperature, the quest for environmentally friendly and renewable energy sources has gained significant momentum. Focus has turned to wind and photovoltaic energy, but their variable inputs necessitate energy storage for reliable power. Economic viability of hybrid renewable power requires meticulous optimization of generating units to ensure uninterrupted and efficient energy production. This paper presents an optimal sizing approach for a Wind-Photovoltaic-Biogas-Battery system using a single objective optimization (SOO) method. A comprehensive comparative analysis is conducted, evaluating the convergence speed and objective mean (for minimization) of seven metaheuristic optimizers: Particle Swarm Optimization (PSO), Aquila Optimizer (AO), Pelican Optimization Algorithm (POA), Dandelion Optimizing Algorithm (DOA), Gazelle Optimization Algorithm (GOA), Zebra Optimization Algorithm (ZOA), and Osprey Optimization Algorithm (OOA). The results demonstrate that the Pelican Optimization Algorithm (POA) outperforms other existing algorithms, exhibiting faster convergence and lower objective mean.
Gradient Correlation Subspace Learning against Catastrophic Forgetting
Authors: Tammuz Dubnov, Vishal Thengane
Date: 2024-03-04 00:00:00
Tasks: Continual Learning

Efficient continual learning techniques have been a topic of significant research over the last few years. A fundamental problem with such learning is severe degradation of performance on previously learned tasks, known also as catastrophic forgetting. This paper introduces a novel method to reduce catastrophic forgetting in the context of incremental class learning called Gradient Correlation Subspace Learning (GCSL). The method detects a subspace of the weights that is least affected by previous tasks and projects the weights to train for the new task into said subspace. The method can be applied to one or more layers of a given network architectures and the size of the subspace used can be altered from layer to layer and task to task. Code will be available at \href{https://github.com/vgthengane/GCSL}{https://github.com/vgthengane/GCSL}
Neural Graph Generator: Feature-Conditioned Graph Generation using Latent Diffusion Models
Authors: Iakovos Evdaimon, Giannis Nikolentzos, Christos Xypolopoulos, Ahmed Kammoun, Michail Chatzianastasis, Hadi Abdine, Michalis Vazirgiannis
Date: 2024-03-03 00:00:00
Tasks: Graph Generation

Graph generation has emerged as a crucial task in machine learning, with significant challenges in generating graphs that accurately reflect specific properties. Existing methods often fall short in efficiently addressing this need as they struggle with the high-dimensional complexity and varied nature of graph properties. In this paper, we introduce the Neural Graph Generator (NGG), a novel approach which utilizes conditioned latent diffusion models for graph generation. NGG demonstrates a remarkable capacity to model complex graph patterns, offering control over the graph generation process. NGG employs a variational graph autoencoder for graph compression and a diffusion process in the latent vector space, guided by vectors summarizing graph statistics. We demonstrate NGG's versatility across various graph generation tasks, showing its capability to capture desired graph properties and generalize to unseen graphs. We also compare our generator to the graph generation capabilities of different LLMs. This work signifies a shift in graph generation methodologies, offering a more practical and efficient solution for generating diverse graphs with specific characteristics.