pwc-archive/papers-with-abstracts huggingface dataset:
BirdNET: A deep learning solution for avian diversity monitoring
Authors: Stefan Kahl, Connor M. Wood, Maximilian Eibl, Holger Klinck
Date: 2021-01-27 00:00:00
Proceeding: Ecological Informatics 2021 1
Tasks: Data Augmentation, Deep Learning, Diversity, Event Detection, object-detection, Object Detection

Variation in avian diversity in space and time is commonly used as a metric to assess environmental changes. Conventionally, such data were collected by expert observers, but passively collected acoustic data is rapidly emerging as an alternative survey technique. However, efficiently extracting accurate species richness data from large audio datasets has proven challenging. Recent advances in deep artificial neural networks (DNNs) have transformed the field of machine learning, frequently outperforming traditional signal processing techniques in the domain of acoustic event detection and classification. We developed a DNN, called BirdNET, capable of identifying 984 North American and European bird species by sound. Our task-specific model architecture was derived from the family of residual networks (ResNets), consisted of 157 layers with more than 27 million parameters, and was trained using extensive data pre-processing, augmentation, and mixup. We tested the model against three independent datasets: (a) 22,960 single-species recordings; (b) 286 h of fully annotated soundscape data collected by an array of autonomous recording units in a design analogous to what researchers might use to measure avian diversity in a field setting; and (c) 33,670 h of soundscape data from a single high-quality omnidirectional microphone deployed near four eBird hotspots frequented by expert birders. We found that domain-specific data augmentation is key to build models that are robust against high ambient noise levels and can cope with overlapping vocalizations. Task-specific model designs and training regimes for audio event recognition perform on-par with very complex architectures used in other domains (e.g., object detection in images). We also found that high temporal resolution of input spectrograms (short FFT window length) improves the classification performance for bird sounds. In summary, BirdNET achieved a mean average precision of 0.791 for single-species recordings, a F0.5 score of 0.414 for annotated soundscapes, and an average correlation of 0.251 with hotspot observation across 121 species and 4 years of audio data. By enabling the efficient extraction of the vocalizations of many hundreds of bird species from potentially vast amounts of audio data, BirdNET and similar tools have the potential to add tremendous value to existing and future passively collected audio datasets and may transform the field of avian ecology and conservation.
MSBoost: Using Model Selection with Multiple Base Estimators for Gradient Boosting
Authors: Agnij Moitra
Date: 2024-06-17 00:00:00
Proceeding: Under review at NeurIPS 2024 6
Tasks: Classification, Model Selection, regression, tabular-classification, tabular-regression

Gradient boosting is a widely used machine learning algorithm for tabular regression, classification and ranking. Although, most of the open source implementations of gradient boosting such as XGBoost, LightGBM and others have used decision trees as the sole base estimator for gradient boosting. This paper, for the first time, takes an alternative path of not just relying on a static base estimator (usually decision tree), and rather trains a list of models in parallel on the residual errors of the previous layer and then selects the model with the least validation error as the base estimator for a particular layer. This paper has achieved state-of-the-art results when compared to other gradient boosting implementations on 50+ tabular regression and classification datasets. Furthermore, ablation studies show that MSBoost is particularly effective for small and noisy datasets. Thereby, it has a significant social impact especially in tabular machine learning problems in the domains where it is not feasible to obtain large high quality datasets.
Predictive microstructure image generation using denoising diffusion probabilistic models
Authors: Erfan Azqadan, Hamid Jahed, Arash Arami
Date: 2023-10-05 00:00:00
Proceeding: Acta Materialia 2023 10
Tasks: Denoising, Image Generation

The rapid progress in artificial intelligence (AI) based image generation led to groundbreaking achievements, like OpenAI's DALL-E 2, showcasing state-of-the-art generative models in deep learning and computer vision. Recently, the Denoising Diffusion Probabilistic Model (DDPM) has emerged as a strong contender, excelling in generating high-resolution images with complex features similar to those found in real-world images. In this study, we investigate DDPM's potential as both generator and predictor of scanning electron microscope (SEM) images, encompassing both known and unseen microstructural conditions. To rigorously evaluate DDPM, we curated a comprehensive dataset comprising 27 distinct cast-forged AZ80 magnesium alloy components with varied process parameters and microstructure features. Some conditions were held back during training to test DDPM's predictive abilities for unseen scenarios. Our study demonstrates the model's remarkable capacity to capture the inherent physical relationships between process parameters and microstructure features. We scrutinize the synthesized images alongside real-world SEM counterparts, undertaking a comprehensive analysis of various morphological properties. Remarkably, the results show the model's performance, with an average error of 6.36 % ± 0.42 for measured microstructural properties in seen conditions and an equally impressive 6.67 % ± 0.85 for unseen conditions. This study envisions a transformative shift in materials science, as advanced AI predictive models offer new potential to streamline the laborious process of microstructure image generation.
Single classifier vs. ensemble machine learning approaches for mental health prediction
Authors: Jetli Chung, Jason Teo
Date: 2023-01-03 00:00:00
Proceeding: Brain Informatics 2023 1

Early prediction of mental health issues among individuals is paramount for early diagnosis and treatment by mental
health professionals. One of the promising approaches to achieving fully automated computer-based approaches for
predicting mental health problems is via machine learning. As such, this study aims to empirically evaluate several
popular machine learning algorithms in classifying and predicting mental health problems based on a given data
set, both from a single classifier approach as well as an ensemble machine learning approach. The data set contains
responses to a survey questionnaire that was conducted by Open Sourcing Mental Illness (OSMI). Machine learning
algorithms investigated in this study include Logistic Regression, Gradient Boosting, Neural Networks, K-Nearest
Neighbours, and Support Vector Machine, as well as an ensemble approach using these algorithms. Comparisons
were also made against more recent machine learning approaches, namely Extreme Gradient Boosting and Deep
Neural Networks. Overall, Gradient Boosting achieved the highest overall accuracy of 88.80% followed by Neural
Networks with 88.00%. This was followed by Extreme Gradient Boosting and Deep Neural Networks at 87.20% and
86.40%, respectively. The ensemble classifier achieved 85.60% while the remaining classifiers achieved between 82.40
and 84.00%. The findings indicate that Gradient Boosting provided the highest classification accuracy for this particular
mental health bi-classification prediction task. In general, it was also demonstrated that the prediction results produced
by all of the machine learning approaches studied here were able to achieve more than 80% accuracy, thereby
indicating a highly promising approach for mental health professionals toward automated clinical diagnosis.
Benchmarking Geospatial Question Answering Engines using the Dataset GeoQuestions1089
Authors: Sergios-Anestis Kefalidis, Dharmen Punjani, Eleni Tsalapati, Konstantinos Plas, Mariangela Pollali, Michail Mitsios, Myrto Tsokanaridou, Manolis Koubarakis, Pierre Maret
Date: 2023-11-06 00:00:00
Proceeding: International Semantic Web Conference 2023 11
Tasks: Benchmarking, Knowledge Base Question Answering, Question Answering

We present the dataset GeoQuestions1089 for benchmarking geospatial question answering engines. GeoQuestions1089 is the
largest such dataset available presently and it contains 1089 questions, their corresponding GeoSPARQL or SPARQL queries and their answers over the geospatial knowledge graph YAGO2geo. We use GeoQuestions1089 to evaluate the effectiveness and efficiency of geospatial question answering engines GeoQA2 (an extension of GeoQA developed by our group) and the system of Hamzei et al. (2022).
Joint Beamforming and Antenna Position Optimization for Movable Antenna-Assisted Spectrum Sharing
Authors: Xin Wei, Weidong Mei, Dong Wang, Boyu Ning, Zhi Chen
Date: 2024-06-28 00:00:00
Tasks: Position

Fluid antennas (FAs) and movable antennas (MAs) have drawn increasing attention in wireless communications recently due to their ability to create favorable channel conditions via local antenna movement within a confined region. In this letter, we advance their application for cognitive radio to facilitate efficient spectrum sharing between primary and secondary communication systems. In particular, we aim to jointly optimize the transmit beamforming and MA positions at a secondary transmitter (ST) to maximize the received signal power at a secondary receiver (SR) subject to the constraints on its imposed co-channel interference power with multiple primary receivers (PRs). However, such an optimization problem is difficult to be optimally solved due to the highly nonlinear functions of the received signal/interference power at the SR/all PRs in terms of the MA positions. To drive useful insights, we first perform theoretical analyses to unveil MAs' capability to achieve maximum-ratio transmission with the SR and effective interference mitigation for all PRs at the same time. To solve the MA position optimization problem, we propose an alternating optimization (AO) algorithm to obtain a high-quality suboptimal solution. Numerical results demonstrate that our proposed algorithms can significantly outperform the conventional fixed-position antennas (FPAs) and other baseline schemes.
Odd-One-Out: Anomaly Detection by Comparing with Neighbors
Authors: Ankan Bhunia, Changjian Li, Hakan Bilen
Date: 2024-06-28 00:00:00
Proceeding: CVPR 2025 1
Tasks: 8k, Anomaly Detection, Odd One Out

This paper introduces a novel anomaly detection (AD) problem that focuses on identifying `odd-looking' objects relative to the other instances in a given scene. In contrast to the traditional AD benchmarks, anomalies in our task are scene-specific, defined by the regular instances that make up the majority. Since object instances may be only partly visible from a single viewpoint, our setting employs multiple views of each scene as input. To provide a testbed for future research in this task, we introduce two benchmarks, ToysAD-8K and PartsAD-15K. We propose a novel method that constructs 3D object-centric representations from multiple 2D views for each instance and detects the anomalous ones through a cross-instance comparison. We rigorously analyze our method quantitatively and qualitatively on the presented benchmarks.
Function+Data Flow: A Framework to Specify Machine Learning Pipelines for Digital Twinning
Authors: Eduardo de Conto, Blaise Genest, Arvind Easwaran
Date: 2024-06-28 00:00:00

The development of digital twins (DTs) for physical systems increasingly leverages artificial intelligence (AI), particularly for combining data from different sources or for creating computationally efficient, reduced-dimension models. Indeed, even in very different application domains, twinning employs common techniques such as model order reduction and modelization with hybrid data (that is, data sourced from both physics-based models and sensors). Despite this apparent generality, current development practices are ad-hoc, making the design of AI pipelines for digital twinning complex and time-consuming. Here we propose Function+Data Flow (FDF), a domain-specific language (DSL) to describe AI pipelines within DTs. FDF aims to facilitate the design and validation of digital twins. Specifically, FDF treats functions as first-class citizens, enabling effective manipulation of models learned with AI. We illustrate the benefits of FDF on two concrete use cases from different domains: predicting the plastic strain of a structure and modeling the electromagnetic behavior of a bearing.
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
Authors: Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, Zhiqiang Shen
Date: 2024-06-28 00:00:00
Tasks: Code Generation, Code Translation

Multimodal large language models (MLLMs) have shown impressive success across modalities such as image, video, and audio in a variety of understanding and generation tasks. However, current MLLMs are surprisingly poor at understanding webpage screenshots and generating their corresponding HTML code. To address this problem, we propose $\texttt{Web2Code}$, a benchmark consisting of a new large-scale webpage-to-code dataset for instruction tuning and an evaluation framework for the webpage understanding and HTML code translation abilities of MLLMs. For dataset construction, we leverage pretrained LLMs to enhance existing webpage-to-code datasets as well as generate a diverse pool of new webpages rendered into images. Specifically, the inputs are webpage images and instructions, while the responses are the webpage's HTML code. We further include diverse natural language QA pairs about the webpage content in the responses to enable a more comprehensive understanding of the web content. To evaluate model performance in these tasks, we develop an evaluation framework for testing MLLMs' abilities in webpage understanding and web-to-code generation. Extensive experiments show that our proposed dataset is beneficial not only to our proposed tasks but also in the general visual domain. We hope our work will contribute to the development of general MLLMs suitable for web-based content generation and task automation. Our data and code are available at https://github.com/MBZUAI-LLM/web2code.
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy
Authors: Xiang Li, Cristina Mata, Jongwoo Park, Kumara Kahatapitiya, Yoo Sung Jang, Jinghuan Shang, Kanchana Ranasinghe, Ryan Burgert, Mu Cai, Yong Jae Lee, Michael S. Ryoo
Date: 2024-06-28 00:00:00
Tasks: Vision-Language-Action, World Knowledge

Vision Language Models (VLMs) have recently been leveraged to generate robotic actions, forming Vision-Language-Action (VLA) models. However, directly adapting a pretrained VLM for robotic control remains challenging, particularly when constrained by a limited number of robot demonstrations. In this work, we introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations and enables an efficient transfer of a pretrained VLM into a powerful VLA, motivated by the success of visual instruction tuning in Computer Vision. First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets, aligning robotic actions with image pixel coordinates. Further, we enhance this dataset in a self-supervised manner by defining six auxiliary tasks, without requiring any additional action annotations. We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control. Through experiments across multiple simulated and real-world tasks, we demonstrate that LLaRA achieves state-of-the-art performance while preserving the generalization capabilities of large language models. The code, datasets, and pretrained models are available at https://github.com/LostXine/LLaRA.
ProgressGym: Alignment with a Millennium of Moral Progress
Authors: Tianyi Qiu, Yang Zhang, Xuchuan Huang, Jasmine Xinze Li, Jiaming Ji, Yaodong Yang
Date: 2024-06-28 00:00:00

Frontier AI systems, including large language models (LLMs), hold increasing influence over the epistemology of human users. Such influence can reinforce prevailing societal values, potentially contributing to the lock-in of misguided moral beliefs and, consequently, the perpetuation of problematic moral practices on a broad scale. We introduce progress alignment as a technical solution to mitigate this imminent risk. Progress alignment algorithms learn to emulate the mechanics of human moral progress, thereby addressing the susceptibility of existing alignment methods to contemporary moral blindspots. To empower research in progress alignment, we introduce ProgressGym, an experimental framework allowing the learning of moral progress mechanics from history, in order to facilitate future progress in real-world moral decisions. Leveraging 9 centuries of historical text and 18 historical LLMs, ProgressGym enables codification of real-world progress alignment challenges into concrete benchmarks. Specifically, we introduce three core challenges: tracking evolving values (PG-Follow), preemptively anticipating moral progress (PG-Predict), and regulating the feedback loop between human and AI value shifts (PG-Coevolve). Alignment methods without a temporal dimension are inapplicable to these tasks. In response, we present lifelong and extrapolative algorithms as baseline methods of progress alignment, and build an open leaderboard soliciting novel algorithms and challenges. The framework and the leaderboard are available at https://github.com/PKU-Alignment/ProgressGym and https://huggingface.co/spaces/PKU-Alignment/ProgressGym-LeaderBoard respectively.
AI for Extreme Event Modeling and Understanding: Methodologies and Challenges
Authors: Gustau Camps-Valls, Miguel-Ángel Fernández-Torres, Kai-Hendrik Cohrs, Adrian Höhl, Andrea Castelletti, Aytac Pacal, Claire Robin, Francesco Martinuzzi, Ioannis Papoutsis, Ioannis Prapas, Jorge Pérez-Aracil, Katja Weigel, Maria Gonzalez-Calabuig, Markus Reichstein, Martin Rabel, Matteo Giuliani, Miguel Mahecha, Oana-Iuliana Popescu, Oscar J. Pellicer-Valero, Said Ouala, Sancho Salcedo-Sanz, Sebastian Sippel, Spyros Kondylatos, Tamara Happé, Tristan Williams
Date: 2024-06-28 00:00:00
Tasks: Disaster Response, parameter estimation, Weather Forecasting

In recent years, artificial intelligence (AI) has deeply impacted various fields, including Earth system sciences. Here, AI improved weather forecasting, model emulation, parameter estimation, and the prediction of extreme events. However, the latter comes with specific challenges, such as developing accurate predictors from noisy, heterogeneous and limited annotated data. This paper reviews how AI is being used to analyze extreme events (like floods, droughts, wildfires and heatwaves), highlighting the importance of creating accurate, transparent, and reliable AI models. We discuss the hurdles of dealing with limited data, integrating information in real-time, deploying models, and making them understandable, all crucial for gaining the trust of stakeholders and meeting regulatory needs. We provide an overview of how AI can help identify and explain extreme events more effectively, improving disaster response and communication. We emphasize the need for collaboration across different fields to create AI solutions that are practical, understandable, and trustworthy for analyzing and predicting extreme events. Such collaborative efforts aim to enhance disaster readiness and disaster risk reduction.
Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification
Authors: Anisha Gunjal, Greg Durrett
Date: 2024-06-28 00:00:00
Tasks: Fact Checking, Fact Verification, Language Modeling, Language Modelling, Large Language Model

Automatic factuality verification of large language model (LLM) generations is becoming more and more widely used to combat hallucinations. A major point of tension in the literature is the granularity of this fact-checking: larger chunks of text are hard to fact-check, but more atomic facts like propositions may lack context to interpret correctly. In this work, we assess the role of context in these atomic facts. We argue that fully atomic facts are not the right representation, and define two criteria for molecular facts: decontextuality, or how well they can stand alone, and minimality, or how little extra information is added to achieve decontexuality. We quantify the impact of decontextualization on minimality, then present a baseline methodology for generating molecular facts automatically, aiming to add the right amount of information. We compare against various methods of decontextualization and find that molecular facts balance minimality with fact verification accuracy in ambiguous settings.
Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptation
Authors: Danny Halawi, Alexander Wei, Eric Wallace, Tony T. Wang, Nika Haghtalab, Jacob Steinhardt
Date: 2024-06-28 00:00:00

Black-box finetuning is an emerging interface for adapting state-of-the-art language models to user needs. However, such access may also let malicious actors undermine model safety. To demonstrate the challenge of defending finetuning interfaces, we introduce covert malicious finetuning, a method to compromise model safety via finetuning while evading detection. Our method constructs a malicious dataset where every individual datapoint appears innocuous, but finetuning on the dataset teaches the model to respond to encoded harmful requests with encoded harmful responses. Applied to GPT-4, our method produces a finetuned model that acts on harmful instructions 99% of the time and avoids detection by defense mechanisms such as dataset inspection, safety evaluations, and input/output classifiers. Our findings question whether black-box finetuning access can be secured against sophisticated adversaries.
Electrostatics-based particle sampling and approximate inference
Authors: Yongchao Huang
Date: 2024-06-28 00:00:00
Tasks: Bayesian Inference

A new particle-based sampling and approximate inference method, based on electrostatics and Newton mechanics principles, is introduced with theoretical ground, algorithm design and experimental validation. This method simulates an interacting particle system (IPS) where particles, i.e. the freely-moving negative charges and spatially-fixed positive charges with magnitudes proportional to the target distribution, interact with each other via attraction and repulsion induced by the resulting electric fields described by Poisson's equation. The IPS evolves towards a steady-state where the distribution of negative charges conforms to the target distribution. This physics-inspired method offers deterministic, gradient-free sampling and inference, achieving comparable performance as other particle-based and MCMC methods in benchmark tasks of inferring complex densities, Bayesian logistic regression and dynamical system identification. A discrete-time, discrete-space algorithmic design, readily extendable to continuous time and space, is provided for usage in more general inference problems occurring in probabilistic machine learning scenarios such as Bayesian inference, generative modelling, and beyond.