pwc-archive/papers-with-abstracts huggingface dataset:
ERIC: Estimating Rainfall with Commodity Doorbell Camera for Precision Residential Irrigation
Authors: Tian Liu, Liuyi Jin, Radu Stoleru, Amran Haroon, Charles Swanson, Kexin Feng
Date: 2024-09-19 00:00:00
Tasks: Raspberry Pi 4

Current state-of-the-art residential irrigation systems, such as WaterMyYard, rely on rainfall data from nearby weather stations to adjust irrigation amounts. However, the accuracy of rainfall data is compromised by the limited spatial resolution of rain gauges and the significant variability of hyperlocal rainfall, leading to substantial water waste. To improve irrigation efficiency, we developed a cost-effective irrigation system, dubbed ERIC, which employs machine learning models to estimate rainfall from commodity doorbell camera footage and optimizes irrigation schedules without human intervention. Specifically, we: a) designed novel visual and audio features with lightweight neural network models to infer rainfall from the camera at the edge, preserving user privacy; b) built a complete end-to-end irrigation system on Raspberry Pi 4, costing only \$75. We deployed the system across five locations (collecting over 750 hours of video) with varying backgrounds and light conditions. Comprehensive evaluation validates that ERIC achieves state-of-the-art rainfall estimation performance ($\sim$ 5mm/day), saving 9,112 gallons/month of water, translating to \$28.56/month in utility savings. Data and code are available at https://github.com/LENSS/ERIC-BuildSys2024.git
CVT-Occ: Cost Volume Temporal Fusion for 3D Occupancy Prediction
Authors: Zhangchen Ye, Tao Jiang, Chenfeng Xu, Yiming Li, Hang Zhao
Date: 2024-09-20 00:00:00
Tasks: Depth Estimation, Prediction

Vision-based 3D occupancy prediction is significantly challenged by the inherent limitations of monocular vision in depth estimation. This paper introduces CVT-Occ, a novel approach that leverages temporal fusion through the geometric correspondence of voxels over time to improve the accuracy of 3D occupancy predictions. By sampling points along the line of sight of each voxel and integrating the features of these points from historical frames, we construct a cost volume feature map that refines current volume features for improved prediction outcomes. Our method takes advantage of parallax cues from historical observations and employs a data-driven approach to learn the cost volume. We validate the effectiveness of CVT-Occ through rigorous experiments on the Occ3D-Waymo dataset, where it outperforms state-of-the-art methods in 3D occupancy prediction with minimal additional computational cost. The code is released at \url{https://github.com/Tsinghua-MARS-Lab/CVT-Occ}.
pyrtklib: An open-source package for tightly coupled deep learning and GNSS integration for positioning in urban canyons
Authors: Runzhi Hu, Penghui Xu, Yihan Zhong, Weisong Wen
Date: 2024-09-19 00:00:00
Tasks: Deep Learning

Artificial intelligence (AI) is revolutionizing numerous fields, with increasing applications in Global Navigation Satellite Systems (GNSS) positioning algorithms in intelligent transportation systems (ITS) via deep learning. However, a significant technological disparity exists as traditional GNSS algorithms are often developed in Fortran or C, contrasting with the Python-based implementation prevalent in deep learning tools. To address this discrepancy, this paper introduces pyrtklib, a Python binding for the widely utilized open-source GNSS tool, RTKLIB. This binding makes all RTKLIB functionalities accessible in Python, facilitating seamless integration. Moreover, we present a deep learning subsystem under pyrtklib, which is a novel deep learning framework that leverages pyrtklib to accurately predict weights and biases within the GNSS positioning process. The use of pyrtklib enables developers to easily and quickly prototype and implement deep learning-aided GNSS algorithms, showcasing its potential to enhance positioning accuracy significantly.
Diabetica: Adapting Large Language Model to Enhance Multiple Medical Tasks in Diabetes Care and Management
Authors: Lai Wei, Zhen Ying, Muyang He, Yutong Chen, Qian Yang, Yanzhe Hong, Jiaping Lu, Kaipeng Zheng, Shaoting Zhang, Xiaoying Li, Weiran Huang, Ying Chen
Date: 2024-09-20 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Management

Diabetes is a chronic disease with a significant global health burden, requiring multi-stakeholder collaboration for optimal management. Large language models (LLMs) have shown promise in various healthcare scenarios, but their effectiveness across diverse diabetes tasks remains unproven. Our study introduced a framework to train and validate diabetes-specific LLMs. We first developed a comprehensive data processing pipeline that includes data collection, filtering, augmentation and refinement. This created a high-quality, diabetes-specific dataset and evaluation benchmarks from scratch. Fine-tuned on the collected training dataset, our diabetes-specific LLM family demonstrated state-of-the-art proficiency in processing various diabetes tasks compared to other LLMs. Furthermore, clinical studies revealed the potential applications of our models in diabetes care, including providing personalized healthcare, assisting medical education, and streamlining clinical tasks. Generally, our introduced framework helps develop diabetes-specific LLMs and highlights their potential to enhance clinical practice and provide personalized, data-driven support for diabetes management across different end users. Our codes, benchmarks and models are available at https://github.com/waltonfuture/Diabetica.
Deep learning and machine learning techniques for head pose estimation: a survey
Authors: Redhwan Algabri, Ahmed Abdu, Sungon Lee
Date: 2024-09-12 00:00:00
Proceeding: Artificial Intelligence Review 2024 9
Tasks: Articles, Head Pose Estimation, Pose Estimation, Survey

Head pose estimation (HPE) has been extensively investigated over the past decade due to its wide range of applications across several domains of artificial intelligence (AI), resulting in progressive improvements in accuracy. The problem becomes more challenging when the application requires full-range angles, particularly in unconstrained environments, making HPE an active research topic. This paper presents a comprehensive survey of recent AI-based HPE tasks in digital images. We also propose a novel taxonomy based on the main steps to implement each method, broadly dividing these steps into eleven categories under four groups. Moreover, we provide the pros and cons of ten categories of the overall system. Finally, this survey sheds some light on the public datasets, available codes, and future research directions, aiding readers and aspiring researchers in identifying robust methods that exhibit a strong baseline within the subcategory for further exploration in this fascinating area. The review compared and analyzed 113 articles published between 2018 and 2024, distributing 70.5% deep learning, 24.1% machine learning, and 5.4% hybrid approaches. Furthermore, it included 101 articles related to datasets, definitions, and other elements for AI-based HPE systems published over the last two decades. To the best of our knowledge, this is the first paper that aims to survey HPE strategies based on artificial intelligence, with detailed explanations of the main steps to implement each method. A regularly updated project page is provided: (https://github.com/Redhwan-A/SurveyPHE/blob/main/README.md).
Head Pose Estimation Based on 5D Rotation Representation
Authors: Redhwan Algabri; Sungon Lee
Date: 2024-09-03 00:00:00
Proceeding: IEEE Symposium on Wireless Technology and Applications (ISWTA 2024 9
Tasks: Head Pose Estimation, Pose Estimation

Head pose estimation (HPE) is a crucial problem in computer vision, as it significantly enhances the performance of face-related tasks involving a frontal view. However, recent applications demand head analysis across the entire 360° range, which poses significant challenges. This paper introduces an end-to-end method for an HPE task. We propose a continuous 5D rotation representation to address the challenge of discontinuous rotation called 5DResNet, which enables robust and efficient direct regression of the head pose. The proposed method adopted the inverse of the stereographic projection (ISP) with the Gram-Schmidt mapping to orthogonalization procedure in the network. This approach allows our model to learn the full-range angles, exceeding the abilities of most previous techniques that confine pose estimation to a limited angle range to achieve acceptable results. Furthermore, we present an ablation study to gain a deeper understanding of the factors influencing the performance of our method. Our proposed approach demonstrates notable competition over other state-of-the-art methods in comprehensive experiments conducted on the publicly available Carnegie Mellon University (CMU) dataset, which achieved error rates of 5.97° and 6.64° for narrow and full-range angles, respectively.
Tackling fluffy clouds: field boundaries detection using time series of S2 and/or S1 imagery
Authors: Foivos I. Diakogiannis, Zheng-Shu Zhou, Jeff Wang, Gonzalo Mata, Dave Henry, Roger Lawes, Amy Parker, Peter Caccetta, Rodrigo Ibata, Ondrej Hlinka, Jonathan Richetti, Kathryn Batchelor, Chris Herrmann, Andrew Toovey, John Taylor
Date: 2024-09-20 00:00:00
Tasks: Field Boundary Delineation, Time Series

Accurate field boundary delineation is a critical challenge in digital agriculture, impacting everything from crop monitoring to resource management. Existing methods often struggle with noise and fail to generalize across varied landscapes, particularly when dealing with cloud cover in optical remote sensing. In response, this study presents a new approach that leverages time series data from Sentinel-2 (S2) and Sentinel-1 (S1) imagery to improve performance under diverse cloud conditions, without the need for manual cloud filtering. We introduce a 3D Vision Transformer architecture specifically designed for satellite image time series, incorporating a memory-efficient attention mechanism. Two models are proposed: PTAViT3D, which handles either S2 or S1 data independently, and PTAViT3D-CA, which fuses both datasets to enhance accuracy. Both models are evaluated under sparse and dense cloud coverage by exploiting spatio-temporal correlations. Our results demonstrate that the models can effectively delineate field boundaries, even with partial (S2 or S2 and S1 data fusion) or dense cloud cover (S1), with the S1-based model providing performance comparable to S2 imagery in terms of spatial resolution. A key strength of this approach lies in its capacity to directly process cloud-contaminated imagery by leveraging spatio-temporal correlations in a memory-efficient manner. This methodology, used in the ePaddocks product to map Australia's national field boundaries, offers a robust, scalable solution adaptable to varying agricultural environments, delivering precision and reliability where existing methods falter. Our code is available at https://github.com/feevos/tfcl.
Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models
Authors: Venkat Karramreddy, Liam Mitchell
Date: 2024-09-20 00:00:00
Tasks: Camera Calibration, Deep Learning, Multimodal Deep Learning, Sensor Fusion

This article presents an innovative study in exploring, evaluating, and implementing deep learning architectures for the calibration of multi-modal sensor systems. The focus behind this is to leverage the use of sensor fusion to achieve dynamic, real-time alignment between 3D LiDAR and 2D Camera sensors. static calibration methods are tedious and time-consuming, which is why we propose utilizing Conventional Neural Networks (CNN) coupled with geometrically informed learning to solve this issue. We leverage the foundational principles of Extrinsic LiDAR-Camera Calibration tools such as RegNet, CalibNet, and LCCNet by exploring open-source models that are available online and comparing our results with their corresponding research papers. Requirements for extracting these visual and measurable outputs involved tweaking source code, fine-tuning, training, validation, and testing for each of these frameworks for equal comparisons. This approach aims to investigate which of these advanced networks produces the most accurate and consistent predictions. Through a series of experiments, we reveal some of their shortcomings and areas for potential improvements along the way. We find that LCCNet yields the best results out of all the models that we validated.
A preliminary study on continual learning in computer vision using Kolmogorov-Arnold Networks
Authors: Alessandro Cacciatore, Valerio Morelli, Federica Paganica, Emanuele Frontoni, Lucia Migliorelli, Daniele Berardini
Date: 2024-09-20 00:00:00
Tasks: class-incremental learning, Class Incremental Learning, Continual Learning, Incremental Learning, Kolmogorov-Arnold Networks

Deep learning has long been dominated by multi-layer perceptrons (MLPs), which have demonstrated superiority over other optimizable models in various domains. Recently, a new alternative to MLPs has emerged - Kolmogorov-Arnold Networks (KAN)- which are based on a fundamentally different mathematical framework. According to their authors, KANs address several major issues in MLPs, such as catastrophic forgetting in continual learning scenarios. However, this claim has only been supported by results from a regression task on a toy 1D dataset. In this paper, we extend the investigation by evaluating the performance of KANs in continual learning tasks within computer vision, specifically using the MNIST datasets. To this end, we conduct a structured analysis of the behavior of MLPs and two KAN-based models in a class-incremental learning scenario, ensuring that the architectures involved have the same number of trainable parameters. Our results demonstrate that an efficient version of KAN outperforms both traditional MLPs and the original KAN implementation. We further analyze the influence of hyperparameters in MLPs and KANs, as well as the impact of certain trainable parameters in KANs, such as bias and scale weights. Additionally, we provide a preliminary investigation of recent KAN-based convolutional networks and compare their performance with that of traditional convolutional neural networks. Our codes can be found at https://github.com/MrPio/KAN-Continual_Learning_tests.
Performance and Power: Systematic Evaluation of AI Workloads on Accelerators with CARAML
Authors: Chelsea Maria John, Stepan Nassyr, Carolin Penke, Andreas Herten
Date: 2024-09-19 00:00:00

The rapid advancement of machine learning (ML) technologies has driven the development of specialized hardware accelerators designed to facilitate more efficient model training. This paper introduces the CARAML benchmark suite, which is employed to assess performance and energy consumption during the training of transformer-based large language models and computer vision models on a range of hardware accelerators, including systems from NVIDIA, AMD, and Graphcore. CARAML provides a compact, automated, extensible, and reproducible framework for assessing the performance and energy of ML workloads across various novel hardware architectures. The design and implementation of CARAML, along with a custom power measurement tool called jpwr, are discussed in detail.
Feature-Centered First Order Structure Tensor Scale-Space in 2D and 3D
Authors: Pawel Tomasz Pieta, Anders Bjorholm Dahl, Jeppe Revall Frisvad, Siavash Arjomand Bigdeli, Anders Nymark Christensen
Date: 2024-09-20 00:00:00

The structure tensor method is often used for 2D and 3D analysis of imaged structures, but its results are in many cases very dependent on the user's choice of method parameters. We simplify this parameter choice in first order structure tensor scale-space by directly connecting the width of the derivative filter to the size of image features. By introducing a ring-filter step, we substitute the Gaussian integration/smoothing with a method that more accurately shifts the derivative filter response from feature edges to their center. We further demonstrate how extracted structural measures can be used to correct known inaccuracies in the scale map, resulting in a reliable representation of the feature sizes both in 2D and 3D. Compared to the traditional first order structure tensor, or previous structure tensor scale-space approaches, our solution is much more accurate and can serve as an out-of-the-box method for extracting a wide range of structural parameters with minimal user input.
Cross-Model Cross-Stream Learning for Self-Supervised Human Action Recognition
Authors: Liu, Mengyuan; Liu, Hong; Guo, Tianyu
Date: 2024-09-23 00:00:00
Proceeding: IEEE Transactions on Human-Machine Systems 2024 9
Tasks: Action Recognition, Contrastive Learning, Ensemble Learning, Pseudo Label, Representation Learning, Self-Supervised Human Action Recognition, Self-supervised Skeleton-based Action Recognition, Skeleton Based Action Recognition, Temporal Action Localization

Considering the instance-level discriminative ability, contrastive learning methods, including MoCo and SimCLR, have been adapted from the original image representation learning task to solve the self-supervised skeleton-based action recognition task. These methods usually use multiple data streams (i.e., joint, motion, and bone) for ensemble learning, meanwhile, how to construct a discriminative feature space within a single stream and effectively aggregate the information from multiple streams remains an open problem. To this end, this paper first applies a new contrastive learning method called BYOL to learn from skeleton data, and then formulate SkeletonBYOL as a simple yet effective baseline for self-supervised skeleton-based action recognition. Inspired by SkeletonBYOL, this paper further presents a Cross-Model and Cross-Stream (CMCS) framework. This framework combines Cross-Model Adversarial Learning (CMAL) and Cross-Stream Collaborative Learning (CSCL). Specifically, CMAL learns single-stream representation by cross-model adversarial loss to obtain more discriminative features. To aggregate and interact with multi-stream information, CSCL is designed by generating similarity pseudo label of ensemble learning as supervision and guiding feature generation for individual streams. Extensive experiments on three datasets verify the complementary properties between CMAL and CSCL and also verify that {the proposed method} can achieve better results than state-of-the-art methods using various evaluation protocols.
The Pricing of Skewness Over Different Return Horizons
Authors: Kevin Aretz, Eser Arisoy
Date: 2020-06-15 00:00:00
Proceeding: Journal of Banking and Finance 2020 6

While recent theoretical and empirical work suggests that the physical skewness of a stock’s future discrete return distribution prices stocks, it does not tell us over which return horizon(s) that physical skewness is priced. Developing a novel block bootstrap estimator that allows us to calculate realized return skewness over arbitrary horizons, we aim to identify those return horizons. In doing so, we first show that our block bootstrap estimator produces more accurate realized skewness estimates than other recent estimators do. Next, we report that the existing skewness proxies used in the empirical asset pricing literature differ in how well they predict skewness over short or long return horizons. Finally, we reveal that the skewness pricing evidence documented in the empirical asset pricing literature is mostly driven by skewness over short (and not long) return horizons.
Generative Aerodynamic Design with Diffusion Probabilistic Models
Authors: Thomas Wagenaar, Simone Mancini, Andrés Mateo-Gabín
Date: 2024-09-20 00:00:00

The optimization of geometries for aerodynamic design often relies on a large number of expensive simulations to evaluate and iteratively improve the geometries. It is possible to reduce the number of simulations by providing a starting geometry that has properties close to the desired requirements, often in terms of lift and drag, aerodynamic moments and surface areas. We show that generative models have the potential to provide such starting geometries by generalizing geometries over a large dataset of simulations. In particular, we leverage diffusion probabilistic models trained on XFOIL simulations to synthesize two-dimensional airfoil geometries conditioned on given aerodynamic features and constraints. The airfoils are parameterized with Bernstein polynomials, ensuring smoothness of the generated designs. We show that the models are able to generate diverse candidate designs for identical requirements and constraints, effectively exploring the design space to provide multiple starting points to optimization procedures. However, the quality of the candidate designs depends on the distribution of the simulated designs in the dataset. Importantly, the geometries in this dataset must satisfy other requirements and constraints that are not used in conditioning of the diffusion model, to ensure that the generated geometries are physical.
SIM-OFE: Structure Information Mining and Object-aware Feature Enhancement for Fine-Grained Visual Categorization
Authors: Hongbo Sun, Xiangteng He, Jinglin Xu, Yuxin Peng
Date: 2024-09-18 00:00:00
Proceeding: journal 2024 9
Tasks: Fine-Grained Image Classification, Fine-Grained Visual Categorization, Object

Fine-grained visual categorization (FGVC) aims to distinguish visual objects from multiple subcategories of the coarse-grained category. Subtle inter-class differences among various subcategories make the FGVC task more challenging. Existing methods primarily focus on learning salient visual patterns while ignoring how to capture the object’s internal structure, causing difficulty in obtaining complete discriminative regions within the object to limit FGVC performance. To address the above issue, we propose a Structure Information Mining and Object-aware Feature Enhancement (SIM-OFE) method for finegrained visual categorization, which explores the visual object’s internal structure composition and appearance traits. Concretely, we first propose a simple yet effective hybrid perception attention module for locating visual objects based on global-scope and local-scope significance analyses. Then, a structure information mining module is proposed to model the distribution and context relation of critical regions within the object, highlighting the whole object and discriminative regions for distinguishing subtle differences. Finally, an object-aware feature enhancement module is proposed to combine global-scope and local-scope discriminative features in an attentive coupling way for powerful visual representations in fine-grained recognition. Extensive experiments on three FGVC benchmark datasets demonstrate that our proposed SIM-OFE method can achieve state-of-the-art performance.