pwc-archive/papers-with-abstracts huggingface dataset:
AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks
Authors: Chongyu Qu, Tiezheng Zhang, Hualin Qiao, Jie Liu, Yucheng Tang, Alan Yuille, Zongwei Zhou
Date: 2023-05-16 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: 8k, Active Learning, Organ Segmentation, Segmentation

Annotating medical images, particularly for organ segmentation, is laborious and time-consuming. For example, annotating an abdominal organ requires an estimated rate of 30-60 minutes per CT volume based on the expertise of an annotator and the size, visibility, and complexity of the organ. Therefore, publicly available datasets for multi-organ segmentation are often limited in data size and organ diversity. This paper proposes an active learning method to expedite the annotation process for organ segmentation and creates the largest multi-organ dataset (by far) with the spleen, liver, kidneys, stomach, gallbladder, pancreas, aorta, and IVC annotated in 8,448 CT volumes, equating to 3.2 million slices. The conventional annotation methods would take an experienced annotator up to 1,600 weeks (or roughly 30.8 years) to complete this task. In contrast, our annotation method has accomplished this task in three weeks (based on an 8-hour workday, five days a week) while maintaining a similar or even better annotation quality. This achievement is attributed to three unique properties of our method: (1) label bias reduction using multiple pre-trained segmentation models, (2) effective error detection in the model predictions, and (3) attention guidance for annotators to make corrections on the most salient errors. Furthermore, we summarize the taxonomy of common errors made by AI algorithms and annotators. This allows for continuous revision of both AI and annotations and significantly reduces the annotation costs required to create large-scale datasets for a wider variety of medical imaging tasks.
Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
Authors: Hee-Youl Kwak, Dae-Young Yun, Yongjune Kim, Sang-Hyo Kim, Jong-Seon No
Date: 2023-10-11 00:00:00
Proceeding: NeurIPS 2023 11
Tasks: Decoder

Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .
PDRAS: Priority‐Based Dynamic Resource Allocation Scheme in 5G Network Slicing
Authors: Haneul Ko1, Jaewook Lee2, Sangheon Pack
Date: 2022-08-09 00:00:00
Proceeding: Journal of Network and Systems Management 2022 8
Tasks: Management

In 5G network slicing environments, if insufficient resources are allocated to the associated resource-intensive service slices, its quality of service (QoS) can be con- siderably degraded. In this paper, we propose a priority-based dynamic resource allocation scheme (PDRAS), in which a resource management agent maintains slic- ing information such as priorities, demand profiles, and average resource adjustment time to change allocated resources to slices. To maximize QoS of slices while main- taining the total amount of allocated resources below a certain level, a constrained Markov decision process problem is formulated and the optimal allocation policy is obtained using linear programming. Extensive evaluation results demonstrate that PDRAS with the optimal policy has better performance regarding QoS and the resource usage efficiency compared with other schemes.
Investigating Shallow and Deep Learning Techniques for Emotion Classification in Short Persian Texts
Authors: Mahdi Rasouli, Vahid Kiani
Date: 2023-12-16 00:00:00
Proceeding: Journal of AI and Data Mining 2023 12
Tasks: Deep Learning, Dimensionality Reduction, Emotion Classification, Transfer Learning

The identification of emotions in short texts of low-resource languages poses a significant challenge, requiring specialized frameworks and computational intelligence techniques. This paper presents a comprehensive exploration of shallow and deep learning methods for emotion detection in short Persian texts. Shallow learning methods employ feature extraction and dimension reduction to enhance classification accuracy. On the other hand, deep learning methods utilize transfer learning and word embedding, particularly BERT, to achieve high classification accuracy. A Persian dataset called "ShortPersianEmo" is introduced to evaluate the proposed methods, comprising 5472 diverse short Persian texts labeled in five main emotion classes. The evaluation results demonstrate that transfer learning and BERT-based text embedding perform better in accurately classifying short Persian texts than alternative approaches. The dataset of this study ShortPersianEmo will be publicly available online at https://github.com/vkiani/ShortPersianEmo.
PHILNet: A Novel Efficient Approach for Time Series Forecasting using Deep Learning
Authors: Manuel Jesús Jiménez Navarro, María Martínez Ballesteros, Francisco Martínez Álvarez, Gualberto Asencio Cortés
Date: 2023-06-01 00:00:00
Proceeding: Information Sciences 2023 6
Tasks: Time Series, Time Series Forecasting, Univariate Time Series Forecasting

Time series is one of the most common data types in the industry nowadays. Forecasting the future of a time series behavior can be useful to plan ahead, save time, resources, and help avoid undesired scenarios. To make the forecasting, historical data is utilized due to the causal nature of the time series. Several deep learning algorithms have been presented in this area, where the input is processed through a series of nonlinear functions to produce the output. We present a novel strategy to improve the performance of deep learning models in time series forecasting in terms of efficiency while reaching similar effectiveness. This approach separates the model into levels, starting with the easiest and continuing to the most difficult. The simpler levels deal with smoothed versions of the input, whereas the most sophisticated level deals with the raw data. This strategy seeks to mimic the human learning process, in which basic tasks are completed initially, followed by more precise and sophisticated ones. Our method achieved promising results and outperformed LSTM and GRU models in a variety of time series.
Embedded Temporal Feature Selection for Time Series Forecasting Using Deep Learning
Authors: Manuel Jesús Jiménez-Navarro, María Martínez-Ballesteros, Francisco Martínez-Álvarez, Gualberto Asencio-Cortés
Date: 2023-10-01 00:00:00
Proceeding: International Work-Conference on Artificial Neural Networks 2023 10
Tasks: Deep Learning, feature selection, Multivariate Time Series Forecasting, Time Series, Time Series Forecasting

Traditional time series forecasting models often use all available variables, including potentially irrelevant or noisy features, which can lead to overfitting and poor performance. Feature selection can help address this issue by selecting the most informative variables in the temporal and feature dimensions. However, selecting the right features can be challenging for time series models. Embedded feature selection has been a popular approach, but many techniques do not include it in their design, including deep learning methods, which can lead to less efficient and effective feature selection. This paper presents a deep learning-based method for time series forecasting that incorporates feature selection to improve model efficacy and interpretability. The proposed method uses a multidimensional layer to remove irrelevant features along the temporal dimension. The resulting model is compared to several feature selection methods and experimental results demonstrate that the proposed approach can improve forecasting accuracy while reducing model complexity.
A New Deep Learning Architecture withInductive Bias Balance for Transformer Oil Temperature Forecasting
Authors: Manuel Jesús Jiménez-Navarro, María Martínez-Ballesteros, Francisco Martínez-Álvarez & Gualberto Asencio-Cortés
Date: 2023-05-28 00:00:00
Proceeding: Journal of Big Data 2023 5
Tasks: Inductive Bias, Time Series, Time Series Forecasting, Univariate Time Series Forecasting

Ensuring optimal performance of power transformers is a laborious task, where the insulation system is essential to decrease their deterioration. The insulation system uses the insulate oil required to control temperature. High temperatures may reduce the lifetime of transformers, leading to expensive maintenance. Deep learning architectures have been shown to obtain remarkable results in a wide range of fields. However, this improvement usually comes with an increase in computing resources, which increases the carbon footprint and hinders the optimization of the architectures. In this work, we develop a new deep learning architecture that obtain an efficacy which compete with the best current architectures in transformer oil temperature forecasting while improve the efficacy. Effective forecasting can help prevent high temperatures and monitor the future condition of power transformers, avoiding unnecessary waste. We attempt to balance the inductive bias included in our architecture through the proposed Smooth Residual Block. This mechanism divides the original problem into multiple subproblems, obtaining different representations of the time series, which collaboratively obtain the final forecasting. Our architecture is applied to the Electricity Transformer datasets, which obtain transformer insulate oil temperature measures from two transformers in China. The results achieve a 13% improvement in MSE and a 57% improvement in performance compared to, as far as we know, the best current architectures. Additionally, we analyze the behavior learned by this architecture to obtain an intuitive interpretation of the achieved solution.
Robust Calibration For Improved Weather Prediction Under Distributional Shift
Authors: Sankalp Gilda, Neel Bhandari, Wendy Mak, Andrea Panizza
Date: 2024-01-08 00:00:00
Tasks: Data Augmentation, Mixture-of-Experts, Prediction

In this paper, we present results on improving out-of-domain weather prediction and uncertainty estimation as part of the \texttt{Shifts Challenge on Robustness and Uncertainty under Real-World Distributional Shift} challenge. We find that by leveraging a mixture of experts in conjunction with an advanced data augmentation technique borrowed from the computer vision domain, in conjunction with robust \textit{post-hoc} calibration of predictive uncertainties, we can potentially achieve more accurate and better-calibrated results with deep neural networks than with boosted tree models for tabular data. We quantify our predictions using several metrics and propose several future lines of inquiry and experimentation to boost performance.
DeepPhysiNet: Bridging Deep Learning and Atmospheric Physics for Accurate and Continuous Weather Modeling
Authors: Wenyuan Li, Zili Liu, Keyan Chen, Hao Chen, Shunlin Liang, Zhengxia Zou, Zhenwei Shi
Date: 2024-01-04 00:00:00
Tasks: Weather Forecasting

Accurate weather forecasting holds significant importance to human activities. Currently, there are two paradigms for weather forecasting: Numerical Weather Prediction (NWP) and Deep Learning-based Prediction (DLP). NWP utilizes atmospheric physics for weather modeling but suffers from poor data utilization and high computational costs, while DLP can learn weather patterns from vast amounts of data directly but struggles to incorporate physical laws. Both paradigms possess their respective strengths and weaknesses, and are incompatible, because physical laws adopted in NWP describe the relationship between coordinates and meteorological variables, while DLP directly learns the relationships between meteorological variables without consideration of coordinates. To address these problems, we introduce the DeepPhysiNet framework, incorporating physical laws into deep learning models for accurate and continuous weather system modeling. First, we construct physics networks based on multilayer perceptrons (MLPs) for individual meteorological variable, such as temperature, pressure, and wind speed. Physics networks establish relationships between variables and coordinates by taking coordinates as input and producing variable values as output. The physical laws in the form of Partial Differential Equations (PDEs) can be incorporated as a part of loss function. Next, we construct hyper-networks based on deep learning methods to directly learn weather patterns from a large amount of meteorological data. The output of hyper-networks constitutes a part of the weights for the physics networks. Experimental results demonstrate that, upon successful integration of physical laws, DeepPhysiNet can accomplish multiple tasks simultaneously, not only enhancing forecast accuracy but also obtaining continuous spatiotemporal resolution results, which is unattainable by either the NWP or DLP.
Generation Z's Ability to Discriminate Between AI-generated and Human-Authored Text on Discord
Authors: Dhruv Ramu, Rishab Jain, Aditya Jain
Date: 2023-12-31 00:00:00
Tasks: Misinformation

The growing popularity of generative artificial intelligence (AI) chatbots such as ChatGPT is having transformative effects on social media. As the prevalence of AI-generated content grows, concerns have been raised regarding privacy and misinformation online. Among social media platforms, Discord enables AI integrations -- making their primarily "Generation Z" userbase particularly exposed to AI-generated content. We surveyed Generation Z aged individuals (n = 335) to evaluate their proficiency in discriminating between AI-generated and human-authored text on Discord. The investigation employed one-shot prompting of ChatGPT, disguised as a text message received on the Discord.com platform. We explore the influence of demographic factors on ability, as well as participants' familiarity with Discord and artificial intelligence technologies. We find that Generation Z individuals are unable to discern between AI and human-authored text (p = 0.011), and that those with lower self-reported familiarity with Discord demonstrated an improved ability in identifying human-authored compared to those with self-reported experience with AI (p << 0.0001). Our results suggest that there is a nuanced relationship between AI technology and popular modes of communication for Generation Z, contributing valuable insights into human-computer interactions, digital communication, and artificial intelligence literacy.
LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training
Authors: Khoi M. Le, Trinh Pham, Tho Quan, Anh Tuan Luu
Date: 2024-01-09 00:00:00
Tasks: Multilingual Paraphrase Generation, Sentence

Paraphrases are texts that convey the same meaning while using different words or sentence structures. It can be used as an automatic data augmentation tool for many Natural Language Processing tasks, especially when dealing with low-resource languages, where data shortage is a significant problem. To generate a paraphrase in multilingual settings, previous studies have leveraged the knowledge from the machine translation field, i.e., forming a paraphrase through zero-shot machine translation in the same language. Despite good performance on human evaluation, those methods still require parallel translation datasets, thus making them inapplicable to languages that do not have parallel corpora. To mitigate that problem, we proposed the first unsupervised multilingual paraphrasing model, LAMPAT ($\textbf{L}$ow-rank $\textbf{A}$daptation for $\textbf{M}$ultilingual $\textbf{P}$araphrasing using $\textbf{A}$dversarial $\textbf{T}$raining), by which monolingual dataset is sufficient enough to generate a human-like and diverse sentence. Throughout the experiments, we found out that our method not only works well for English but can generalize on unseen languages as well. Data and code are available at https://github.com/VinAIResearch/LAMPAT.
Generic Knowledge Boosted Pre-training For Remote Sensing Images
Authors: Ziyue Huang, Mingming Zhang, Yuan Gong, Qingjie Liu, Yunhong Wang
Date: 2024-01-09 00:00:00
Tasks: Change Detection, Deep Learning, General Knowledge, object-detection, Object Detection, Scene Classification, Semantic Segmentation

Deep learning models are essential for scene classification, change detection, land cover segmentation, and other remote sensing image understanding tasks. Most backbones of existing remote sensing deep learning models are typically initialized by pre-trained weights obtained from ImageNet pre-training (IMP). However, domain gaps exist between remote sensing images and natural images (e.g., ImageNet), making deep learning models initialized by pre-trained weights of IMP perform poorly for remote sensing image understanding. Although some pre-training methods are studied in the remote sensing community, current remote sensing pre-training methods face the problem of vague generalization by only using remote sensing images. In this paper, we propose a novel remote sensing pre-training framework, Generic Knowledge Boosted Remote Sensing Pre-training (GeRSP), to learn robust representations from remote sensing and natural images for remote sensing understanding tasks. GeRSP contains two pre-training branches: (1) A self-supervised pre-training branch is adopted to learn domain-related representations from unlabeled remote sensing images. (2) A supervised pre-training branch is integrated into GeRSP for general knowledge learning from labeled natural images. Moreover, GeRSP combines two pre-training branches using a teacher-student architecture to simultaneously learn representations with general and special knowledge, which generates a powerful pre-trained model for deep learning model initialization. Finally, we evaluate GeRSP and other remote sensing pre-training methods on three downstream tasks, i.e., object detection, semantic segmentation, and scene classification. The extensive experimental results consistently demonstrate that GeRSP can effectively learn robust representations in a unified manner, improving the performance of remote sensing downstream tasks.
Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation
Authors: Xiyi Chen, Marko Mihajlovic, Shaofei Wang, Sergey Prokudin, Siyu Tang
Date: 2024-01-09 00:00:00
Proceeding: CVPR 2024 1
Tasks: Novel View Synthesis

Recent advances in generative diffusion models have enabled the previously unfeasible capability of generating 3D assets from a single input image or a text prompt. In this work, we aim to enhance the quality and functionality of these models for the task of creating controllable, photorealistic human avatars. We achieve this by integrating a 3D morphable model into the state-of-the-art multi-view-consistent diffusion approach. We demonstrate that accurate conditioning of a generative pipeline on the articulated 3D model enhances the baseline model performance on the task of novel view synthesis from a single image. More importantly, this integration facilitates a seamless and accurate incorporation of facial expression and body pose control into the generation process. To the best of our knowledge, our proposed framework is the first diffusion model to enable the creation of fully 3D-consistent, animatable, and photorealistic human avatars from a single image of an unseen subject; extensive quantitative and qualitative evaluations demonstrate the advantages of our approach over existing state-of-the-art avatar creation models on both novel view and novel expression synthesis tasks. The code for our project is publicly available.
RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation
Authors: Mahdi Nikdan, Soroush Tabesh, Elvir Crnčević, Dan Alistarh
Date: 2024-01-09 00:00:00
Tasks: GPU, Math, parameter-efficient fine-tuning, Quantization

We investigate parameter-efficient fine-tuning (PEFT) methods that can provide good accuracy under limited computational and memory budgets in the context of large language models (LLMs). We present a new PEFT method called Robust Adaptation (RoSA) inspired by robust principal component analysis that jointly trains $\textit{low-rank}$ and $\textit{highly-sparse}$ components on top of a set of fixed pretrained weights to efficiently approximate the performance of a full-fine-tuning (FFT) solution. Across a series of challenging generative tasks such as grade-school math and SQL query generation, which require fine-tuning for good performance, we show that RoSA outperforms LoRA, pure sparse fine-tuning, and alternative hybrid methods at the same parameter budget, and can even recover the performance of FFT on some tasks. We provide system support for RoSA to complement the training algorithm, specifically in the form of sparse GPU kernels which enable memory- and computationally-efficient training, and show that it is also compatible with low-precision base weights, resulting in the first joint representation combining quantization, low-rank and sparse approximations. Our code is available at https://github.com/IST-DASLab/RoSA.
Benchmark Analysis of Various Pre-trained Deep Learning Models on ASSIRA Cats and Dogs Dataset
Authors: Galib Muhammad Shahriar Himel, Md. Masudul Islam
Date: 2024-01-09 00:00:00
Tasks: Benchmarking, image-classification, Image Classification

As the most basic application and implementation of deep learning, image classification has grown in popularity. Various datasets are provided by renowned data science communities for benchmarking machine learning algorithms and pre-trained models. The ASSIRA Cats & Dogs dataset is one of them and is being used in this research for its overall acceptance and benchmark standards. A comparison of various pre-trained models is demonstrated by using different types of optimizers and loss functions. Hyper-parameters are changed to gain the best result from a model. By applying this approach, we have got higher accuracy without major changes in the training model. To run the experiment, we used three different computer architectures: a laptop equipped with NVIDIA GeForce GTX 1070, a laptop equipped with NVIDIA GeForce RTX 3080Ti, and a desktop equipped with NVIDIA GeForce RTX 3090. The acquired results demonstrate supremacy in terms of accuracy over the previously done experiments on this dataset. From this experiment, the highest accuracy which is 99.65% is gained using the NASNet Large.