pwc-archive/papers-with-abstracts huggingface dataset:
Dual-feature Fusion Attention Network for Small Object Segmentation
Authors: Xin Fei a, Xiaojie Li a, 1, Canghong Shi b, Hongping Ren a, Imran Mumtaz c, Jun Guo d, Yu Wu e, Yong luo f, ∗, Jiancheng Lv g, Xi Wu a
Date: 2023-04-27 00:00:00
Proceeding: Computers in Biology and Medicine 2023 4
Tasks: Object, Segmentation, Semantic Segmentation

Accurate segmentation of medical images is an important step during radiotherapy planning and clinical
diagnosis. However, manually marking organ or lesion boundaries is tedious, time-consuming, and prone to
error due to subjective variability of radiologist. Automatic segmentation remains a challenging task owing
to the variation (in shape and size) across subjects. Moreover, existing convolutional neural networks based
methods perform poorly in small medical objects segmentation due to class imbalance and boundary ambiguity.
In this paper, we propose a dual feature fusion attention network (DFF-Net) to improve the segmentation
accuracy of small objects. It mainly includes two core modules: the dual-branch feature fusion module (DFFM)
and the reverse attention context module (RACM). We first extract multi-resolution features by multi-scale
feature extractor, then construct DFFM to aggregate the global and local contextual information to achieve
information complementarity among features, which provides sufficient guidance for accurate small objects
segmentation. Moreover, to alleviate the degradation of segmentation accuracy caused by blurred medical
image boundaries, we propose RACM to enhance the edge texture of features. Experimental results on datasets
NPC, ACDC, and Polyp demonstrate that our proposed method has fewer parameters, faster inference, and
lower model complexity, and achieves better accuracy than more state-of-the-art methods.
A hybrid integrated deep learning model for predicting various air pollutants
Authors: Wenjing MaoORCID Icon, Limin JiaoORCID Icon, Weilin Wang, Jianlong Wang, Xueli Tong &Suli Zhao
Date: 2021-09-29 00:00:00
Proceeding: journal 2021 9
Tasks: Deep Learning

Air pollution is a significant urban issue, with practical applications for pollution control, urban environmental management planning, and urban construction. However, owing to the complexity and differences in spatiotemporal changes for various types of pollution, it is challenging to establish a framework that can capture the spatiotemporal correlations of different types of air pollution and obtain high prediction accuracy. In this paper, we proposed a deep learning framework suitable for predicting various air pollutants: a graph convolutional temporal sliding long short-term memory (GT-LSTM) model. The hybrid integrated model combines graph convolutional networks and long short-term networks based on a strategy with temporal sliding. Herein, the graph convolution networks gather neighbor information for spatial dependency modeling based on the spatial adjacency matrices of different pollutants and the graph convolution operator with parameter sharing. LSTM networks with a temporal sliding strategy are used to learn dynamic air pollution changes for temporal dependency modeling. The framework was applied to predict the average concentrations of PM2.5, PM10, O3, CO, SO2, and NO2 in the Bejing-Tianjin-Hebei (BTH) region for the next 24 hours. Experiments demonstrated that the proposed GT-LSTM model could extract high-level spatiotemporal features and achieve higher accuracy and stability than state-of-the-art baselines. Advancement in this methodology can assist in providing decision support capabilities to mitigate air quality issues.
A ResNet attention model for classifying mosquitoes from wing‑beating sounds
Authors: Xutong Wei, Md Zakir Hossain, Khandaker Asif Ahmed
Date: 2022-06-20 00:00:00
Proceeding: Scientific Reports 2022 6
Tasks: Classification, Data Augmentation, Sound Classification

Mosquitoes are vectors of numerous deadly diseases, and mosquito classification task is vital for their control programs. To ease manual labor and time-consuming classification tasks, numerous image-based machine-learning (ML) models have been developed to classify different mosquito species. Mosquito wing-beating sounds can serve as a unique classifier for mosquito classification tasks, which can be adopted easily in field applications. The current study aims to develop a deep neural network model to identify six mosquito species of three different genera, based on their wing-beating sounds. While existing models focused on raw audios, we developed a comprehensive pre-processing step to convert raw audios into more informative Mel-spectrograms, resulting in more robust and noise-free extracted features. Our model, namely ’Wing-beating Network’ or ’WbNet’, combines the state-of-art residual neural network (ResNet) model as a baseline, with self-attention mechanism and data-augmentation technique, and outperformed other existing models. The WbNet achieved the highest performance of 89.9% and 98.9% for WINGBEATS and ABUZZ data respectively. For species of Aedes and Culex genera, our model achieved 100% precision, recall and F1-scores, whereas, for Anopheles, the WbNet reached above 95%. We also compared two existing wing-beating datasets, namely WINGBEATS and ABUZZ, and found our model does not need sophisticated audio devices, hence performed better on ABUZZ audios, captured on usual mobile devices. Overall, our model has potential to serve in mosquito monitoring and prevalence studies in mosquito eradication programs, along with potential implementation in classification tasks of insect pests or other sound-based classifications.
Audio signal based danger detection using signal processing and deep learning
Authors: Awal Ahmed Fime, Md. Ashikuzzaman, Abdul Aziz
Date: 2023-09-19 00:00:00
Proceeding: Expert Systems with Applications 2023 9
Tasks: Audio Signal Processing

Since there have been more and more incidents of women being harassed in the recent past, girls need to think twice before going out of their houses. Sometimes, they are not even safe in their house or workplace. These circumstance doesn’t change for children of all genders who stay alone because of their working parents or for other reasons. Also, there is no such organized procedure to ensure safety and take women and children out of such violence and harassment. To address this problem, the authors of this paper developed an Android-based automated system to detect danger for women and children using audio from the surroundings. As the Android phone is available to everyone nowadays, they focused on using this device rather than developing a system on some external hardware. Different signal processing methods with deep learning techniques are used for this work. This work also addresses noise from the environment for any chaos and nullifies them using different noise reduction techniques such as Reduce Energy Noise, Reduce Mel Frequency Cepstrum Coefficient (MFCC) up Noise, Reduce Median Noise, Reduce Centroid Noise, Audio DeNoise, Noisereduce by Sainburf et al. & Butterworth high pass filter. The Noisereduce by Sainburg et al. along with the InceptionV3 model architecture turns out to be the best to classify audio with 95.51% accuracy. A new model called AudioViT is introduced. It uses a Visual Transformer and Residual Network to identify the audio signal. The Android application also takes necessary action when any unfavorable situation is detected. Android device users can use this application without any cost, which will pave the way to ensure the safety of women and children.
Chinese Text Recognition with A Pre-Trained CLIP-Like Model Through Image-IDS Aligning
Authors: Haiyang Yu, Xiaocong Wang, Bin Li, xiangyang xue
Date: 2023-09-03 00:00:00
Proceeding: ICCV 2023 1
Tasks: Scene Text Recognition

Scene text recognition has been studied for decades due to its broad applications. However, despite Chinese characters possessing different characteristics from Latin characters, such as complex inner structures and large categories, few methods have been proposed for Chinese Text Recognition (CTR). Particularly, the characteristic of large categories poses challenges in dealing with zero-shot and few-shot Chinese characters. In this paper, inspired by the way humans recognize Chinese texts, we propose a two-stage framework for CTR. Firstly, we pre-train a CLIP-like model through aligning printed character images and Ideographic Description Sequences (IDS). This pre-training stage simulates humans recognizing Chinese characters and obtains the canonical representation of each character. Subsequently, the learned representations are employed to supervise the CTR model, such that traditional single-character recognition can be improved to text-line recognition through image-IDS matching. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on both Chinese character recognition (CCR) and CTR. The experimental results demonstrate that the proposed method performs best in CCR and outperforms previous methods in most scenarios of the CTR benchmark. It is worth noting that the proposed method can recognize zero-shot Chinese characters in text images without fine-tuning, whereas previous methods require fine-tuning when new classes appear. The code is available at https://github.com/FudanVI/FudanOCR/tree/main/image-ids-CTR.
Examining the Influence of Varied Levels of Domain Knowledge Base Inclusion in GPT-based Intelligent Tutors
Authors: Blake Castleman, Mehmet Kerem Turkcan
Date: 2023-09-16 00:00:00

Recent advancements in large language models (LLMs) have facilitated the development of chatbots with sophisticated conversational capabilities. However, LLMs exhibit frequent inaccurate responses to queries, hindering applications in educational settings. In this paper, we investigate the effectiveness of integrating a knowledge base (KB) with LLM intelligent tutors to increase response reliability. To achieve this, we design a scaleable KB that affords educational supervisors seamless integration of lesson curricula, which is automatically processed by the intelligent tutoring system. We then detail an evaluation, where student participants were presented with questions about the artificial intelligence curriculum to respond to. GPT-4 intelligent tutors with varying hierarchies of KB access and human domain experts then assessed these responses. Lastly, students cross-examined the intelligent tutors' responses to the domain experts' and ranked their various pedagogical abilities. Results suggest that, although these intelligent tutors still demonstrate a lower accuracy compared to domain experts, the accuracy of the intelligent tutors increases when access to a KB is granted. We also observe that the intelligent tutors with KB access exhibit better pedagogical abilities to speak like a teacher and understand students than those of domain experts, while their ability to help students remains lagging behind domain experts.
EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning
Authors: Rajasekhar Reddy Mekala, Yasaman Razeghi, Sameer Singh
Date: 2023-09-16 00:00:00
Tasks: Date Understanding, GSM8K, In-Context Learning, Learning to Execute, Logical Reasoning, Misconceptions, Reading Comprehension

Language models are achieving impressive performance on various tasks by aggressively adopting inference-time prompting techniques, such as zero-shot and few-shot prompting. In this work, we introduce EchoPrompt, a simple yet effective approach that prompts the model to rephrase its queries before answering them. EchoPrompt is adapted for both zero-shot and few-shot in-context learning with standard and chain-of-thought prompting. Experimental results show that EchoPrompt yields substantial improvements across all these settings for four families of causal language models. These improvements are observed across various numerical reasoning (e.g. GSM8K, SVAMP), reading comprehension (e.g. DROP), and logical reasoning (e.g. Coin Flipping) tasks. On average, EchoPrompt improves the Zero-shot-CoT performance of code-davinci-002 by 5% in numerical tasks and 13% in reading comprehension tasks. We investigate the factors contributing to EchoPrompt's effectiveness through ablation studies, which reveal that both the original query and the model-generated rephrased version are instrumental in its performance gains. Our empirical results indicate that EchoPrompt is an effective technique that enhances in-context learning performance. We recommend incorporating EchoPrompt into various baseline prompting strategies to achieve performance boosts.
Automatic Answerability Evaluation for Question Generation
Authors: Zifan Wang, Kotaro Funakoshi, Manabu Okumura
Date: 2023-09-22 00:00:00
Tasks: Question Generation, Question-Generation, Text Generation

Conventional automatic evaluation metrics, such as BLEU and ROUGE, developed for natural language generation (NLG) tasks, are based on measuring the n-gram overlap between the generated and reference text. These simple metrics may be insufficient for more complex tasks, such as question generation (QG), which requires generating questions that are answerable by the reference answers. Developing a more sophisticated automatic evaluation metric, thus, remains an urgent problem in QG research. This work proposes PMAN (Prompting-based Metric on ANswerability), a novel automatic evaluation metric to assess whether the generated questions are answerable by the reference answers for the QG tasks. Extensive experiments demonstrate that its evaluation results are reliable and align with human evaluations. We further apply our metric to evaluate the performance of QG models, which shows that our metric complements conventional metrics. Our implementation of a GPT-based QG model achieves state-of-the-art performance in generating answerable questions.
Detect Everything with Few Examples
Authors: Xinyu Zhang, YuHan Liu, Yuting Wang, Abdeslam Boularias
Date: 2023-09-22 00:00:00
Tasks: Binary Classification, Cross-Domain Few-Shot Object Detection, Few-Shot Object Detection, Object, object-detection, Object Detection, One-Shot Object Detection, Open Vocabulary Object Detection

Few-shot object detection aims at detecting novel categories given only a few example images. It is a basic skill for a robot to perform tasks in open environments. Recent methods focus on finetuning strategies, with complicated procedures that prohibit a wider application. In this paper, we introduce DE-ViT, a few-shot object detector without the need for finetuning. DE-ViT's novel architecture is based on a new region-propagation mechanism for localization. The propagated region masks are transformed into bounding boxes through a learnable spatial integral layer. Instead of training prototype classifiers, we propose to use prototypes to project ViT features into a subspace that is robust to overfitting on base classes. We evaluate DE-ViT on few-shot, and one-shot object detection benchmarks with Pascal VOC, COCO, and LVIS. DE-ViT establishes new state-of-the-art results on all benchmarks. Notably, for COCO, DE-ViT surpasses the few-shot SoTA by 15 mAP on 10-shot and 7.2 mAP on 30-shot and one-shot SoTA by 2.8 AP50. For LVIS, DE-ViT outperforms few-shot SoTA by 17 box APr. Further, we evaluate DE-ViT with a real robot by building a pick-and-place system for sorting novel objects based on example images. The videos of our robot demonstrations, the source code and the models of DE-ViT can be found at https://mlzxy.github.io/devit.
An Efficient Intelligent Semi-Automated Warehouse Inventory Stocktaking System
Authors: Chunan Tong
Date: 2023-09-13 00:00:00
Tasks: Decision Making, Management

In the context of evolving supply chain management, the significance of efficient inventory management has grown substantially for businesses. However, conventional manual and experience-based approaches often struggle to meet the complexities of modern market demands. This research introduces an intelligent inventory management system to address challenges related to inaccurate data, delayed monitoring, and overreliance on subjective experience in forecasting. The proposed system integrates bar code and distributed flutter application technologies for intelligent perception, alongside comprehensive big data analytics to enable data-driven decision-making. Through meticulous analysis, system design, critical technology exploration, and simulation validation, the effectiveness of the proposed system is successfully demonstrated. The intelligent system facilitates second-level monitoring, high-frequency checks, and artificial intelligence-driven forecasting, consequently enhancing the automation, precision, and intelligence of inventory management. This system contributes to cost reduction and optimized inventory sizes through accurate predictions and informed decisions, ultimately achieving a mutually beneficial scenario. The outcomes of this research offer
Accurate and Fast Compressed Video Captioning
Authors: Yaojie Shen, Xin Gu, Kai Xu, Heng Fan, Longyin Wen, Libo Zhang
Date: 2023-09-22 00:00:00
Proceeding: ICCV 2023 1
Tasks: Video Captioning

Existing video captioning approaches typically require to first sample video frames from a decoded video and then conduct a subsequent process (e.g., feature extraction and/or captioning model learning). In this pipeline, manual frame sampling may ignore key information in videos and thus degrade performance. Additionally, redundant information in the sampled frames may result in low efficiency in the inference of video captioning. Addressing this, we study video captioning from a different perspective in compressed domain, which brings multi-fold advantages over the existing pipeline: 1) Compared to raw images from the decoded video, the compressed video, consisting of I-frames, motion vectors and residuals, is highly distinguishable, which allows us to leverage the entire video for learning without manual sampling through a specialized model design; 2) The captioning model is more efficient in inference as smaller and less redundant information is processed. We propose a simple yet effective end-to-end transformer in the compressed domain for video captioning that enables learning from the compressed video for captioning. We show that even with a simple design, our method can achieve state-of-the-art performance on different benchmarks while running almost 2x faster than existing approaches. Code is available at https://github.com/acherstyx/CoCap.
E(2)-Equivariant Graph Planning for Navigation
Authors: Linfeng Zhao, Hongyu Li, Taskin Padir, Huaizu Jiang, Lawson L. S. Wong
Date: 2023-09-22 00:00:00
Tasks: Robot Navigation

Learning for robot navigation presents a critical and challenging task. The scarcity and costliness of real-world datasets necessitate efficient learning approaches. In this letter, we exploit Euclidean symmetry in planning for 2D navigation, which originates from Euclidean transformations between reference frames and enables parameter sharing. To address the challenges of unstructured environments, we formulate the navigation problem as planning on a geometric graph and develop an equivariant message passing network to perform value iteration. Furthermore, to handle multi-camera input, we propose a learnable equivariant layer to lift features to a desired space. We conduct comprehensive evaluations across five diverse tasks encompassing structured and unstructured environments, along with maps of known and unknown, given point goals or semantic goals. Our experiments confirm the substantial benefits on training efficiency, stability, and generalization. More details can be found at the project website: https://lhy.xyz/e2-planning/.
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
Authors: Jiahao Xie, Wei Li, Xiangtai Li, Ziwei Liu, Yew Soon Ong, Chen Change Loy
Date: 2023-09-22 00:00:00
Tasks: Data Augmentation, Instance Segmentation, Semantic Segmentation

We present MosaicFusion, a simple yet effective diffusion-based data augmentation approach for large vocabulary instance segmentation. Our method is training-free and does not rely on any label supervision. Two key designs enable us to employ an off-the-shelf text-to-image diffusion model as a useful dataset generator for object instances and mask annotations. First, we divide an image canvas into several regions and perform a single round of diffusion process to generate multiple instances simultaneously, conditioning on different text prompts. Second, we obtain corresponding instance masks by aggregating cross-attention maps associated with object prompts across layers and diffusion time steps, followed by simple thresholding and edge-aware refinement processing. Without bells and whistles, our MosaicFusion can produce a significant amount of synthetic labeled data for both rare and novel categories. Experimental results on the challenging LVIS long-tailed and open-vocabulary benchmarks demonstrate that MosaicFusion can significantly improve the performance of existing instance segmentation models, especially for rare and novel categories. Code: https://github.com/Jiahao000/MosaicFusion.
Memory-augmented conformer for improved end-to-end long-form ASR
Authors: Carlos Carvalho, Alberto Abad
Date: 2023-09-22 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, Form, speech-recognition, Speech Recognition

Conformers have recently been proposed as a promising modelling approach for automatic speech recognition (ASR), outperforming recurrent neural network-based approaches and transformers. Nevertheless, in general, the performance of these end-to-end models, especially attention-based models, is particularly degraded in the case of long utterances. To address this limitation, we propose adding a fully-differentiable memory-augmented neural network between the encoder and decoder of a conformer. This external memory can enrich the generalization for longer utterances since it allows the system to store and retrieve more information recurrently. Notably, we explore the neural Turing machine (NTM) that results in our proposed Conformer-NTM model architecture for ASR. Experimental results using Librispeech train-clean-100 and train-960 sets show that the proposed system outperforms the baseline conformer without memory for long utterances.