pwc-archive/papers-with-abstracts huggingface dataset:
Real-time detection of abnormal driving behavior based on long short-term memory network and regression residuals
Authors: Yongfeng Ma, Zhuopeng Xie, Shuyan Chen, Fengxiang Qiao, Zeyang Li
Date: 2022-12-14 00:00:00
Proceeding: Transportation Research Part C: Emerging Technologies 2022 12

Abnormal driving behavior is one of the main causes of roadway collisions. In most studies of abnormal
driving behavior, the abnormal driving status is detected and analyzed using classification algorithms
directly or using unsupervised learning algorithms to classify reconstruction or prediction residuals.
However, abnormal driving behavior data are difficult to acquire and label. Also, a class imbalance issue is
inherent in the algorithm training process due to the relatively sparse data for abnormal driving behavior.
Moreover, current studies that include residual analysis tend to focus on individual points and thus fail to
capture the continuity characteristic of abnormal driving behavior.
To address these problems, a long short-term memory-residual (LSTM-R) algorithm is proposed to
detect abnormal driving behavior in real time. The proposed algorithm (referred to simply as LSTMR) has two steps. First, an LSTM network is used to fit the current vehicle kinematic data based on
historical data to obtain the root mean square residual at each moment. Second, a time windowbased residual algorithm is designed and employed to detect abnormal driving behavior according to the magnitude and continuity of the residuals. To verify the effectiveness of LSTM-R, an
experimental test was conducted in Nanjing, China. The vehicle kinematic data were collected nonintrusively using a smartphone.
In addition, AdaCost, SMOTEBoost, EasyEnsemble, LightGBM-residual, and linear regressionresidual algorithms were employed for comparison with the proposed algorithm to assess its
effectiveness. The effects of (1) the degree-of-fit of the LSTM network, (2) the LSTM-R parameters,
and (3) the abnormal driving behavior percentage on the detection results were analyzed in detail.
First, both the underfitting and overfitting of the LSTM network compromise the detection performance. Second, within a certain range of values, the LSTM-R parameters have little effect on the
detection results. Third, the detection results are affected only slightly by the abnormal proportion.
The results show that LSTM-R, with a maximum F1-score of 0.866, significantly outperforms the
other five algorithms. Furthermore, even if only 10% abnormal driving behavior is in the training
set, LSTM-R’s F1-score can still be close to 0.86, indicating a significant relaxation of the requirements for labeled data. Also, the required data are easy to collect, which indicates LSTM-R’s
extensive application possibilities. This paper thus provides an effective method for the real-time
detection of abnormal driving behavior and also supports driving risk assessment and driving
behavior improvement with the overall goal to enhance roadway safety.
Convolutional Matrix Factorization for Document Context-Aware Recommendation
Authors: Donghyun Kim, Chanyoung Park, Jinoh Oh, Sungyoung Lee, Hwanjo Y
Date: 2016-09-07 00:00:00
Proceeding: RecSys 2016 9
Tasks: Recommendation Systems

Sparseness of user-to-item rating data is one of the major factors that deteriorate the quality of recommender system. To handle the sparsity problem, several recommendation techniques have been proposed that additionally consider auxiliary information to improve rating prediction accuracy. In particular, when rating data is sparse, document modeling-based approaches have improved the accuracy by additionally utilizing textual data such as reviews, abstracts, or synopses. However, due to the inherent limitation of the bag-of-words model, they have difficulties in effectively utilizing contextual information of the documents, which leads to shallow understanding of the documents. This paper proposes a novel context-aware recommendation model, convolutional matrix factorization (ConvMF) that integrates convolutional neural network (CNN) into probabilistic matrix factorization (PMF). Consequently, ConvMF captures contextual information of documents and further enhances the rating prediction accuracy. Our extensive evaluations on three real-world datasets show that ConvMF significantly outperforms the state-of-the-art recommendation models even when the rating data is extremely sparse. We also demonstrate that ConvMF successfully captures subtle contextual difference of a word in a document. Our implementation and datasets are available at http://dm.postech.ac.kr/ConvMF.
A Deep Learning Approach for Meibomian Gland Atrophy Evaluation in Meibography Images
Authors: Jiayun Wang, Thao N. Yeh, Rudrasis Chakraborty, Stella X. Yu, Meng C. Lin
Date: 2019-12-01 00:00:00
Proceeding: Translational Vision Science & Technology 2019 12
Tasks: Deep Learning

Purpose: To develop a deep learning approach to digitally segmenting meibomian gland atrophy area and computing percent atrophy in meibography images.

Methods: A total of 706 meibography images with corresponding meiboscores were collected and annotated for each one with eyelid and atrophy regions. The dataset was then divided into the development and evaluation sets. The development set was used to train and tune the deep learning model, while the evaluation set was used to evaluate the performance of the model.

Results: Four hundred ninety-seven meibography images were used for training and tuning the deep learning model while the remaining 209 images were used for evaluations. The algorithm achieves 95.6% meiboscore grading accuracy on average, largely outperforming the lead clinical investigator (LCI) by 16.0% and the clinical team by 40.6%. Our algorithm also achieves 97.6% and 95.4% accuracy for eyelid and atrophy segmentations, respectively, as well as 95.5% and 66.7% mean intersection over union accuracies (mean IU), respectively. The average root-mean-square deviation (RMSD) of the percent atrophy prediction is 6.7%.

Conclusions: The proposed deep learning approach can automatically segment the total eyelid and meibomian gland atrophy regions, as well as compute percent atrophy with high accuracy and consistency. This provides quantitative information of the gland atrophy severity based on meibography images.

Translational Relevance: Based on deep neural networks, the study presents an accurate and consistent gland atrophy evaluation method for meibography images, and may contribute to improved understanding of meibomian gland dysfunction.
Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources
Authors: Yongfeng Zhang, Qingyao Ai, Xu Chen, W. Bruce Croft
Date: 2017-11-06 00:00:00
Proceeding: CIKM 2017 11
Tasks: Context Aware Product Recommendation, Learning-To-Rank, Recommendation Systems, Representation Learning

The Web has accumulated a rich source of information, such as text, image, rating, etc, which represent different aspects of user preferences. However, the heterogeneous nature of this information makes it difficult for recommender systems to leverage in a unified framework to boost the performance. Recently, the rapid development of representation learning techniques provides an approach to this problem. By translating the various information sources into a unified representation space, it becomes possible to integrate heterogeneous information for informed recommendation.

In this work, we propose a Joint Representation Learning (JRL) framework for top-N recommendation. In this framework, each type of information source (review text, product image, numerical rating, etc) is adopted to learn the corresponding user and item representations based on available (deep) representation learning architectures. Representations from different sources are integrated with an extra layer to obtain the joint representations for users and items. In the end, both the per-source and the joint representations are trained as a whole using pair-wise learning to rank for top-N recommendation. We analyze how information propagates among different information sources in a gradient-descent learning paradigm, based on which we further propose an extendable version of the JRL framework (eJRL), which is rigorously extendable to new information sources to avoid model re-training in practice.

By representing users and items into embeddings offline, and using a simple vector multiplication for ranking score calculation online, our framework also has the advantage of fast online prediction compared with other deep learning approaches to recommendation that learn a complex prediction network for online calculation.
DSAT-Net: Dual Spatial Attention Transformer for Building Extraction from Aerial Images
Authors: Renhe Zhang, Zhechun Wan, Qian Zhang, Guixu Zhang
Date: 2023-08-16 00:00:00
Proceeding: IEEE Geoscience and Remote Sensing Letters 2023 8
Tasks: Extracting Buildings In Remote Sensing Images, Semantic Segmentation

Both local and global context dependencies are essential for building extraction from remote sensing (RS) images. Convolutional Neural Network (CNN) can extract local spatial details well but lacks the ability to model long-range dependency. In recent years, Vision Transformer (ViT) have shown great potential in modeling global context dependency. However, it usually brings huge computational cost, and spatial details can not be fully retained in the process of feature extraction. To maximize the advantages of CNNs and ViTs, we propose DSAT-Net, which combine them in one model. In DSAT-Net, we design an efficient Dual Spatial Attention Transformer (DSAFormer) to solve the defects of standard ViT. It has a dual attention structure to complement each other. Specifically, the global attention path (GAP) conducts a large scale down sampling of the feature maps before the global self-attention computing, to reduce the computational cost. The local attention path (LAP) uses efficient stripe convolution to generate local attention, which can alleviate the loss of information caused by down-sampling operation in the GAP and supplement the spatial details. In addition, we design a feature refining module called Channel Mixing Feature Refine Module (CM-FRM) to fuse low-level and high-level features. Our model achieved competitive results on three public building extraction datasets. Code will be available at: https://github.com/stdcoutzrh/BuildingExtraction.
Feature-Based Transfer Learning for Network Security
Authors: Juan Zhao, Sachin Shetty, Jan Wei Pan
Date: 2017-12-11 00:00:00
Proceeding: MILCOM 2017 12
Tasks: Transfer Learning

New and unseen network attacks pose a great threat
to the signature-based detection systems. Consequently, machine
learning-based approaches are designed to detect attacks, which
rely on features extracted from network data. The problem is
caused by different distribution of features in the training and
testing datasets, which affects the performance of the learned
models. Moreover, generating labeled datasets is very time-consuming and expensive, which undercuts the effectiveness of
supervised learning approaches. In this paper, we propose using
transfer learning to detect previously unseen attacks. The main
idea is to learn the optimized representation to be invariant
to the changes of attack behaviors from labeled training sets
and non-labeled testing sets, which contain different types of
attacks and feed the representation to a supervised classier.
To the best of our knowledge, this is the first effort to use
a feature-based transfer learning technique to detect unseen
variants of network attacks. Furthermore, this technique can be
used with any common base classier. We evaluated the technique
on publicly available datasets, and the results demonstrate the
effectiveness of transfer learning to detect new network attacks.
ChatHaruhi: Reviving Anime Character in Reality via Large Language Model
Authors: Cheng Li, Ziang Leng, Chenxi Yan, Junyi Shen, Hao Wang, Weishi MI, Yaying Fei, Xiaoyang Feng, Song Yan, HaoSheng Wang, Linkang Zhan, Yaokai Jia, Pingyu Wu, Haozhen Sun
Date: 2023-08-18 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, Text2text Generation, Text Generation

Role-playing chatbots built on large language models have drawn interest, but better techniques are needed to enable mimicking specific fictional characters. We propose an algorithm that controls language models via an improved prompt and memories of the character extracted from scripts. We construct ChatHaruhi, a dataset covering 32 Chinese / English TV / anime characters with over 54k simulated dialogues. Both automatic and human evaluations show our approach improves role-playing ability over baselines. Code and data are available at https://github.com/LC1332/Chat-Haruhi-Suzumiya .
Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training
Authors: Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, Hengshuang Zhao
Date: 2023-08-18 00:00:00
Proceeding: CVPR 2024 1
Tasks: 3D Semantic Segmentation, LIDAR Semantic Segmentation, Representation Learning, Semantic Segmentation

The rapid advancement of deep learning models often attributes to their ability to leverage massive training data. In contrast, such privilege has not yet fully benefited 3D deep learning, mainly due to the limited availability of large-scale 3D datasets. Merging multiple available data sources and letting them collaboratively train a single model is a potential solution. However, due to the large domain gap between 3D point cloud datasets, such mixed supervision could adversely affect the model's performance and lead to degenerated performance (i.e., negative transfer) compared to single-dataset training. In view of this challenge, we introduce Point Prompt Training (PPT), a novel framework for multi-dataset synergistic learning in the context of 3D representation learning that supports multiple pre-training paradigms. Based on this framework, we propose Prompt-driven Normalization, which adapts the model to different datasets with domain-specific prompts and Language-guided Categorical Alignment that decently unifies the multiple-dataset label spaces by leveraging the relationship between label text. Extensive experiments verify that PPT can overcome the negative transfer associated with synergistic learning and produce generalizable representations. Notably, it achieves state-of-the-art performance on each dataset using a single weight-shared model with supervised multi-dataset training. Moreover, when served as a pre-training framework, it outperforms other pre-training approaches regarding representation quality and attains remarkable state-of-the-art performance across over ten diverse downstream tasks spanning both indoor and outdoor 3D scenarios.
Shadow Removal of Text Document Images Using Background Estimation and Adaptive Text Enhancement
Authors: Wenjie Liu, Bingshu Wang, Jiangbin Zheng, Wenmin Wang
Date: 2023-05-05 00:00:00
Proceeding: ICASSP 2023 5
Tasks: Binarization, Document Shadow Removal, Image Shadow Removal, Shadow Removal

This paper proposes a simple yet effective method to re-move shadows from text document images. It mainly includes several parts. Firstly, we propose a text elimination-based background extraction strategy to estimate shadow map. It indicates the shadow regions accurately and helps to predict global background. Secondly, a binarization-based text ex-traction algorithm is designed to obtain texts from document image. By fusing texts and global background, a preparatory shadow-free image can be obtained. Thirdly, we propose an adaptive text contrast enhancement strategy to generate shadow-free results with comfortable visual perception across shadow and non-shadow regions. Quantitative and visual results performed on open datasets indicate that the proposed method can generate clear shadow-free images from text document images. Our code will be publicly available soon.
Shadow Removal of Text Document Images by Estimating Local and Global Background Colors
Authors: Jian-Ren Wang, Yung-Yu Chuang
Date: 2020-04-09 00:00:00
Proceeding: ICASSP 2020 4
Tasks: Shadow Removal

This paper proposes a simple yet effective method for removing shadows from text document images. Assuming that the document mainly contains texts, our method estimates the global and local background colors using statistical analysis of the whole image and local neighborhoods. By estimating the global and local background colors, we obtain the shadow map indicating the shadow ratio for each pixel. With the shadow map, a shadow-free image can be recovered by intrinsic decomposition. Experiments confirm that our method effectively removes the shadow of text document images regardless of the intensity, scope, and the number of shadows.
An Effective Background Estimation Method for Shadows Removal of Document Images
Authors: Bingshu Wang, C. L. Philip Chen
Date: 2019-08-26 00:00:00
Proceeding: ICIP 2019 8
Tasks: Document Shadow Removal, Shadow Detection, Shadow Removal

Shadows of document images bring about difficulties for digitization application and uncomfortable perception in vision. This paper proposes an effective method to remove shadows from the single document images, which contains two stages: shadow detection and shadow removal. For the shadow detection, an iterative neighboring information-based approach is designed to estimate pixel-wise local background color image, which can be used to generate a shadow map. For the shadow removal, a global reference background color is obtained from the local background color image. The shadow scale, which is applied to relight shadowed regions, can be estimated by local and global background color. Moreover, a tone fine-tuning process is designed to make the output colors be normal values. Experiments on document images indicate that the proposed method can produce high-quality unshad-owed document images with a comparable efficiency.
An Iterative Approach for Shadow Removal in Document Images
Authors: Vatsal Shah, Vineet Gandhi
Date: 2018-09-13 00:00:00
Proceeding: ICASSP 2018 9
Tasks: Document Shadow Removal, Shadow Removal

Uneven illumination and shadows in document images cause a challenge for digitization applications and automated workflows. In this work, we propose a new method to recover unshadowed document images from images with shadows/un-even illumination. We pose this problem as one of estimating the shading and reflectance components of the given original image. Our method first estimates the shading and uses it to compute the reflectance. The output reflectance map is then used to improve the shading and the process is repeated in an iterative manner. The iterative procedure allows for a gradual compensation and allows our algorithm to even handle difficult hard shadows without introducing any artifacts. Experiments over two different datasets demonstrate the efficacy of our algorithm and the low computation complexity makes it suitable for most practical applications.
Removing Shadows from Images of Documents
Authors: Steve Bako, Soheil Darabi, Eli Shechtman, Jue Wang, Kalyan Sunkavalli, Pradeep Sen
Date: 2017-03-11 00:00:00
Proceeding: ACCV 2017 3
Tasks: Document Shadow Removal

In this work, we automatically detect and remove distracting shadows from photographs of documents and other text-based items. Documents typically have a constant colored background; based on this observation, we propose a technique to estimate background and text color in local image blocks. We match these local background color estimates to a global reference to generate a shadow map. Correcting the image with this shadow map produces the final unshadowed output. We demonstrate that our algorithm is robust and produces high-quality results, qualitatively and quantitatively, in both controlled and real-world settings containing large regions of significant shadow.
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization
Authors: Soumik Mukhopadhyay, Saksham Suri, Ravi Teja Gadde, Abhinav Shrivastava
Date: 2023-08-18 00:00:00

The task of lip synchronization (lip-sync) seeks to match the lips of human faces with different audio. It has various applications in the film industry as well as for creating virtual avatars and for video conferencing. This is a challenging problem as one needs to simultaneously introduce detailed, realistic lip movements while preserving the identity, pose, emotions, and image quality. Many of the previous methods trying to solve this problem suffer from image quality degradation due to a lack of complete contextual information. In this paper, we present Diff2Lip, an audio-conditioned diffusion-based model which is able to do lip synchronization in-the-wild while preserving these qualities. We train our model on Voxceleb2, a video dataset containing in-the-wild talking face videos. Extensive studies show that our method outperforms popular methods like Wav2Lip and PC-AVS in Fr\'echet inception distance (FID) metric and Mean Opinion Scores (MOS) of the users. We show results on both reconstruction (same audio-video inputs) as well as cross (different audio-video inputs) settings on Voxceleb2 and LRW datasets. Video results and code can be accessed from our project page ( https://soumik-kanad.github.io/diff2lip ).
SimDA: Simple Diffusion Adapter for Efficient Video Generation
Authors: Zhen Xing, Qi Dai, Han Hu, Zuxuan Wu, Yu-Gang Jiang
Date: 2023-08-18 00:00:00
Proceeding: CVPR 2024 1
Tasks: Super-Resolution, Transfer Learning, Video Editing, Video Generation, Video Super-Resolution

The recent wave of AI-generated content has witnessed the great development and success of Text-to-Image (T2I) technologies. By contrast, Text-to-Video (T2V) still falls short of expectations though attracting increasing interests. Existing works either train from scratch or adapt large T2I model to videos, both of which are computation and resource expensive. In this work, we propose a Simple Diffusion Adapter (SimDA) that fine-tunes only 24M out of 1.1B parameters of a strong T2I model, adapting it to video generation in a parameter-efficient way. In particular, we turn the T2I model for T2V by designing light-weight spatial and temporal adapters for transfer learning. Besides, we change the original spatial attention to the proposed Latent-Shift Attention (LSA) for temporal consistency. With similar model architecture, we further train a video super-resolution model to generate high-definition (1024x1024) videos. In addition to T2V generation in the wild, SimDA could also be utilized in one-shot video editing with only 2 minutes tuning. Doing so, our method could minimize the training effort with extremely few tunable parameters for model adaptation.
Graph of Thoughts: Solving Elaborate Problems with Large Language Models
Authors: Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, Torsten Hoefler
Date: 2023-08-18 00:00:00

We introduce Graph of Thoughts (GoT): a framework that advances prompting capabilities in large language models (LLMs) beyond those offered by paradigms such as Chain-of-Thought or Tree of Thoughts (ToT). The key idea and primary advantage of GoT is the ability to model the information generated by an LLM as an arbitrary graph, where units of information ("LLM thoughts") are vertices, and edges correspond to dependencies between these vertices. This approach enables combining arbitrary LLM thoughts into synergistic outcomes, distilling the essence of whole networks of thoughts, or enhancing thoughts using feedback loops. We illustrate that GoT offers advantages over state of the art on different tasks, for example increasing the quality of sorting by 62% over ToT, while simultaneously reducing costs by >31%. We ensure that GoT is extensible with new thought transformations and thus can be used to spearhead new prompting schemes. This work brings the LLM reasoning closer to human thinking or brain mechanisms such as recurrence, both of which form complex networks.