Reviewerly/exHarmony huggingface dataset:
Few-Shot Learning Network for Moving Object Detection Using Exemplar-Based Attention Map

Moving object detection is a core task in computer vision. However, existing deep learning-based moving object detection methods require a large number of labeled frames to achieve good generalization and performance. This paper proposes a novel deep learning network called FeSh-Net. This network can learn to extract an exemplar-based attention map using a few labeled frames, which guides the network to know which object is foreground and which is a background in the current frame. FeSh-Net is trained using a novel meta-learning technique to be able to segment moving objects from new unseen videos. The proposed network is evaluated using the benchmark CDNet. The results of the proposed FeSh-Net are compared with current state-of-the-art methods, and the results show that FeSh-Net outperforms the best reported state-of-the-art method by 4.4% on average. Additionally, FeSh-Net performs better than other methods when tested using new unseen videos.
Tensor-Based Deepfake Detection in Scaled and Compressed Images

When deepfakes are widespread on chatting platforms, they are expected to be subject to heavy resizing and compressing steps. In this paper, we present a tensor-based representation of compressed and resized images. Tensor embeds DCT features computed on multi-scaled and multi-compressed versions of the input facial image. Moreover, a custom deep-architecture is designed and trained on the proposed representation. Experimental results show its pros and cons with respect to state-of-the-art methods.
Content-Adaptive Neural Network Post-Processing Filter with NNR-Coded Weight-Updates

Neural Network (NN) filters improve the perceptual quality of reconstructed videos by reducing compression artefacts. For content adaptation, a few NN-filters use over-fitting. As the adaptation signal is a weight-update, compression is required to minimise significant bitrate overheads. Most approaches, however, use generic data compression algorithms, which are inadequate for coding NN weight-updates. This work introduces a content-adaptive NN post-processing filter with weight-updates coded using the Neural Network compression and Representation (NNR) standard. The bitrate overhead is further decreased by over-fitting only a subset of weights, selected via energy-based analysis. The proposed filter saved about 4.57% (Y), 10.33% (Cb), 6.53% (Cr) Bjøntegaard Delta rate (BD-rate) on top of the Versatile Video Coding (VVC) Test Model (VTM) 11.0 with NN-based Video Coding (NNVC) 1.0, in Random Access (RA) configuration. Compared to the non-over-fitted NN, the performance was doubled; and compared to 7z, NNR reduced the bitrate of the weight-update by ∼64%.
Residual Swin Transformer Unet with Consistency Regularization for Automatic Breast Ultrasound Tumor Segmentation

Automatic Breast Ultrasound (ABUS) image segmentation is of great significance for breast cancer diagnosis and treatment. However, similar to most medical datasets, ABUS image datasets are often small-scale and seriously imbalanced, which makes ABUS image segmentation become a challenge. To solve this problem, we propose the Residual Swin Transformer Unet with Consistency Regularization (RSTUnet-CR) which can make full use of non-lesion and unlabeled images for high-precision tumor segmentation on ABUS images. We design a consistency-regularization decoder to reconstruct the input image, which can learn well from non-lesion and unlabeled data. The reconstruction task makes the model more suitable for the imbalanced medical image datasets. In addition, observing that the ABUS images have global semantic correlation, we establish long-distance dependence of images by the residual Swin Transformer block to improve segmentation performance. We evaluate our method on the ABUS dataset collected from 256 subjects and demonstrate the superiority of the proposed method over other state-of-the-art methods in this imbalanced dataset.
Which Metrics for Network Pruning: Final Accuracy? Or Accuracy Drop?

Network pruning enables the utilization of deep neural networks in low-resource environments by removing redundant elements in a pre-trained network. To appraise each pruning method, two evaluation metrics are generally adopted, i.e., final accuracy and accuracy drop. Final accuracy represents the ultimate performance of the pruned sub-network after the pruning completes. On the other hand, accuracy drop, a more traditional way, measures the accuracy difference between the baseline model and the final pruned model. In this work, we present several surprising observations which reveal the unfairness of both metrics when assessing the efficacy of pruning approaches. Depending on the choice of baseline network, the value of each pruning method may be completely changed. Specifically, a lower baseline tends to be advantageous for the accuracy drop, whereas a higher baseline usually yields a higher final accuracy. Moreover, to reduce the undesirable dependency on the baseline network, we propose a new reliable averaging method Average from Scratches which uses multiple distinct baselines rather than using a single baseline. Our various investigations point to the necessity for a more thorough analysis on network pruning metrics.
Ablation-CAM++: Grouped Recursive Visual Explanations for Deep Convolutional Networks

Recently, providing explainable deep learning models has sparked a lot of attention. In this paper, we take a further step in this direction. We introduce a time-efficient method, called Ablation-CAM++, which can generate smooth visual explanations of CNN model predictions. Our approach uses the concept of studying the ablation analysis to determine the importance of activation maps w.r.t. the target class, similar to Ablation-CAM. However, instead of focusing on the individual importance of each activation map, we group activation maps using a clustering technique. Then, we construct a binary tree for each group by recursively splitting these groups, studying the ablation of each subgroup, and applying tree pruning. We perform qualitative and quantitative evaluations of our visual explanations against Ablation-CAM and Grad-CAM. Our approach can provide visual explanations in less than half of the time of Ablation-CAM. Using average drop and average increase evaluation metrics on 2000 images of the ImageNet validation set, we provide a comparison of the effect of applying different clustering techniques in our method.
Transformer-Based Approach for Document Layout Understanding

We present an end-to-end transformer-based framework named TRDLU for the task of Document Layout Understanding (DLU). DLU is the fundamental task to automatically understand document structures. To accurately detect content boxes and classify them into semantically meaningful classes from various formats of documents is still an open challenge. Recently, transformer-based detection neural networks have shown their capability over traditional convolutional-based methods in the object detection area. In this paper, we consider DLU as a detection task, and introduce TRDLU which integrates transformer-based vision backbone and transformer encoder-decoder as detection pipeline. TRDLU is only a visual feature-based framework, but its performance is even better than multi-modal feature-based models. To the best of our knowledge, this is the first study of employing a fully transformer-based framework in DLU tasks. We evaluated TRDLU on three different DLU benchmark datasets, each with strong baselines. TRDLU outperforms the current state-of-the-art methods on all of them.
NCTR: Neighborhood Consensus Transformer for Feature Matching

This paper presents NCTR, a feature matcher that enhances input descriptors and finds the correspondences between them. In NCTR, Transformer is applied to aggregate global context for each descriptor. To solve the lack of neighborhood consensus that Transformer may bring, we propose a novel method to evaluate the neighborhood consensus of each key-point and integrate it into the attentional aggregation. The combination of global and local information greatly enhances the model capability and improves the match quality. The experiments on homography estimation and outdoor pose estimation show that NCTR outperforms other hand-designed or learning-based methods and achieves state-of-the-art results.
Downsampling Based Light Field Video Coding with Restoration Network Using Joint Spatio-Angular and Epipolar Information

This paper proposes a new downsampling-based light field video coding (D-LFVC) framework whose success relies on how to design an effective restoration method that can remove artifacts brought by both downsampling and compression. Since light field (LF) video is of high dimensionality data, the restoration methods designed for conventional 2D video are sub-optimal solutions for our D-LFVC. In this regard, we design a new restoration network, named "LF-QEN," for our D-LFVC framework. Specifically, the network contains three different feature extractor modules, allowing us to simultaneously exploit information from different kinds of 4D LF representation: spatial, angular, and epipolar image information. Our experimental results show that, compared to compression by HEVC-SCC standard, the proposed framework can obtain not only nearly 50% bitrate savings but also can significantly enhance the quality of decoded LF video.
Exploring Occlusion-Sensitive Deep Network for Single-View 3D Face Reconstruction

Recovering 3D geometry from a single-view 2D face image is an ill-posed task full of various challenges, especially under occlusion conditions commonly seen with large poses, while partial facial information missing makes the burden much heavier. Although existing methods could solve this problem in the end-to-end fashion, they still have limited performance in the occluded scenes. It is intuitively for many methods to depress the influence of those occluded regions for reconstruction. But we propose an occlusion-sensitive weighting mechanism for balancing the contributions among occluded and non-occluded regions. Meanwhile, considering no dataset contains various occlusions for learning, the data augmentation technique is exploited to expand the training dataset, which further facilitates the learning of the occlusion-sensitive deep network. Extensive experiments on two challenging datasets validate the advanced performance of our method for both 3D face reconstruction and face alignment. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>
Latent Vector Prototypes Guided Conditional Face Synthesis

Recent advances in deep neural networks, especially in generative adversarial networks (GAN), have shown remarkable progress in face image generations. However, most of the existing face image generators can only synthesize random face images, but are not able to control the attributes of the generated face images. Though conditional GAN based methods can manipulate the attributes to some extent, but can only generate low-resolution face images up to 256 × 256. In this study, based on StyleGAN, one of the state-of-the-art image generators for synthesizing high-quality face images, we propose a simple but efficient approach to generate high-resolution and hyper-realistic face images with any desired attribute. By training an attribute classifier to assign attribute labels to given synthesized face images, we build the links between latent vectors and face attributes. In such a way, the latent vectors can be grouped into different clusters, one cluster corresponding to one face attribute, respectively. We then extract the prototypes for the clusters, which are used to control the attribute of the generated face image. Extensive experiments demonstrate the effectiveness of the proposed approach for high-quality face image generation with predefined attributes.
AI-Based Compression: A New Unintended Counter Attack on JPEG-Related Image Forensic Detectors?

The detection of forged images is an important topic in digital image forensics. There are two main types of forgery: copy-move and splicing. These forgeries are created with image editors that apply JPEG compression by default, when saving the forged images. As a result, the authentic and falsified areas have different compression statistics, including histograms of DCT coefficients that show inconsistencies in the case of double JPEG compression. Therefore, the detection of double JPEG compression (DJPEG-C) is an important topic for JPEG-related image forensic detectors. Since the emergence of deep learning in image processing, AI-based compression methods have been proposed. This paper is the first to consider AI-based compression with digital image analysis tools. The objective is to understand whether AI-based compression can be a new unintended counter-attack for JPEG-related image forensic detectors. To verify our hypothesis, we selected the best detector to date, an AI-based compression method and the Casia v2 database that contains both splicing and copy-move (all publicly available). We focused our experiment on benign post-processing operations: AI-based and JPEG recompressions (with different quality levels). The evaluation is performed using different metrics (average precision, F1 score and accuracy, PSNR, SSIM) to take into account both the impact on detection and image quality. At similar image quality, AI-based recompression achieves a decrease in performance at least twice higher than JPEG, while preserving high visual image quality. Thus, AI-based compression is a new unintended counter-attack, which can no longer be ignored in future studies on image forensic detectors.
Synchronized Audio-Visual Frames with Fractional Positional Encoding for Transformers in Video-to-Text Translation

Video-to-text (VTT) is the task of automatically generating descriptions for short audio-visual video clips.It can help visually impaired people to understand scenes shown in a YouTube video, for example.Transformer architectures have shown great performance in both machine translation and image captioning.In this work, we transfer promising approaches from image captioning and video processing to VTT and develop a straightforward Transformer architecture.Then, we expand this Transformer by a novel way of synchronizing audio and video features in Transformers which we call Fractional Positional Encoding (FPE).We run multiple experiments on the VATEX dataset and improve the CIDEr and BLEU-4 scores by 21.72 and 8.38 points compared to a vanilla Transformer network and achieve state-of-the art results on the MSR-VTT and MSVD datasets.Also, our novel FPE helps increase the CIDEr score by relative 8.6 %.
Quadtree-based Guided CNN for AV1 In-loop Filtering

Recently, learning-based in-loop filtering has attracted lots of attention. State-of-the-art works generally deploy computationally expensive, large-scale neural networks, which is unfriendly to practical applications. Besides, since these models are generally pre-trained using a limited dataset and applied to various videos, they may fail in video contents excluded in the training dataset. To address these issues, this paper develops a Guide CNN in-loop filtering framework to obtain the restored signal. Our basic idea is to construct a subspace and use the projection of the original signal into this subspace to approximate the original signal itself. Specifically, we employ CNN to transform the degraded signal into M subsignals to construct the optimal subspace since the training of CNN is essentially an optimization procedure. Furthermore, unlike existing CNN models that process all blocks uniformly, our method leverages a quadtree structure to implement the Guided CNN through R-D optimization. As such, the best partition to Guided CNN can be determined. We exemplify the proposed method in AV1 codec. Experimental results show that the Guided CNN framework achieves 2.19% and 1.31% BD-Rate gains over the AV1 anchor in intra and inter coding mode, respectively, while the normal CNN achieves only 1.64% and 1.04%.
Latent Preserving Generative Adversarial Network for Imbalance Classification

Many real-world classification problems have imbalanced frequency of class labels; a well-known issue known as the "class imbalance" problem. Classic classification algorithms tend to be biased towards the majority class, leaving the classifier vulnerable to misclassification of the minority class. While the literature is rich with methods to fix this problem, as the dimensionality of the problem increases, many of these methods do not scale-up and the cost of running them become prohibitive. In this paper, we present an end-to-end deep generative classifier. We propose a domain-constraint autoencoder to preserve the latent-space as prior for a generator, which is then used to play an adversarial game with two other deep networks, a discriminator and a classifier. Extensive experiments are carried out on three different multi-class imbalanced problems and a comparison with state-of-the-art methods. Experimental results confirmed the superiority of our method over popular algorithms in handling high-dimensional imbalanced classification problems. Our code is available on https://github.com/TanmDL/SLPPL-GAN
Hyperbolic Spatial Temporal Graph Convolutional Networks

Spatial-temporal graph convolutional networks (ST-GCNs) have been successfully applied for dynamic graphs representation learning, such as modeling skeleton-based human actions. However, ST-GCNs embed these non-Euclidean graph structures into Euclidean space, which is not the natural space to represent such structures as embedding them in this space incurs a large distortion. In this work, we make use of hyperbolic non-Euclidean geometry and construct compact ST-GCNs in the hyperbolic space. It can be shown that hyperbolic ST-GCNs (HST-GCNs) outperform the corresponding Euclidean counterparts. Additionally, these compact hyperbolic models can be used to increase the performance of large complex Euclidean models. Moreover, we show that the same or even better performance of large Euclidean models can be achieved by fusing the scores of smaller Euclidean models and a compact hyperbolic model. This in turn leads to reducing the total number of model parameters and hence model size. To validate the performance of these hyperbolic networks, we conducted extensive experiments on NTU RGB+D, NTU RGB+D 120 and Kinectics-Skeleton datasets for human action recognition.
Improved Hard Example Mining Approach for Single Shot Object Detectors

Hard example mining methods generally improve the performance of the object detectors, which suffer from imbalanced training sets. In this work, two existing hard example mining approaches (LRM and focal loss, FL) are adapted and combined in a state-of-the-art real-time object detector, YOLOv5. The effectiveness of the proposed approach for improving the performance on hard examples is extensively evaluated. The proposed method increases mAP by 3% compared to using the original loss function and around 1-2% compared to using the hard-mining methods (LRM or FL) individually on 2021 Anti-UAV Challenge Dataset.
Movie Rating Prediction Recommendation Algorithm based on XGBoost-DNN

In the traditional movie recommendation, because the features of users and movies are not considered, only the users' ratings of movies are considered, so there is a problem that the recommendation is not accurate enough. In response to this problem, this paper proposes a movie rating prediction recommendation algorithm based on XGBoost-DNN. First, XG-Boost is used to screen user features and movie features, and the features that have a great impact on movie rating prediction are screened out, and then the screened features are used as the input of DNN, the user network, and the movie network is trained to obtain the user feature vector and movie feature vector respectively, and then the user's predicted rating of the movie is obtained through the neural network, and finally compared with LightGBM, SVR, KNN, and RandomForest, this paper proposed XGBoost-DNN model reduces the MSE indicator by 0.223, 0.75, 0.451, and 0.306 respectively, which effectively improves the accuracy of rating prediction, and thus improves the accuracy of movie recommendation.
View position prior-supervised light field angular super-resolution network with asymmetric feature extraction and spatial-angular interaction

Light field imaging can record the intensity and direction information of light rays in space, which attracts extensive attention. However, the trade-off between angular and spatial resolution cannot be avoided due to the limitation of sensors in commercial light field cameras. To mitigate this problem, this paper proposes the view position prior-supervised light field angular super-resolution network with asymmetric feature extraction and spatial-angular interaction. First, there is a severe information asymmetry between the spatial and angular dimensions in light fields with sparse views. The asymmetric feature extraction block is proposed to extract spatial and angular features with different receptive fields in an asymmetric manner. As a result, more light field intrinsic features are extracted, which improves the utilization rate of the limited light field information. Second, the existing methods usually ignore the correlations among the newly synthesized views. The spatial-angular interaction module is proposed to collect the local and global information, build relations between any two points in the feature space, and reconstruct light field consistencies. Thus, the complete view correlations can be established. Last but not least, we investigate the impact of the given views on each viewpoint and propose a loss function based on the view position prior, which reduces the quality difference among the synthesized sub-aperture images and further improves the network performance. Comprehensive experiments demonstrate that our method can perform best on all datasets, and the depth estimation results present a new perspective to show the superiority of the proposed method.
The role of data analytics for detecting indications of fraud in the public sector

Technological developments play an important role in the audit process, one of which is the use of data analytics that are useful to assist auditors in analyzing data, collecting audit evidence, predicting risks that occur and will occur, and other things. The use of data analytics is also applied by public sector auditors to maintain accountability and responsibility for state finances. This study aims to examine the effect of using data analytics on indications of fraud for public sector examiners in Indonesia. Testing and data analysis techniques used STATA version 14, which processed answers from 33 auditors from two representative offices of public sector auditors in Java Province and Sumatra Province. The results of the study state that the use of data analytics has a positive and significant effect on indications of fraud for public sector examiners in the examination process. This means that public sector auditors can detect fraud using data analytics. Keywords: Audit Process; Public Sector Auditors; Data Analytics; Fraud Detection