Reviewerly/exHarmony huggingface dataset:
Joint Visual Perception and Linguistic Commonsense for Daily Events Causality Reasoning

Multimodal networks that juxtapose visual and linguistic modalities are currently widely adopted for solving vision-and-language tasks. They perform well in simple and intu-itive tasks, but are prone to mistakes in tasks involving latent or implicit details, due to the difficulty of capturing crucial but imperceptible visual signals in the real world. Perception errors lead to nonsensical results, but can be corrected by commonsense knowledge. To this end, we combine visual perception and linguistic commonsense to solve the challenging daily events causality reasoning task. We propose a novel Object-Aware Reasoning Network to focus on object inter-action while ignoring distracting information to refine visual perception. Further, a language branch with an independent prediction head is supervised to learn causality commonsense to help correct obvious perception errors, resulting in more plausible conclusions. Extensive experiments demonstrate that our method achieves new state-of-the-art results on Vis-Causal dataset.
Perceptual-Aware and Restorable Real-Time Image Downscaling

Image downscaling has been a classical problem and has recently been linked to super-resolution (SR). In this paper, we aim to propose a learning-based image downscaling model, FastDownscaler, which can efficiently produce low-resolution (LR) images that not only preserve the rich details of the original high-resolution images but also be highly restorable for existing SR models. We first present two separate lightweight networks with different upsampling losses, the bilinear loss and the bicubic loss, which are better for SR restoration and LR downscaling, respectively. To produce versatile LR images, we then propose to distill bilinear loss guided network with bicubic loss guided one. To our best knowledge, we establish the first image downscaling quality assessment dataset to evaluate the downscaling performance. Experimental results demonstrate the superior performance of the proposed model on image downscaling and SR. Furthermore, our model can achieve over 600 FPS for downscaling a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$1920\times 1280$</tex> image.
TC-ShadowGAN: A Target-Consistency Generative Adversarial Network for Unpaired Shadow Removal

Deep learning based shadow removal aims to learn a mapping that translates shadow image from shadow domain to non-shadow domain. In this work, we develop a sim-ple yet effective target-consistency generative adversarial net-work (TC-ShadowGAN) for the task of unpaired shadow re-moval without any additional label for supervision. Com-pared with the popular bidirectional mapping based methods, TC-ShadowGAN targets to directly learn a one-side mapping to translate shadow images into shadow-free ones. With the proposed target-consistency constraint that is designed to connect a couple of GAN-based sub-networks, the corre-lations between shadow image and the output shadow-free image, and the realness of recovered shadow-free image are strictly confined. Extensive quantitative and qualitative evalu-ation experiments results show that TC-ShadowGAN outper-forms the state-of-the-art unpaired shadow removal methods by 14.9% in terms of FID and 31.5% in terms of KID on the unpaired shadowremoval dataset. The code isavailable at https://github.com/chaotan/TC-GAN.
CMANET: Curvature-Aware Soft Mask Guided Attention Fusion Network for 2D+3D Facial Expression Recognition

As 2D texture and 3D structural information can describe facial features complementarily, 2D+3D facial expression recognition (FER) has received widespread attention. Though recent methods for 2D+3D FER have reached excellent performance, they still face two challenges: the way for attending to critical face areas and the strategy for fusing multi-modal information. To address these issues, we propose a curvature-aware soft mask guided attention fusion network (CMANet), which mainly consists of two components: curvature-aware attention module and multi-modal attention fusion module. The former utilizes the curvature-aware soft mask guiding the homo-modal attention mechanism to focus on potentially important areas with soft weights, while the latter applies pixel-level fusion on multi-modal features to retain the significant information from different modalities and also allows multi-modal features to interact in a larger field of view. Extensive experimental results show that our CMANet achieves outstanding accuracies (90.24% on BU-3DFE and 89.36% on Bosphorus) and outperforms the state-of-the-art methods.
D<sup>2</sup>S: Dynamic Distribution Supervision for Multi-Label Facial Expression Recognition

Due to facial images usually evoking multiple emotions with different intensities, there exists much ambiguity in facial expression recognition (FER). Previous methods jointly optimize multi-label learning (MLL) and label distribution learning (LDL) to suppress ambiguity, which have achieved excellent performance. However, the different convergence speed of MLL and LDL make the model easily over-fitting. To address this problem, we propose a dynamic distribution supervision (D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> S) method, where the label distribution information is introduced as auxiliary supervision for multilabel classification. Specifically, we develop a multi-task framework in which MLL and LDL are optimized simultaneously. The losses are dynamically weighted to overcome the inconsistency of inter-losses optimization between the two tasks. Extensive experiments on the largest benchmark dataset, i.e., RAF-ML, demonstrate the superiority of the proposed method.
A Design of Extensible Architecture Based on Consortium Blockchain

At present, the blockchain technology is often used to solve the security, privacy and other issues faced in the field of data sharing. But there are multiple implementations of the blockchain, each with their own advantages and disadvantages. In view of the many problems faced by the current data sharing field, it is obviously inappropriate to use a fixed technical solution, and a flexible and changeable method must be designed to meet different application requirements. This paper first analyzes the key technologies and classifications of blockchain. Since the scope of application of public blockchain and private blockchain is relatively narrow, this paper chooses the consortium blockchain as the underlying support for data sharing. Based on the consortium blockchain, this paper proposes an extensible blockchain framework to cope with changing scenarios and needs. Finally, the extensible architecture based on consortium blockchain is tested and analyzed, and the final result verifies the feasibility and effectiveness of the architecture proposed in this paper.
A Chinese-Malay Neural Machine Translation Model Based on CA-Transformer and Transfer Learning

Neural machine translation (NMT) has achieved good results in many applications, but large-scale corpora are needed as support, and high-quality corpora are often difficult to obtain, especially for low-resource corpora such as small languages, resulting in poor translation results. Therefore, a transfer learning method is proposed, in which the training parameters of Chinese-English high resource corpus are used to replace the model parameters of Chinese-Malay and English-Malay respectively, so as to solve the problem of insufficient training and scarce corpus. Compared with the basic transformer, this method can effectively reduce the number of parameters, speed up the training, and solve the problem that the pretrained language model is too large and difficult to optimize. Moreover, the performance is basically similar to transformer. Compared with multiple models, the translation model in this paper is faster, with better translation effect and better BLEU.
Anomaly Detection Method Based on Sample Constrained Training

According to the problems of the iForest algorithm which may cause the poor accuracy and efficiency by selecting root node samples and cutting points randomly, an anomaly detection algorithm named SCT-iTree is presented based on sample constrained training. In this algorithm. Firstly, the distribution form is considered when the training data set is extracted, which makes sure the training data has the same distribution with the original data set. Secondly, the split attribute is chosen according to the information entropy of the current sample data when the SCT-iTree is formed. Thirdly, the center of gravity of particle system is adopted to solve the split point of the attribute when selected the split point to make the point at the end where the data points are sparse. Through the constraints of the above three stages, the quality of the training tree is improved. The simulations shown that the SCT-iForest can improve iForest and LOF in aspects of detection quality and running speed of anomaly detection by preprocessing the sample data, which can obtain the higher quality training sample sets for isolate the outlier data. Especially, this algorithm has better stability than iForest for data sets with moderate amount of data and high dimensional data with large proportion of abnormal data.
Regularized Minimum Maximum Probability SVM Based on Radius-Margin Ratio

In this paper, the nonlinear support vector machine (I-FSVM) based on radius-margin ratio is improved to achieve the goal of data feature dimension reduction and data classification integration learning. On this basis, nonlinear feature regularized minimum maximum probability support vector machine (nonlinear FRMPSVM) is proposed. The experimental results show that the classification performance of nonlinear I-FSVM is better than that of F-SVM and nonlinear SVM, and the classification performance of nonlinear FRMPSVM is not only better than that of nonlinear RMPSVM, but also relatively stable.
FMP-Net: Fractal Multi-Gate Mixture-of-Experts Panoramic Segmentation for Point Cloud

Panoramic segmentation in LiDAR point clouds is in its nascent stage, which unifies the semantic and instance seg-mentation tasks that receive increasing attentions. This paper proposed a fractal multi-gate mixture-of-experts panoramic segmentation network (FMP-Net) consists of encode and de-code, which is able to perform panoramic segmentation tasks on large-scale scenes point clouds. The fractal multi-gate mixture-of-experts (FMMoE) module is proposed in the de-code part, which integrates the multi-gate mixture-of-experts (MMoE) module and binary tree. The FMMoE module is able to divide the features extracted by the encoder step by step, eventually determining the features to be used for each sub-task in panoramic segmentation. We used a hierarchical clus-tering algorithm to obtain instance information while improving the overall network inference speed. The experimental results illustrated that our proposed method achieve Panoptic Quality(PQ) of 40.7% and the Intersection over Union(mIoU) of 62.3% on the public dataset NuScenes. The source code is available at: https://github.com/ChazYang/FMP-Net.
Enhanced Semantic Head for Cascade Instance Segmentation

Recently, cascade instance segmentation inspired by cascade object detection has achieved notable performance. Due to the lack of global information, many methods suffer from incomplete segmentation such as missing edge regions and discontinuities within instances. To solve this problem, we proposed an effective and flexible semantic head to extract enhanced spatial context information. A vision transformer is utilized to generate global context features, and a convolution network is adopted to generate spatial context features. After combining the two modules, we obtain enhanced semantic segmentation features for segmentation. Extensive experiments show that the enhanced semantic head achieves 40.6% and 42.3% mask AP for cascade predictor HTC and DSC, which surpass about 0.9 and 1.4 percentage points respectively. The enhanced semantic head is universal and effective to improve the performance of different cascade predictors.
Usability Evaluation of Wedding Administrative Information System using System Usability Scale

The Marriage Administration Information System at the Religious Affairs Office of Lubuk Barumun District is a system that is run by the Admin of the Religious Affairs Office of Lubuk Barumun District and the Community in Lubuk Barumun District. The main function provided by this marriage administration information system is to facilitate the Religious Affairs office in compiling data and delivering information to the public, this system can also facilitate the public in submitting marriage requirements and reconciliation files. In using the marriage administration information system, some people are still confused in running this marriage administration information system. Therefore, this study aims to evaluate the usefulness of the marriage administration information system using the System Usability Scale (SUS) method to determine the extent of the effectiveness, efficiency and satisfaction of using the marriage administration information system that has been built.
HMAI-BERT: Hierarchical Multimodal Alignment and Interaction Network-Enhanced BERT for Multimodal Sentiment Analysis

Human language is multimodal, including textual, visual and acoustic information. The task of multimodal sentiment analysis is to use human multimodal information for sentiment recognition. Among the three modalities, text contains richer information than other modalities. With the development of pre-trained representation models on text, most of multimodal sentiment analysis methods use text as primary information and the other modalities as supplementary information. The existing methods suffer from the following limitations: 1) inherent heterogeneity of multimodal data, which makes multimodal fusion difficult as different modalities reside in different feature spaces; 2) asynchronism caused by the inconsistent sampling rates of the time series data of different modalities. To alleviate the heterogeneity and asynchronism of multimodal data, we propose HMAI-BERT, a hierarchical multimodal alignment and interaction network-enhanced BERT. In HMAI-BERT, to improve the efficiency of multimodal interaction, we introduce a memory network to align the different multimodal representations before fusion. After multimodal alignment, we propose a modal update method to address the problem of asynchronism, where each modality is reinforced by interacting with other modalities. In addition, we introduce a fusion module to integrate the three reinforced modalities, and a sentiment enhanced memory to enhance multimodal representation. Our experiments on two public datasets show that the proposed HMAI-BERT outperforms the state-of-the-art methods.
Feature Decoupled Autoencoder: Semi-Supervised Learning for Image Dehazing

Single image dehazing plays an important role in image processing. Now, most image dehazing methods only use synthetic datasets to train models which produce some poor results on the real hazy images. To solve this problem, we propose a new network named feature decoupled autoencoder which can be trained by semi-supervised learning due to the real hazy images lacking labels. Our autoencoder includes a feature decoupled encoder and a feature fusion decoder. The clear image feature and the hazy feature are decomposed by the encoder and merged by the decoder. When the hazy feature is set to 0, the decoder will produce a dehazing image relying on the clear image feature. By training on the synthetic datasets and real datasets together, our model has good dehazing ability on the real-world hazy images. On the HazeRD and SOTS, our model's performance is comparable to the state-of-the-art algorithms.
Improved Path Planning Algorithm Based on RRT Algorithm and Quintic B-spline Curve

Aiming at the problems of strong randomness, unsmooth path and local optimum in the path planning of the Rapidly-exploring Random Trees (RRT) algorithm. The paper designs an improved RRT algorithm with variable step size and goal orientation (VO-RRT), which introduces variable step size, oriented expansion node, optimized initial path and smoothed path four parts. Through several experiments in 2D and 3D space to prove that the VO-RRT algorithm has enhanced obstacle avoidance ability and improved path smoothness in different obstacle spaces. The path nodes generated by the VO-RRT algorithm in 3D space are tested in URsim simulation software and manipulator body to verify the executability of the optimized path.
Human vs. Automatic Detection of Deepfake Videos Over Noisy Channels

Identification of DeepFake video content is a challenging scientific problem that addresses a growing societal concern. We investigate the relationship between DeepFake detection by humans and by automatic methods based on state-of-the-art deep learning algorithms. The main novelty of our work is the consideration of videos that are transmitted through noisy channels and arrive with distortions. This reflects many practical environments, including surveillance based on cameras connected via noisy wireless links and videoconferencing in driving vehicles. We conduct a user study with 192 probands who classify real (genuine) and DeepFake videos with and without various classes of distortions. We find that today's deep neural networks (DNNs) outperform humans by far, whereas humans are heavily distracted by random noise from the channel. Moreover, DNNs are robust under distortions, achieving perfect classification on distorted data even when trained on distortion-free content. It appears that the human visual system and DNNs are approaching the DeepFake classification problem quite differently and their respective strengths and weaknesses are largely uncorrelated.
Pruning Based Training-Free Neural Architecture Search

Neural Architecture Search (NAS) plays an important role in searching for high-performance neural networks. How-ever, NAS algorithms are slow and require a terrific amount of computing resources, because they need to be trained on supernet or dense candidate networks to obtain information for evaluation. If the high-performance network architecture could be selected without training, it would eliminate a signif-icant part of the computational cost. Therefore, we propose a zero-cost metric called EX-score, which can represent the ex-pressivity of the network and rank the untrained architectures. To further reduce cost, we design a pruning based zero-cost neural architecture search framework (PZ-NAS) using EX-score. PZ-NAS can prune the initialised supernet rapidly and obtains hundreds of times faster speed performance, whilst archieving comparable accuracy property on CIFAR-IO and ImageNet.
Region-Interactive Proposal Network and Class-Interactive Feature Learning for Few-Shot Object Detection

Few-shot object detection is a promising approach to solving the problem of detecting novel objects with only limited annotated data for training. Most existing methods are developed based on the progress in few-shot classification, which pay little attention to improving the localization module and modelling class interrelation. To address these issues, this paper proposes two novel modules, namely Region-interactive Proposal Network (Ri-PN) and Class-interactive Feature Learning (Ci-FL), for better localization and classification performance, respectively. In the Ri-PN, regions of novel classes are interacted with base classes via graph convolution instead of background due to the stronger relevance between base and novel classes together with the guidance of supervised regions loss. On the other hand, the Ci-FL refines class-specific features in prototypical learning by attentive graph convolutional network. Experimental results on PASCAL VOC and MS COCO datasets verify the superiority of our method for few-shot object detection.
The Topic Evolution of Danmaku Text Based on BTM -- Taking the Context of COVID-19 as an Example

The danmaku text is a popular interactive mode of instantaneous video comment. Websites with danmaku texts have been widely used recently, and this vast number of texts can be regarded as a short text mining resource. This paper uses Perplexity and Rényi entropy to evaluate the BTM (Biterm Topic Model), by extracting the topic from the danmaku texts to explore the evolution of danmaku text topics in videos relevant to COVID-19. The results show that Rényi entropy is an effective way to decide the optimal number of topics, and the topics captured by BTM indicate that video viewers showed positive attitudes in the face of this public health emergency.
SIM: A Scenario IMagination Based Deep Reinforcement Learning Method for Outdoor Transportation Environment Exploration

Autonomous exploration is very important for robotics, especially for mapping, navigation, and planning in an unknown environment. In recent years, automatic exploration methods in the indoor environment have been extensively studied, but there is little research on exploration in the outdoor transportation environment. Due to the limitations of outdoor traffic rules and the scale of the environment, the methods for the indoor environment are difficult to apply to the transportation environment. Aiming at exploration in the transportation environment, this paper proposes a deep reinforcement learning algorithm based on Scenario IMagination(SIM), which has two important components: 1) a mid-level action space, which combines the classical robot control algorithm, addressing the inefficient learning and unstable navigation of deep reinforcement learning algorithms in automatic exploration. With this action space, the deep reinforcement learning algorithm achieves excellent exploration performance in both normal scale environments and large-scale branchless environments; 2) a scenarios buffer, which relieves hard exploration problems of deep reinforcement learning due to serious imbalances of samples in large-scale multibranch scenarios. Compared to the map-less navigation approaches, SIM achieves excellent exploration performance in large-scale multi-branch environments.
Deixis Analysis on Joe Biden’s Speech “Remarks on Afghanistan”

In communication, a speaker sometime uses language features which to point or indicate objects, places, persons, times, and text passages in a specific context. This language features are called deixis or deictic expression. The aim of this study is to find out and describe what kind of deixis and reference used by Joe Biden in his speech. In conducting the study, the researcher conducts the descriptive qualitative method because the result and discussion of this study is presented in a word and descriptive form and focused on quality of the researcher’s interpretation. This study applied the classification of deixis by Stephen C. Levinson (1983). There are 203 words of person deixis, 54 data of place deixis, 45 data of time deixis, 12 data of social deixis, and 31 data od discourse deixis. The result show that person deixis is mostly use by Joe Biden in his speech. This implies that the speech is considered as important so that the speaker express himself as the president as the center of the speech. The result show that the speech by Joe Biden is not only contains of deixis, but it also contains of reference. The result show that there are two types of reference in Joe Biden’s speech, namely Personal reference and demonstrative reference. This study also applied the theory of reference by Halliday and Hasan (1976). This implies that the speech is not only attract people to understand of what, who, when, and where the speaker means, but the speech is also considered as a text and meaningful.