Reviewerly/exHarmony huggingface dataset:
Embedding Uncertain Temporal Knowledge Graphs

Knowledge graph (KG) embedding for predicting missing relation facts in incomplete knowledge graphs (KGs) has been widely explored. In addition to the benchmark triple structural information such as head entities, tail entities, and the relations between them, there is a large amount of uncertain and temporal information, which is difficult to be exploited in KG embeddings, and there are some embedding models specifically for uncertain KGs and temporal KGs. However, these models either only utilize uncertain information or only temporal information, without integrating both kinds of information into the underlying model that utilizes triple structural information. In this paper, we propose an embedding model for uncertain temporal KGs called the confidence score, time, and ranking information embedded jointly model (CTRIEJ), which aims to preserve the uncertainty, temporal and structural information of relation facts in the embedding space. To further enhance the precision of the CTRIEJ model, we also introduce a self-adversarial negative sampling technique to generate negative samples. We use the embedding vectors obtained from our model to complete the missing relation facts and predict their corresponding confidence scores. Experiments are conducted on an uncertain temporal KG extracted from Wikidata via three tasks, i.e., confidence prediction, link prediction, and relation fact classification. The CTRIEJ model shows effectiveness in capturing uncertain and temporal knowledge by achieving promising results, and it consistently outperforms baselines on the three downstream experimental tasks.
Program Source-Code Re-Modularization Using a Discretized and Modified Sand Cat Swarm Optimization Algorithm

One of expensive stages of the software lifecycle is its maintenance. Software maintenance will be much simpler if its structural models are available. Software module clustering is thought to be a practical reverse engineering method for building software structural models from source code. The most crucial goals in software module clustering are to minimize connections between created clusters, maximize internal connections within clusters, and maximize clustering quality. It is thought that finding the best software clustering model is an NP-complete task. The key shortcomings of the earlier techniques are their low success rates, low stability, and insufficient modularization quality. In this paper, for effective clustering of software source code, a discretized sand cat swarm optimization (SCSO) algorithm has been proposed. The proposed method takes the dependency graph of the source code and generates the best clusters for it. Ten standard and real-world benchmarks were used to assess the performance of the suggested approach. The outcomes show that the quality of clustering is improved when a discretized SCSO algorithm was used to address the software module clustering issue. The suggested method beats the previous heuristic approaches in terms of modularization quality, convergence speed, and success rate.
Efficient Video Watermarking Algorithm Based on Convolutional Neural Networks with Entropy-Based Information Mapper

This paper presents a method for the transparent, robust, and highly capacitive watermarking of video signals using an information mapper. The proposed architecture is based on the use of deep neural networks to embed the watermark in the luminance channel in the YUV color space. An information mapper was used to enable the transformation of a multi-bit binary signature of varying capacitance reflecting the entropy measure of the system into a watermark embedded in the signal frame. To confirm the effectiveness of the method, tests were carried out for video frames with a resolution of 256 × 256 pixels, with a watermark capacity of 4 to 16,384 bits. Transparency metrics (SSIM and PSNR) and a robustness metric-the bit error rate (BER)-were used to assess the performance of the algorithms.
Cascaded feature fusion with multi-level self-attention mechanism for object detection

Object detection has been a challenging task due to the complexity and diversity of objects. The emergence of self-attention mechanism provides a new clue for feature fusion in object detection task. Most existing self-attention mechanisms focus on extracting the correlation between global and local information in space or among channels, however it remains problematic issues of how to effectively fuse all those features. To address the above problems, we propose a Pooling and Global feature Fusion Self-attention Mechanism (PGFSM) to capture multi-level correlations among a variety of features, so as to perform cascaded aggregations upon them. PGFSM consists of three parts: Spatial Self-attention Pooling Fusion Module (SSPFM), Channel Self-attention Pooling Fusion Module (CSPFM), and Spatial and Channel Global Self-attention Fusion Module (SCGSFM). SSPFM and CSPFM respectively carried out in space and channel, extract the global maximum pooling and global average pooling self-attention features; SCGSFM extracts the spatial and channel fused characteristic relationship in the global. Finally, the three fused feature relations are added on the original feature to achieve an enhanced trait representation. In test, our PGFSM is embedded into YOLOv4, YOLOv5, and EfficientDet network respectively, and evaluated in PASCAL VOC and MS COCO datasets. The experiment results show that the feature fusion self-attention mechanism improves the performance of object detection compared to each original framework and also the state-of-the-art modules, which proves the effectiveness of our method.
Rust-Style Patch: A Physical and Naturalistic Camouflage Attacks on Object Detector for Remote Sensing Images

Deep neural networks (DNNs) can improve the image analysis and interpretation of remote sensing technology by extracting valuable information from images, and has extensive applications such as military affairs, agriculture, environment, transportation, and urban division. The DNNs for object detection can identify and analyze objects in remote sensing images through fruitful features of images, which improves the efficiency of image processing and enables the recognition of large-scale remote sensing images. However, many studies have shown that deep neural networks are vulnerable to adversarial attack. After adding small perturbations, the generated adversarial examples will cause deep neural network to output undesired results, which will threaten the normal recognition and detection of remote sensing systems. According to the application scenarios, attacks can be divided into the digital domain and the physical domain, the digital domain attack is directly modified on the original image, which is mainly used to simulate the attack effect, while the physical domain attack adds perturbation to the actual objects and captures them with device, which is closer to the real situation. Attacks in the physical domain are more threatening, however, existing attack methods generally generate the patch with bright style and a large attack range, which is easy to be observed by human vision. Our goal is to generate a natural patch with a small perturbation area, which can help some remote sensing images used in the military to avoid detection by object detectors and im-perceptible to human eyes. To address the above issues, we generate a rust-style adversarial patch generation framework based on style transfer. The framework takes a heat map-based interpretability method to obtain key areas of target recognition and generate irregular-shaped natural-looking patches to reduce the disturbance area and alleviates suspicion from humans. To make the generated adversarial examples have a higher attack success rate in the physical domain, we further improve the robustness of the adversarial patch through data augmentation methods such as rotation, scaling, and brightness, and finally, make it impossible for the object detector to detect the camouflage patch. We have attacked the YOLOV3 detection network on multiple datasets. The experimental results show that our model has achieved a success rate of 95.7% in the digital domain. We also conduct physical attacks in indoor and outdoor environments and achieve an attack success rate of 70.6% and 65.3%, respectively. The structural similarity index metric shows that the adversarial patches generated are more natural than existing methods.
A levy flight based strategy to improve the exploitation capability of arithmetic optimization algorithm for engineering global optimization problems

Abstract The existing arithmetic optimization algorithm is a meta‐heuristics algorithm that utilizes distribution behaviors for the different parameters in mathematics. The different mathematical operator like division, subtraction, addition, and multiplication holds the inherent capability to explore global maxima and minima. In the proposed research, levy flight‐based improved arithmetic optimization algorithm has been proposed for better optimal solutions to various engineering design problems. The fundamental arithmetic optimization algorithm's local search is slow and has a slow convergence rate due to its weak exploitation capacity. In the proposed work, the exploration and exploitation phase of the existing arithmetic optimization algorithm has been enhanced using the levy flight mechanism. In order to validate the effectiveness of the proposed optimizer, the improved algorithm has been tested for 23 standard benchmark problems and 10 real‐life engineering design problems. The proposed algorithm has been compared with other classical algorithms like biogeography based optimization algorithm, arithmetic optimization algorithm, moth‐flame optimization algorithm, genetic algorithm, flower pollination algorithm, particle swarm optimization, gray wolf optimization algorithm, BAT algorithm, chi‐square algorithm, firefly algorithm, gravitational search algorithm, and differential evolution algorithm. The obtained result reveals that the proposed hybrid levy flight arithmetic optimization algorithm performs best on the number of test functions including engineering design problems with excellent fitness value and excellent convergence. This article is helpful to improve the exploitation capability of arithmetic optimization algorithms for engineering global optimization problems.
Learnable Human Mesh Triangulation for 3D Human Pose and Shape Estimation

Compared to joint position, the accuracy of joint rotation and shape estimation has received relatively little attention in the skinned multi-person linear model (SMPL)-based human mesh reconstruction from multi-view images. The work in this field is broadly classified into two categories. The first approach performs joint estimation and then produces SMPL parameters by fitting SMPL to resultant joints. The second approach regresses SMPL parameters directly from the input images through a convolutional neural network (CNN)-based model. However, these approaches suffer from the lack of information for resolving the ambiguity of joint rotation and shape reconstruction and the difficulty of network learning. To solve the aforementioned problems, we propose a two-stage method. The proposed method first estimates the coordinates of mesh vertices through a CNN-based model from input images, and acquires SMPL parameters by fitting the SMPL model to the estimated vertices. Estimated mesh vertices provide sufficient information for determining joint rotation and shape, and are easier to learn than SMPL parameters. According to experiments using Human3.6M and MPI-INF-3DHP datasets, the proposed method significantly outperforms the previous works in terms of joint rotation and shape estimation, and achieves competitive performance in terms of joint location estimation.
Semantic Segmentation with Active Semi-Supervised Learning

Using deep learning, we now have the ability to create exceptionally good semantic segmentation systems; however, collecting the prerequisite pixel-wise annotations for training images remains expensive and time-consuming. Therefore, it would be ideal to minimize the number of human annotations needed when creating a new dataset. Here, we address this problem by proposing a novel algorithm that combines active learning and semi-supervised learning. Active learning is an approach for identifying the best unlabeled samples to annotate. While there has been work on active learning for segmentation, most methods require annotating all pixel objects in each image, rather than only the most informative regions. We argue that this is inefficient. Instead, our active learning approach aims to minimize the number of annotations per-image. Our method is enriched with semi-supervised learning, where we use pseudo labels generated with a teacher-student framework to identify image regions that help disambiguate confused classes. We also integrate mechanisms that enable better performance on imbalanced label distributions, which have not been studied previously for active learning in semantic segmentation. In experiments on the CamVid and CityScapes datasets, our method obtains over 95% of the network’s performance on the full-training set using less than 17% of the training data, whereas the previous state of the art required 40% of the training data.
PSENet: Progressive Self-Enhancement Network for Unsupervised Extreme-Light Image Enhancement

The extremes of lighting (e.g. too much or too little light) usually cause many troubles for machine and human vision. Many recent works have mainly focused on under-exposure cases where images are often captured in low-light conditions (e.g. nighttime) and achieved promising results for enhancing the quality of images. However, they are inferior to handling images under over-exposure. To mitigate this limitation, we propose a novel unsupervised enhancement framework which is robust against various lighting conditions while does not require any well-exposed images to serve as the ground-truths. Our main concept is to construct pseudo-ground-truth images synthesized from multiple source images that simulate all potential exposure scenarios to train the enhancement network. Our extensive experiments show that the proposed approach consistently outperforms the current state-of-the-art unsupervised counterparts in several public datasets in terms of both quantitative metrics and qualitative results. Our code is available at https://github.com/VinAIResearch/PSENet-Image-Enhancement.
Learning Across Domains and Devices: Style-Driven Source-Free Domain Adaptation in Clustered Federated Learning

Federated Learning (FL) has recently emerged as a possible way to tackle the domain shift in real-world Semantic Segmentation (SS) without compromising the private nature of the collected data. However, most of the existing works on FL unrealistically assume labeled data in the re-mote clients. Here we propose a novel task (FFreeDA) in which the clients’ data is unlabeled and the server accesses a source labeled dataset for pre-training only. To solve FFreeDA, we propose LADD, which leverages the knowledge of the pre-trained model by employing self-supervision with ad-hoc regularization techniques for local training and introducing a novel federated clustered aggregation scheme based on the clients’ style. Our experiments show that our algorithm is able to efficiently tackle the new task out-performing existing approaches. The code is available at https://github.com/Erosinho13/LADD.
Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation

Unsupervised video object segmentation (VOS) aims to detect the most salient object in a video sequence at the pixel level. In unsupervised VOS, most state-of-the-art methods leverage motion cues obtained from optical flow maps in addition to appearance cues to exploit the property that salient objects usually have distinctive movements compared to the background. However, as they are overly dependent on motion cues, which may be unreliable in some cases, they cannot achieve stable prediction. To reduce this motion dependency of existing two-stream VOS methods, we propose a novel motion-as-option network that optionally utilizes motion cues. Additionally, to fully exploit the property of the proposed network that motion is not always required, we introduce a collaborative network learning strategy. On all the public benchmark datasets, our proposed network affords state-of-the-art performance with real-time inference speed. Code and models are available at https://github.com/suhwan-cho/TMO.
Perceptual Image Enhancement for Smartphone Real-Time Applications

Recent advances in camera designs and imaging pipelines allow us to capture high-quality images using smartphones. However, due to the small size and lens limitations of the smartphone cameras, we commonly find artifacts or degradation in the processed images. The most common unpleasant effects are noise artifacts, diffraction artifacts, blur, and HDR overexposure. Deep learning methods for image restoration can successfully remove these artifacts. However, most approaches are not suitable for real-time applications on mobile devices due to their heavy computation and memory requirements.In this paper, we propose LPIENet, a lightweight network for perceptual image enhancement, with the focus on deploying it on smartphones. Our experiments show that, with much fewer parameters and operations, our model can deal with the mentioned artifacts and achieve competitive performance compared with state-of-the-art methods on standard benchmarks. Moreover, to prove the efficiency and reliability of our approach, we deployed the model directly on commercial smartphones and evaluated its performance. Our model can process 2K resolution images under 1 second in mid-level commercial smartphones.
Semantic Segmentation in Aerial Imagery Using Multi-level Contrastive Learning with Local Consistency

Semantic segmentation in large-scale aerial images is an extremely challenging task. On one hand, the limited ground truth, as compared to the vast area the images cover, greatly hinders the development of supervised representation learning. On the other hand, the large footprint from remote sensing raises new challenges for semantic segmentation. In addition, the complex and ever changing image acquisition conditions further complicate the problem where domain shifting commonly occurs. In this paper, we exploit self-supervised contrastive learning (CL) methodologies for semantic segmentation in aerial imagery. In addition to performing CL at the feature level as most practices do, we add another level of contrastive learning, at the semantic level, taking advantage of the segmentation output from the downstream task. Further, we embed local mutual information in the semantic-level CL to enforce local consistency. This has largely enhanced the representation power at each pixel and improved the generalization capacity of the trained model. We refer to the proposed approach as multi-level contrastive learning with local consistency (mCL-LC). The experimental results on different benchmarks indicate that the proposed mCL-LC exhibits superior performance as compared to other state-of-the-art contrastive learning frameworks for the semantic segmentation task. mCL-LC also carries better generalization capacity especially when domain shifting exists.
Unsupervised Video Object Segmentation via Prototype Memory Network

Unsupervised video object segmentation aims to segment a target object in the video without a ground truth mask in the initial frame. This challenging task requires extracting features for the most salient common objects within a video sequence. This difficulty can be solved by using motion information such as optical flow, but using only the information between adjacent frames results in poor connectivity between distant frames and poor performance. To solve this problem, we propose a novel prototype memory network architecture. The proposed model effectively extracts the RGB and motion information by extracting superpixel-based component prototypes from the input RGB images and optical flow maps. In addition, the model scores the usefulness of the component prototypes in each frame based on a self-learning algorithm and adaptively stores the most useful prototypes in memory and discards obsolete proto-types. We use the prototypes in the memory bank to predict the next query frame’s mask, which enhances the association between distant frames to help with accurate mask prediction. Our method is evaluated on three datasets, achieving state-of-the-art performance. We prove the effectiveness of the proposed model with various ablation studies.
Discrete-modulation continuous-variable quantum key distribution with a high key rate

Abstract Discrete-modulation continuous-variable (CV) quantum key distribution has the potential for large-scale deployment in secure quantum communication networks owing to its low implementation complexity and compatibility with the current coherent optical telecommunication. However, current discrete-modulation protocols require relatively large constellation sizes to achieve a key rate comparable to that of the Gaussian modulation. Here, we show that a high key rate comparable to the Gaussian modulation can be achieved using only ten or so coherent states by implementing suitable key map and numerical convex optimization techniques. Specifically, the key rate of the two-ring constellation with 12 coherent states (four states in the inner ring and eight states in the outer ring) can reach 2.4 times of that of original quadrature phase shift keying and 70% of the Gaussian modulation protocol at 50 km. Such an approach can easily be applied to existing systems, making the discrete-modulation protocol an attractive alternative for high-rate and low-cost applications in secure quantum communication networks.
FedKL: Tackling Data Heterogeneity in Federated Reinforcement Learning by Penalizing KL Divergence

One of the fundamental issues for Federated Learning (FL) is data heterogeneity, which causes accuracy degradation, slow convergence, and the communication bottleneck issue. Although the impact of data heterogeneity on supervised FL has been widely studied, the related investigation for Federated Reinforcement Learning (FRL) is still in its infancy. In this paper, we first define the type and level of data heterogeneity for FRL systems. By inspecting the connection between the global and local objective functions, we prove that local training can benefit the global objective, if the local update is properly penalized by the total variation (TV) distance between the local and global policies. A necessary condition for the global policy to be learn-able from the local environments is also derived, which is directly related to the heterogeneity level. Based on the theoretical result, a Kullback-Leibler (KL) divergence based penalty is proposed to directly constrain the model outputs in the distribution space and the convergence proof of the proposed algorithm is also provided. By jointly penalizing the divergence of the local policy from the global policy with a global penalty and penalizing each iteration of the local training with a local penalty, the proposed method achieves a better trade-off between training speed (step size) and convergence. Experiment results on two popular Reinforcement Learning (RL) experiment platforms demonstrate the advantage of the proposed algorithm over existing methods in accelerating and stabilizing the training process with heterogeneous data.