Reviewerly/exHarmony huggingface dataset:
MFCosface: A Masked-Face Recognition Algorithm Based on Large Margin Cosine Loss

The world today is being hit by COVID-19. As opposed to fingerprints and ID cards, facial recognition technology can effectively prevent the spread of viruses in public places because it does not require contact with specific sensors. However, people also need to wear masks when entering public places, and masks will greatly affect the accuracy of facial recognition. Accurately performing facial recognition while people wear masks is a great challenge. In order to solve the problem of low facial recognition accuracy with mask wearers during the COVID-19 epidemic, we propose a masked-face recognition algorithm based on large margin cosine loss (MFCosface). Due to insufficient masked-face data for training, we designed a masked-face image generation algorithm based on the detection of the detection of key facial features. The face is detected and aligned through a multi-task cascaded convolutional network; and then we detect the key features of the face and select the mask template for coverage according to the positional information of the key features. Finally, we generate the corresponding masked-face image. Through analysis of the masked-face images, we found that triplet loss is not applicable to our datasets, because the results of online triplet selection contain fewer mask changes, making it difficult for the model to learn the relationship between mask occlusion and feature mapping. We use a large margin cosine loss as the loss function for training, which can map all the feature samples in a feature space with a smaller intra-class distance and a larger inter-class distance. In order to make the model pay more attention to the area that is not covered by the mask, we designed an Att-inception module that combines the Inception-Resnet module and the convolutional block attention module, which increases the weight of any unoccluded area in the feature map, thereby enlarging the unoccluded area’s contribution to the identification process. Experiments on several masked-face datasets have proved that our algorithm greatly improves the accuracy of masked-face recognition, and can accurately perform facial recognition with masked subjects.
Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation

Self-supervised monocular depth estimation has been widely studied, owing to its practical importance and recent promising improvements. However, most works suffer from limited supervision of photometric consistency, especially in weak texture regions and at object boundaries. To overcome this weakness, we propose novel ideas to improve self-supervised monocular depth estimation by leveraging cross-domain information, especially scene semantics. We focus on incorporating implicit semantic knowledge into geometric representation enhancement and suggest two ideas: a metric learning approach that exploits the semantics-guided local geometry to optimize intermediate depth representations and a novel feature fusion module that judiciously utilizes cross-modality between two heterogeneous feature representations. We comprehensively evaluate our methods on the KITTI dataset and demonstrate that our method outperforms state-of-the-art methods. The source code is available at https://github.com/hyBlue/FSRE-Depth.
Towards Real-World Prohibited Item Detection: A Large-Scale X-ray Benchmark

Automatic security inspection using computer vision technology is a challenging task in real-world scenarios due to various factors, including intra-class variance, class imbalance, and occlusion. Most of the previous methods rarely solve the cases that the prohibited items are deliberately hidden in messy objects due to the lack of large-scale datasets, restricted their applications in real-world scenarios. Towards real-world prohibited item detection, we collect a large-scale dataset, named as PIDray, which covers various cases in real-world scenarios for prohibited item detection, especially for deliberately hidden items. With an intensive amount of effort, our dataset contains 12 categories of prohibited items in 47, 677 X-ray images with high-quality annotated segmentation masks and bounding boxes. To the best of our knowledge, it is the largest prohibited items detection dataset to date. Meanwhile, we design the selective dense attention network (SDANet) to construct a strong baseline, which consists of the dense attention module and the dependency refinement module. The dense attention module formed by the spatial and channel-wise dense attentions, is designed to learn the discriminative features to boost the performance. The dependency refinement module is used to exploit the dependencies of multi-scale features. Extensive experiments conducted on the collected PIDray dataset demonstrate that the proposed method performs favorably against the state-of-the-art methods, especially for detecting the deliberately hidden items.
Semantic bug seeding: a learning-based approach for creating realistic bugs

When working on techniques to address the wide-spread problem of software bugs, one often faces the need for a large number of realistic bugs in real-world programs. Such bugs can either help evaluate an approach, e.g., in form of a bug benchmark or a suite of program mutations, or even help build the technique, e.g., in learning-based bug detection. Because gathering a large number of real bugs is difficult, a common approach is to rely on automatically seeded bugs. Prior work seeds bugs based on syntactic transformation patterns, which often results in unrealistic bugs and typically cannot introduce new, application-specific code tokens.
Role and Impact of Digital Technologies in E-Learning amidst COVID-19 Pandemic

Research has focused on the implementation of E-Learning amidst the COVID-19 pandemic. During COVID-19, the school and educational institutions were closed due to lockdown. During this period the classes of students are taken online. The digital technology used for e-learning during the COVID-19 pandemic has gained popularity in a very short period. Online classes are taken using Microsoft team, any desk, Zoom, WhatsApp applications. Educational contents are transferred frequently over the internet. Research is considering the impact of e-learning amidst COVID-19 and considering issues such as performance and security during transmission of digital content. The education is provided over cloud environment in a more secure manner with better performance. It has been observed that there have been several kinds of research in the area of cloud computing to provide online education. Issues in such research are performance and security of data. There is a need for a high-speed network to transfer educational content from one place to another. The educational contents needed to be secured and compressed at the time of data transfer. Cloud computing applications and the role of the cloud in e-learning are considered during research. The proposed work is supposed to integrate the proposed mechanism in the educational module. The proposed system is supposed to be secure and fast because data is compressed first then data is encrypted on the sender side. On receiving end the data is decrypted and decompressed. Delay in transmission issue is resolved because the size of data is less during transmission. Moreover, the packet dropping ratio gets reduced. The probability of cracking encrypted files also gets reduced as the data is encrypted after compression.
Unsupervised Depth Completion with Calibrated Backprojection Layers

We propose a deep neural network architecture to infer dense depth from an image and a sparse point cloud. It is trained using a video stream and corresponding synchronized sparse point cloud, as obtained from a LIDAR or other range sensor, along with the intrinsic calibration parameters of the camera. At inference time, the calibration of the camera, which can be different than the one used for training, is fed as an input to the network along with the sparse point cloud and a single image. A Calibrated Backprojection Layer backprojects each pixel in the image to three-dimensional space using the calibration matrix and a depth feature descriptor. The resulting 3D positional encoding is concatenated with the image descriptor and the previous layer output to yield the input to the next layer of the encoder. A decoder, exploiting skip-connections, produces a dense depth map. The resulting Calibrated Backprojection Network, or KBNet, is trained without supervision by minimizing the photometric reprojection error. KBNet imputes missing depth value based on the training set, rather than on generic regularization. We test KBNet on public depth completion benchmarks, where it outperforms the state of the art by 30% indoor and 8% outdoor when the same camera is used for training and testing. When the test camera is different, the improvement reaches 62%.
Deep learning detects acute myeloid leukemia and predicts NPM1 mutation status from bone marrow smears

The evaluation of bone marrow morphology by experienced hematopathologists is essential in the diagnosis of acute myeloid leukemia (AML); however, it suffers from a lack of standardization and inter-observer variability. Deep learning (DL) can process medical image data and provides data-driven class predictions. Here, we apply a multi-step DL approach to automatically segment cells from bone marrow images, distinguish between AML samples and healthy controls with an area under the receiver operating characteristic (AUROC) of 0.9699, and predict the mutation status of Nucleophosmin 1 (NPM1)-one of the most common mutations in AML-with an AUROC of 0.92 using only image data from bone marrow smears. Utilizing occlusion sensitivity maps, we observed so far unreported morphologic cell features such as a pattern of condensed chromatin and perinuclear lightening zones in myeloblasts of NPM1-mutated AML and prominent nucleoli in wild-type NPM1 AML enabling the DL model to provide accurate class predictions.
Self-supervised Video Representation Learning by Context and Motion Decoupling

A key challenge in self-supervised video representation learning is how to effectively capture motion information besides context bias. While most existing works implicitly achieve this with video-specific pretext tasks (e.g., predicting clip orders, time arrows, and paces), we develop a method that explicitly decouples motion supervision from context bias through a carefully designed pretext task. Specifically, we take the key frames and motion vectors in compressed videos (e.g., in H.264 format) as the supervision sources for context and motion, respectively, which can be efficiently extracted at over 500 fps on CPU. Then we design two pretext tasks that are jointly optimized: a context matching task where a pairwise contrastive loss is cast between video clip and key frame features; and a motion prediction task where clip features, passed through an encoderdecoder network, are used to estimate motion features in a near future. These two tasks use a shared video backbone and separate MLP heads. Experiments show that our approach improves the quality of the learned video representation over previous works, where we obtain absolute gains of 16.0% and 11.1% in video retrieval recall on UCF101 and HMDB51, respectively. Moreover, we find the motion prediction to be a strong regularization for video networks, where using it as an auxiliary task improves the accuracy of action recognition with a margin of 7.4% ∼ 13.8%.
Instruction-Set Accelerated Implementation of CRYSTALS-Kyber

Large scale quantum computers will break classical public-key cryptography protocols by quantum algorithms such as Shor’s algorithm. Hence, designing quantum-safe cryptosystems to replace current classical algorithms is crucial. Luckily there are some post-quantum candidates that are assumed to be resistant against future attacks from quantum computers, and NIST is considering standardizing them. Among these candidates, lattice-based cryptography sounds more interesting than others due to the performance results as well as confidence in the security. There are few works in the literature evaluating the performance of lattice-based cryptography in hardware. In this paper, we focus on Cryptographic Suite for Algebraic Lattices (CRYSTALS) key exchange mechanisms known as Kyber and provide an instruction-set hardware architecture and implement on Xilinx Artix-7 FPGA for performance evaluation and testing. Our proposed architecture provides an efficient and high-performance set of components to perform polynomial sampling, number-theoretic transform (NTT), and point-wise multiplication to speed up lattice-based post-quantum cryptography (PQC). This architecture implemented on ASIC outperforms state-of-the-art implementations.
Integrated MFFNN-MVO approach for PV solar power forecasting considering thermal effects and environmental conditions

Photovoltaic (PV) panels are commonly used as clean energy sources. However, their performance is sensitive to environmental conditions. In this work, a hybrid model of artificial neural networks (ANNs) and multiverse optimization (MVO)/genetic algorithm (GA) were implemented to predict PV output power, efficiency, and cell temperature. The main goal of the optimizer was to develop and update an ANN methodology based on training and forecasting. The numbers of neurons in the hidden layers, weights, and biases of the proposed ANNs were optimized with MVO and GA. The multilayer feedforward neural network (MFFNN) was used, and its accuracy was investigated through the results obtained from MFFNN-MVO and the MFFNN-GA models. Three parameters controlled the panel output power and temperature, and in turn, panel efficiency: ambient temperature, wind speed, and solar irradiance. The training and testing data were measured from a 4-kW PV plant installed in Shaqra City, Saudi Arabia, for two years. Moreover, the relationship between the PV panel efficiency and cell temperature was investigated. The PV efficiency relied on the sensitivity to the panel temperature factor (−0.06) and the maximum efficiency at 0 ⁰C (14.74). The efficiency of PV panels was predicted with a normalized root mean square error (NRMSE) of 3.65E-4 and 2.82E-4 for MFFNN-GA and MFFNN-MVO models, respectively.
Determination of drivers for investing in cryptocurrencies through a fuzzy full consistency method-Bonferroni (FUCOM-F’B) framework

Cryptocurrencies have brought many innovations and discussions to economic life. Digital assets, which are very popular by investors, are frequently used for many purposes such as store of value, exchange, and speculation. It creates a research area that intentions cryptocurrency experts prioritize in crypto investments. In this paper, therefore, the fuzzy Full Consistency Method-Bonferroni (FUCOM-F’B) model is conducted to determine the priorities of drivers for investing in cryptocurrencies. The selected twenty-three drivers are classified based on five aspects, including functionality, financial, legal infrastructure, technology, and security. Based on the findings, “strong electronic encryption” and “use of digital signature” are the most significant drivers for preferring a cryptocurrency. A validation check is performed to verify the reliability, usefulness, and stability of the proposed approach. Further, the introduced approach allows taking the ambiguities and subjectivity into account which exist in the decision-making procedure. The suggested framework can be a helpful decision support tool for regulators, policymakers, practitioners, and cryptocurrency investors.
DPSNet: Multitask Learning Using Geometry Reasoning for Scene Depth and Semantics

Multitask joint learning technology continues gaining more attention as a paradigm shift and has shown promising performance in many applications. Depth estimation and semantic understanding from monocular images emerge as a challenging problem in computer vision. While the other joint learning frameworks establish the relationship between the semantics and depth from stereo pairs, the lack of learning camera motion renders the frameworks that fail to model the geometric structure of the image scene. We make a further step in this article by proposing a multitask learning method, namely DPSNet, which can jointly perform depth and camera pose estimation and semantic scene segmentation. Our core idea for depth and camera pose prediction is that we present the rigid semantic consistency loss to overcome the limitation of moving pixels from image reconstruction technology and further infer the segmentation of moving instances based on them. In addition, our proposed model performs semantic segmentation by reasoning the geometric correspondences between the pixel semantic outputs and the semantic labels at multiscale resolutions. Experiments on open-source datasets and a video dataset captured on a micro-smart car show the effectiveness of each component of DPSNet, and DPSNet achieves state-of-the-art results in all three tasks compared with the best popular methods. All our models and code are available at https://github.com/jn-z/DPSNet: Multitask Learning Using Geometry Reasoning for Scene Depth and semantics.
Graph Clustering via Variational Graph Embedding

Graph clustering based on embedding aims to divide nodes with higher similarity into several mutually disjoint groups, but it is not a trivial task to maximumly embed the graph structure and node attributes into the low dimensional feature space. Furthermore, most of the current advanced methods of graph nodes clustering adopt the strategy of separating graph embedding technology and clustering algorithm, and ignore the potential relationship between them. Therefore, we propose an innovative end-to-end graph clustering framework with joint strategy to handle the complex problem in a non-Euclidean space. In terms of learning the graph embedding, we propose a new variational graph auto-encoder algorithm based on the Graph Convolution Network (GCN), which takes into account the boosting influence of joint generative model of graph structure and node attributes on the embedding output. On the basis of embedding representation, we implement a self-training mechanism through the construction of auxiliary distribution to further enhance the prediction of node categories, thereby realizing the unsupervised clustering mode. In addition, the loss contribution of each cluster is normalized to prevent large clusters from distorting the embedding space. Extensive experiments on real-world graph datasets validate our design and demonstrate that our algorithm has highly competitive in graph clustering over state-of-the-art methods.
An Adaptive Rapidly-Exploring Random Tree

Sampling-based planning algorithms play an important role in high degree-of-freedom motion planning (MP) problems, in which rapidly-exploring random tree (RRT) and the faster bidirectional RRT (named RRT-Connect) algorithms have achieved good results in many planning tasks. However, sampling-based methods have the inherent defect of having difficultly in solving planning problems with narrow passages. Therefore, several algorithms have been proposed to overcome these drawbacks. As one of the improved algorithms, Rapidly-exploring random vines (RRV) can achieve better results, but it may perform worse in cluttered environments and has a certain environmental selectivity. In this paper, we present a new improved planning method based on RRT-Connect and RRV, named adaptive RRT-Connect (ARRT-Connect), which deals well with the narrow passage environments while retaining the ability of RRT algorithms to plan paths in other environments. The proposed planner is shown to be adaptable to a variety of environments and can accomplish path planning in a short time.
A deep neural network-based method for deep information extraction using transfer learning strategies to support automated compliance checking

Existing automated compliance checking (ACC) systems require the extraction of requirements from regulatory documents into computer-processable representations. These information extraction (IE) processes are either fully manual, semi-automated, or automated. Semi-automated and manual approaches typically use manual annotations or predefined IE rules, which lack sufficient flexibility and scalability; the annotations and rules typically need adaptation if the characteristics of the regulatory document change. There is, thus, a need for a fully automated IE approach that can achieve high and consistent performance across different types of regulatory documents for supporting ACC. To address this need, this paper proposes a deep neural network-based method for deep IE – extracting semantic and syntactic information elements – from regulatory documents in the architectural, engineering, and construction (AEC) domain. The proposed method was evaluated in extracting information from multiple regulatory documents in the AEC domain. It achieved average precision and recall of 93.1% and 92.9%, respectively.
Mitigating Confounding Bias in Recommendation via Information Bottleneck

How to effectively mitigate the bias of feedback in recommender systems is an important research topic. In this paper, we first describe the generation process of the biased and unbiased feedback in recommender systems via two respective causal diagrams, where the difference between them can be regarded as the source of bias. We then define this difference as a confounding bias, which can be regarded as a collection of some specific biases that have previously been studied. For the case with biased feedback alone, we derive the conditions that need to be satisfied to obtain a debiased representation from the causal diagrams. Based on information theory, we propose a novel method called debiased information bottleneck (DIB) to optimize these conditions and then find a tractable solution for it. In particular, the proposed method constrains the model to learn a biased embedding vector with independent biased and unbiased components in the training phase, and uses only the unbiased component in the test phase to deliver more accurate recommendations. Finally, we conduct extensive experiments on a public dataset and a real product dataset to verify the effectiveness of the proposed method and discuss its properties.
Clothing Status Awareness for Long-Term Person Re-Identification

Long-Term person re-identification (LT-reID) exposes extreme challenges because of the longer time gaps between two recording footages where a person is likely to change clothing. There are two types of approaches for LT-reID: biometrics-based approach and data adaptation based approach. The former one is to seek clothing irrelevant biometric features. However, seeking high quality biometric feature is the main concern. The latter one adopts fine-tuning strategy by using data with significant clothing change. However, the performance is compromised when it is applied to cases without clothing change. This work argues that these approaches in fact are not aware of clothing status (i.e., change or no-change) of a pedestrian. Instead, they blindly assume all footages of a pedestrian have different clothes. To tackle this issue, a Regularization via Clothing Status Awareness Network (RCSANet) is proposed to regularize descriptions of a pedestrian by embedding the clothing status awareness. Consequently, the description can be enhanced to maintain the best ID discriminative feature while improving its robustness to real-world LT-reID where both clothing-change case and no-clothing-change case exist. Experiments show that RCSANet performs reasonably well on three LT-reID datasets.