Reviewerly/exHarmony huggingface dataset:
Deep Autoencoder-like NMF with Contrastive Regularization and Feature Relationship Preservation

Nonnegative Matrix Factorization is a data analysis method to discover parts-based, linear representations of data. It has been successfully used in a great variety of applications. Deep Nonnegative Matrix Factorization (deep NMF) was recently established to cope with the extraction of hierarchical latent feature representation, and it has been demonstrated to achieve outstanding results in unsupervised representation learning. However, defining a suitable regularization for the deep models is a key challenge, and the existing Deep NMF approaches lack a well-suited regularization. In this paper, we propose the Deep Autoencoder-like NMF with Contrastive Regularization and Feature Relationship preservation (DANMF-CRFR) to address the above problem. Inspired by contrastive learning, this deep model is able to learn discriminative and instructive deep features while adequately enforcing the local and global structures of the data to its decoder and encoder components. Meanwhile, DANMF-CRFR also imposes feature correlations on the basis matrices during feature learning to improve part-based learning capabilities. Multiplicative updating rules and convergence guarantees are also provided. Extensive experimental results demonstrate the advantages of the proposed model. The source code for reproducing our results can be found at https://github.com/NavidSalahian/DANMF_CRFR.
Understanding and defending against White-box membership inference attack in deep learning

Membership inference attacks (MIA) exploit the fact that deep learning algorithms leak information about their training data through the learned model. It has been treated as an indicator which reveals the privacy leakage of machine learning models. In this work, we aim to understand the advantage achieved by White-box MIA, and defend against White-box MIA. Firstly, we estimate the KL divergence on the hidden layers’ features between training set and test set as the internal generalization gap. By comparing the internal generalization gap and the generalization gap on the output layer, we raise two insights including (1) the existence of larger generalization gaps on hidden layers and (2) feasibility of generalization gap minimization in defending White-box MIA. Based on our insights, we further design a novel defense method named Nirvana. It intentionally minimizes generalization gap to defend White-box MIA. Formally, Nirvana works by selecting a hidden layer with large generalization gaps and executing a multi-samples convex combination among features on the layer during the training to defend against White-box MIA. Finally, we empirically evaluate Nirvana with state-of-the-art defense methods on CIFAR100 dataset, Purchase100 dataset, and Texas dataset. The experiment results show that Nirvana achieves a trade-off between utility and privacy. It can defend both White-box MIA and Black-box MIA while the test accuracy of the model is maintained. It outperforms previous defense methods in defending against White-box MIA.
DNN self-embedding watermarking: Towards tampering detection and parameter recovery for deep neural network

In recent years, a large number of deep neural networks (DNN) models have been built and deployed, which need to be protected against malicious tampering by the adversary. This work is the first to propose a recoverable, self-embedding fragile watermarking scheme for DNN models to protect the model integrity. This scheme can not only identify and locate the tampered parameter blocks in the model, but can also recover the damaged parameters accurately. Detailedly, through exploiting the characteristics of the to-be-protected DNN model, the authentication data and recovery data are generated, and then the reference sharing mechanism is used to embed these data into the model without affecting its original functionality, which can realize the model parameter recovery under different tampering rates. Experimental results demonstrate that, the proposed scheme can achieve satisfactory performance of tampering detection and parameter recovery with low device requirements and can be effectively adaptable to a variety of existing DNNs.
GramBeddings: A New Neural Network for URL Based Identification of Phishing Web Pages Through N-gram Embeddings

There has been ever-growing use of Internet and progress within many communication channels such as social media and this escalates the need for rapid and low source demanding phishing detection mechanisms. In this very study, we propose a new deep neural model for phishing URL identification so-called GramBeddings introducing some distinguishing novelties by (1) proposing the use of n-gram embeddings, computed on the fly, requiring no pre-training stage, (2) removing the necessity of word and sub-word level information, (3) providing a smart and efficient n-gram selection pipeline, and benefiting from attention mechanism. Other than that, we share a publicly available, large-scale and novel dataset2 including 800K real-world phishing and legitimate URLs. Our scheme suggests an adjustable and automated n-gram selection and filtering mechanism along with a new neural network architecture concatenating four-channel information flow through cascading CNN, LSTM, and attention layers. With that, discriminative multi-level character patterns can be discovered without any hand-crafted operation and are enabled to contribute to prediction. As a result, the proposed system provides the following features in the problem domain: (i) real-time, end-to-end and high performance inference, (ii) language-agnostic prediction, and (iii) removal of the necessity of any third-party service or hand-crafted feature. These experiments show that our approach outperforms the other models in the literature with an accuracy of 98.27%. Moreover, the comparative study conducted with several datasets clearly verifies the superiority of our model in all tests. We also examine the robustness of our model against a real-world adversarial attack and discuss the methods of overcoming such an attack. Our codebase3 is shared with the community to be used for benchmarking purposes in the future.
Partially-federated learning: A new approach to achieving privacy and effectiveness

In Machine Learning, the data for training the model are stored centrally. However, when the data come from different sources and contain sensitive information, we can use federated learning to implement a privacy-preserving distributed machine learning framework. In this case, multiple client devices participate in global model training by sharing only the model updates with the server while keeping the original data local. In this paper, we propose a new approach, called partially-federated learning, that combines machine learning with federated learning. This hybrid architecture can train a unified model across multiple clients, where the individual client can decide whether a sample must remain private or can be shared with the server. This decision is made by a privacy module that can enforce various techniques to protect the privacy of client data. The proposed approach improves the performance compared to classical federated learning.
Generating BPMN diagram from textual requirements

An interesting challenge in software requirements engineering is converting textual requirements to Business Process Model and Notation (BPMN) diagrams. In this study, the BPMN diagram is used as an intermediate representation before measuring the functional software size from Natural Language (NL) input. The methods currently used for converting NL input to BPMN diagrams are not able to generate complete BPMN diagrams, nor can they handle complex and compound-complex sentences in the NL input. This study proposes conversion from textual requirements to a BPMN diagram for improving the weaknesses of existing methods. The proposed method has two stages: 1) analyzing the textual requirements using natural language processing and 2) generating the BPMN diagram. The output of the first stage is fact types as the basis for generating the BPMN diagram in the second phase. The BPMN diagram is generated using a set of informal mapping rules that were created in this study. The proposed method was applied to ten textual requirements of an enterprise application, which involved simple, compound, complex, and compound-complex sentences. The experiments resulted in a suitable BPMN diagram with higher accuracy than obtained by other methods.
A comprehensive experiment-based review of low-light image enhancement methods and benchmarking low-light image quality assessment

Low-light image enhancement is a notoriously challenging problem. Enhancement of low-light images is intended to increase contrast, adjust the tone, suppress noise, and produce better aesthetic quality images. Over the years, a large number of state-of-the-art classical and deep learning-based methods have been designed. The purpose of this experimental review is to examine the generalization ability of these methods. In order to examine the generalization ability of these methods (classical and deep learning-based) empirical analysis and relevant comparisons are performed on a wide range of commonly used test datasets using a variety of evaluation techniques. In addition, we have studied the inconsistency of low-light evaluation methods and highlighted their drawbacks. This inconsistency raises the question of whether the existing methods are able to fairly evaluate enhancement methods. In order to address this question, we propose a large no-reference perceptual image quality assessment (PIQA) dataset. Additionally, different deep learning-based methods have been trained on this PIQA dataset in order to provide benchmarking for developing learning-based low-light assessment methods. Finally, this review paper is concluded with current challenges and suggestions for future work.
A novel image enhancement algorithm to determine the dust level on photovoltaic (PV) panels

An accurate evaluation of the dust accumulation on photovoltaic (PV) panels enables the development of cleaning plans and improves the grid connection security of PV power stations. In this paper, a novel image enhancement algorithm is proposed to evaluate the dust accumulation on PV panels. An atmospheric scattering model was used to analyze the difference in the image characteristics of clean and dusty PV panels. A model was proposed to describe the relationship between the model coefficient and the dust level by minimizing the pixel difference between the images of the soiled and clean PV panels. An experiment system was designed to acquire the images of clean and soiled PV panels with different dust levels. The model parameters were determined by data fitting, and the model’s generalization ability was improved by adding a noise item. The results show that the model can estimate the dust level on PV panels with an accuracy of 83.78%. The average root mean square error (RMSE) of samples 1–4 is 1.67, and the error range is 2–3.5 g/m2. In addition, the proposed method has higher accuracy than the optical attenuation and weighing methods, with a mean absolute error (MAE) of 1.61 g/m2. The method is simple and reliable for intelligent cleaning of PV panels.
Game theory based Bi-domanial deep subspace clustering

During high-dimensional subspace clustering, data is generally corrupted, which destroys the self-expressive property and results in self-expressive noise, adversely affecting the clustering process. Compared with data noise, self-expressive noise is complex, and its exact estimate is a difficult problem. This paper proposes a Bi-domanial Deep Subspace Clustering model (BD-DSC), which is a coupled game model. BD-DSC leverages two networks to extract the clean data in the reconstructed spatial domain and clean features in the feature domain respectively. The ability of the deep network to clean data and features allows us to avoid estimating self-expressive noise and efficiently construct two self-expressive models in both domains. Furthermore, game theory is used to couple these two self-expressive models in order to achieve efficient information interaction and increase the robustness and stability of the results. By means of co-supervising each other during training, a three-stage algorithm is provided to solve the game model. Extensive experiments have been conducted on four benchmark datasets. Owing to avoiding the self-expressive noise, fusing two modules dynamically and choosing the optimal parameters carefully, our proposed approach is robust to data noise and significantly superior to the state-of-the-art subspace clustering methods with the lowest clustering errors.
3D Vehicle Detection and Segmentation Based on EfficientNetB3 and CenterNet Residual Blocks

In this paper, we present a two stages solution to 3D vehicle detection and segmentation. The first stage depends on the combination of EfficientNetB3 architecture with multiparallel residual blocks (inspired by CenterNet architecture) for 3D localization and poses estimation for vehicles on the scene. The second stage takes the output of the first stage as input (cropped car images) to train EfficientNet B3 for the image recognition task. Using predefined 3D Models, we substitute each vehicle on the scene with its match using the rotation matrix and translation vector from the first stage to get the 3D detection bounding boxes and segmentation masks. We trained our models on an open-source dataset (ApolloCar3D). Our method outperforms all published solutions in terms of 6 degrees of freedom error (6 DoF err).
Automatic facial emotion recognition at the COVID-19 pandemic time

People use various nonverbal communicative channels to convey emotions, among which facial expressions are considered the most important ones. Thus, automatic Facial Expression Recognition (FER) is a fundamental task to increase the perceptive skills of computers, especially in human-computer interaction. Like humans, state-of-art FER systems are able to recognize emotions from the entire face of a person. However, the COVID-19 pandemic has imposed a massive use of face masks that help in preventing infection but may hamper social communication and make the recognition of facial expressions a very challenging task due to facial occlusion. In this paper we propose a FER system capable to recognize emotions from masked faces. The system checks for the presence of a mask on the face image and, in case of mask detection, it extracts the eyes region and recognizes the emotion only considering that portion of the face. The effectiveness of the developed FER system was tested in recognizing emotions and their valence only from the eyes region and comparing the results when considering the entire face. As it was expected, emotions that are related mainly to the mouth region (e.g., disgust) are barely recognized, while positive emotions are better identified by considering only the eyes region. Moreover, we compared the results of our FER system to the human annotation of emotions on masked faces. We found out that the FER system outperforms the human annotation, thus showing that the model is able to learn proper features for each emotion leveraging only the eyes region.
Complementary features based prototype self-updating for few-shot learning

The goal of few-shot learning is to use limited labeled samples to complete independent classification tasks. The feature extractor of few-shot learning needs to have a stronger feature expression ability to generalize in unseen novel classes. To further enhance the expressive ability, in this paper, we propose an inherited feature extraction method, named Base and Meta Feature Extraction (BMFE). Base feature represents the task-irrelevant classification information of each sample. Meta feature obtained by the proposed Triplet Meta-train Mechanism (TMM) inherits the classification information and also contains the task-related meta information of each sample. We concatenate both the base and meta features to complementarily express the rich information of each sample. Besides, instead of relying on limited support samples to obtain the prototype, we propose a novel unsupervised prototype correction module, named Prototype Self-updating (PSU). All unlabeled query samples in a few-shot test task participate in the iterative updating of each prototype in the task without training. Extensive experiments prove that our overall method can obtain richer features by BMFE and more accurate prototypes by PSU. Our overall method outperforms state-of-the-art methods on miniImageNet and tiredImageNet datasets, and especially under the 1-shot case we obtains 78.45% and 81.21% classification accuracy respectively.
A privacy-preserving scheme to support the detection of multiple similar request-real-time services in IoT application systems

Many Internet of Things (IoT) network and security studies have suggested to design a secure transmission scheme based on cryptographic methodology to guarantee that a transmitted requested service is securely delivered to the end-user. However, most of the IoT security schemes presented in the literature focus only on protecting the transmission data between sender and receiver without considering any potential cost that may result from their schemes. Cryptographic mathematical operation always incur some cost in transforming plaintext to ciphertext that may result in increasing the latency of providing the IoT application in real-time to end-user. With sending thousands of queries messages, encryption, verifying and decrypting each received message separately will result in a time delay, especially when a large number of similar queries are received at the same time. Therefore, applying incompatible cryptographic schemes in an IoT network system may not be practical for protecting IoT applications. In this work, we design a lightweight privacy-preserving (LPP) scheme to protect the IoT transmission network system with a feature for reducing the latency issues resulting from cryptographic computation cost on each ciphertext. The results of a security analysis substantiate the ability of the proposed scheme to withstand a range of security attacks against the IoT transmission network. Additionally, these results show that the proposed scheme registers a high efficiency in reducing both ciphertext size and computational cost.
A survey of large-scale graph-based semi-supervised classification algorithms

Semi-supervised learning is an effective method to study how to use both labeled data and unlabeled data to improve the performance of the classifier, which has become the hot field of machine learning in recent years. Graph-based semi-supervised learning is very promising among these Semi-supervised methods for its good performance and universality, it represents all the data as a graph and the label information is propagated from the labeled data to the unlabeled data along with the constructed graph. To improve the scalability of graph-based semi-supervised methods for large-scale data, an increasing number of methods with granulation mechanisms are proposed. However, there exist few papers to solely analyze the recent research progress. Following the process of graph-based semi-supervised learning, this paper concludes with the fundamental principles of the granulation mechanism for graph construction and label inference respectively. Moreover, a new taxonomy is generated according to the granulation criterion for each process. Therefore, it provides an important reference for the use and research of semi-supervised classification algorithms facing massive data.
Advanced Dipper-Throated Meta-Heuristic Optimization Algorithm for Digital Image Watermarking

Recently, piracy and copyright violations of digital content have become major concerns as computer science has advanced. In order to prevent unauthorized usage of content, digital watermarking is usually employed. This work proposes a new approach to digital image watermarking that makes use of the discrete cosine transform (DCT), discrete wavelet transform (DWT), dipper-throated optimization (DTO), and stochastic fractal search (SFS) algorithms. The proposed approach involves computing the discrete wavelet transform (DWT) on the cover image to extract its sub-components, followed by the performance of a discrete cosine transform (DCT) to convert these sub-components into the frequency domain. Finding the best scale factor for watermarking is a significant challenge in most watermarking methods. The authors used an advanced optimization algorithm, which is referred to as DTOSFS, to determine the best two parameters—namely, the scaling factor and embedding coefficient—to be used while inserting a watermark into a cover image. Using the optimal values of these parameters, a watermark image can be inserted into a cover image more efficiently. The suggested approach is evaluated in comparison with the current gold standard. The normalized cross-correlation (NCC), peak-signal-to-noise ratio (PSNR), and image fidelity (IF) are used to measure the success of the proposed approach. In addition, a statistical analysis is performed to evaluate the significance and superiority of the proposed approach. The experimental results confirm the effectiveness of the proposed approach in improving upon standard watermarking methods based on the DWT and DCT. Moreover, a set of attacks is considered to study the robustness of the proposed approach, and the results confirm the expected outcomes. It is shown by the achieved results that the proposed approach can be utilized for practical digital image watermarking, and that it significantly outperforms other digital image watermarking methods.
Systematic review of content analysis algorithms based on deep neural networks

Today according to social media, the internet, Etc. Data is rapidly produced and occupies a large space in systems that have resulted in enormous data warehouses; the progress in information technology has significantly increased the speed and ease of data flow.text mining is one of the most important methods for extracting a useful model through extracting and adapting knowledge from data sets. However, many studies have been conducted based on the usage of deep learning for text processing and text mining issues.The idea and method of text mining are one of the fields that seek to extract useful information from unstructured textual data that is used very today. Deep learning and machine learning techniques in classification and text mining and their type are discussed in this paper as well. Neural networks of various kinds, namely, ANN, RNN, CNN, and LSTM, are the subject of study to select the best technique. In this study, we conducted a Systematic Literature Review to extract and associate the algorithms and features that have been used in this area. Based on our search criteria, we retrieved 130 relevant studies from electronic databases between 1997 and 2021; we have selected 43 studies for further analysis using inclusion and exclusion criteria in Section 3.2. According to this study, hybrid LSTM is the most widely used deep learning algorithm in these studies, and SVM in machine learning method high accuracy in result shown.
Fall detection based on OpenPose and MobileNetV2 network

The proposed fall detection approach is aimed at building a support system for the elders. In this work, a method based on human pose estimation and lightweight neural network is used to detect falls. First, the OpenPose is used to extract human keypoints and label them in the images. After that, the modified MobileNetV2 network is used to detect falls by integrating both human keypoint information and pose information in the original images. The above operation can use the original image information to correct the deviation in the keypoint labeling process. Through experiments, the accuracy of the proposed method is 98.6% and 99.75% on the UR and Le2i datasets, which is higher than the listed comparison methods.