Reviewerly/exHarmony huggingface dataset:
Local Visual and Global Deep Features Based Blind Stitched Panoramic Image Quality Evaluation Using Ensemble Learning

Image stitching technology aims to generate a Stitched Panoramic Image (SPI) by stitching multiple narrow-view images containing over-lapping regions. However, heterogeneous artifacts may be introduced during the process of image stitching, which affects the visual perception. To automatically and accurately evaluate the perceptual quality of SPIs, a novel local visual and global deep features based blind SPI quality evaluation method is proposed in this paper. To be specific, with the consideration that image stitching mainly destroys structure, texture and color information, we first design a color structure-texture joint dictionary trained from the constructed stitched-specific image patches dataset. Given an input SPI, its local visual and global deep features are extracted to characterize the stitched-specific distortions. For local visual features, the trained dictionary is employed to capture structure, texture and color distortions by sparse features extraction. Then, considering that sparse features are insensitive to weak structural distortions, weighted local binary pattern features are extracted to measure various weak distortions. For global perceptual features, deep features are extracted via a pre-trained convolutional neural network model to represent the high-level semantics. Finally, considering the diversity of the extracted features, an ensemble learning strategy is adopted to promote the generalization performance and prediction accuracy of the proposed model. Experimental results show that, compared with the conventional 2D and SPI quality measurement methods, the proposed method can measure the stitched-specific distortions more accurately, and is more coincident with subjective ratings.
Consensus Adversarial Defense Method Based on Augmented Examples

Deep learning has been used in many computer-vision-based industrial Internet of Things applications. However, deep neural networks are vulnerable to adversarial examples that have been crafted specifically to fool a system while being imperceptible to humans. In this article, we propose a consensus defense (Cons-Def) method to defend against adversarial attacks. Cons-Def implements classification and detection based on the consensus of the classifications of the augmented examples, which are generated based on an individually implemented intensity exchange on the red, green, and blue components of the input image. We train a CNN using augmented examples together with their original examples. For the test image to be assigned to a specific class, the class occurrence of the classifications on its augmented images should be the maximum and reach a defined threshold. Otherwise, it is detected as an adversarial example. The comparison experiments are implemented on MNIST, CIFAR-10, and ImageNet. The average defense success rate (DSR) against white-box attacks on the test sets of the three datasets is 80.3%. The average DSR against black-box attacks on CIFAR-10 is 91.4%. The average classification accuracies of Cons-Def on benign examples of the three datasets are 98.0%, 78.3%, and 66.1%. The experimental results show that Cons-Def shows a high classification performance on benign examples and is robust against white-box and black-box adversarial attacks.
Dive: End-to-End Speech Diarization Via Iterative Speaker Embedding

We introduce DIVE, an end-to-end speaker diarization sys-tem. DIVE presents the diarization task as an iterative pro-cess: it repeatedly builds a representation for each speaker before predicting their voice activity conditioned on the ex-tracted representations. This strategy intrinsically resolves the speaker ordering ambiguity without requiring the classi-cal permutation invariant training loss. In contrast with prior work, our model does not rely on pretrained speaker represen-tations and jointly optimizes all parameters of the system with a multi-speaker voice activity loss. DIVE does not require the training speaker identities and allows efficient window-based training. Importantly, our loss explicitly excludes unreliable speaker turn boundaries from training, which is adapted to the standard collar-based Diarization Error Rate (DER) eval-uation. Overall, these contributions yield a system redefining the state-of-the-art on the CALLHOME benchmark, with 6.7% DER compared to 7.8% for the best alternative.
Cognitive Modeling of Complex Systems: State and Prospects

The author’s methodology of cognitive modeling of complex systems, which is intended for the study of socio-economic, ecological, cyber-physical and other complex systems, is briefly presented. Its features are considered, the basic sources and the current state of research in this direction are indicated. The purpose of the article was to outline the possibilities, prospects for further development and practical application of the apparatus for cognitive modeling of complex systems. Cognitive modeling of complex systems is multi-stage, cyclical, consists of a set of methods for solving problems: methods for analyzing the properties of models (structural and dynamic), methods for modeling scenarios and predicting possible future development of the system, decision support, and includes a software system for Cognitive Modeling Complex Systems (CMCS). The methodology of cognitive modeling of complex systems belongs to the direction of intelligent systems in the group of cognitive sciences; it can serve to support managerial decision-making. The possibilities of cognitive modeling of complex systems are illustrated by an example of several stages in the study of the regional socio-economic mechanism.
A Survey of Machine Learning and Deep Learning Based DGA Detection Techniques

Botnets are the most commonly used mechanisms for current cyberattacks such as DDoS, ransomware, email spamming, phishing data, etc. Botnets deploy the Domain Generation Algorithm (DGA) to conceal domain names of Command & Control (C&C) servers by generating several fake domain names. A sophisticated DGA can circumvent the traditional detection methods and successfully communicate with the C&C. Several detection methods like DNS sinkhole, DNS filtering and DNS logs analysis have been intensively studied to neutralize DGA. However, these methods have a high noise rate and require a massive amount of computational resources. To tackle this issue, several researchers leveraged Machine learning (ML) and Deep Learning (DL) algorithms to develop lightweight and cost-effective detection methods. The purpose of this paper is to investigate and evaluate the DGA detection methods based on ML/DL published in the last three years. After analyzing the relevant literature strengths and limitations, we conclude that low detection speed, encrypted DNS sensitivity, data imbalance sensitivity, and low detection accuracy with variant or unknown DGA are most likely the current research trends and opportunities. As far as we know, this survey is the first of its kind to discuss DGA detection techniques based on ML/DL in-depth, as well as analysis of their limitations and future trends.
Spatiotemporal Calibration of Camera-LiDAR Using Nonlinear Angular Constraints on Multiplanar Target

The combinations of data captured from diverse sensors can yield reliable geometric and semantic information. However, calibration process must be preceded to make use of combined data. Calibration methods are classified by the adjusted parameters. Spatial calibration, which merges data to a single spatial reference frame, adjusts spatial parameters. Moreover, temporal offset parameter must be adjusted for sensors to acquire data under dynamic conditions, which is known as temporal calibration. In this study, a novel constraint adjustment method was proposed for spatiotemporal calibration, which combines mobile light detection and ranging (LiDAR) sensor with visual camera. It estimates seven parameters–three lever arm parameters, three bore-sight parameters, and temporal offset parameter–simultaneously. A nearly orthogonal multiplanar chessboard target was built for nonlinear angular constraints. B-spline interpolation method was employed to interpolate the plane parameters obtained from the camera and facilitate estimation of the temporal offset. Additionally, the calibration results were evaluated using correlation values and standard errors of the estimated parameters. The sequential quadratic programming method, which yields robust results for nonlinear optimization problem, was used to optimize a point-to-plane distance model subjected to nonlinear angular constraints. Assorted experiments with different sensor motions were conducted to evaluate the effects of the nonlinear angular constraints. Compared to the results of nonconstrained calibration methods, the results of the proposed method indicate lower correlation values between the calibration parameters, reduced standard errors, and fewer iterations in the optimization process, which validates the superiority of the proposed calibration method.
VR Viewport Pose Model for Quantifying and Exploiting Frame Correlations

The importance of the dynamics of the viewport pose, i.e., the location and the orientation of users’ points of view, for virtual reality (VR) experiences calls for the development of VR viewport pose models. In this paper, informed by our experimental measurements of viewport trajectories across 3 different types of VR interfaces, we first develop a statistical model of viewport poses in VR environments. Based on the developed model, we examine the correlations between pixels in VR frames that correspond to different viewport poses, and obtain an analytical expression for the visibility similarity (ViS) of the pixels across different VR frames. We then propose a lightweight ViS-based ALG-ViS algorithm that adaptively splits VR frames into the background and the foreground, reusing the background across different frames. Our implementation of ALG-ViS in two Oculus Quest 2 rendering systems demonstrates ALG-ViS running in real time, supporting the full VR frame rate, and outperforming baselines on measures of frame quality and bandwidth consumption.
Fold2Vec: Towards a Statement-Based Representation of Code for Code Comprehension

We introduce a novel approach to source code representation to be used in combination with neural networks. Such a representation is designed to permit the production of a continuous vector for each code statement. In particular, we present how the representation is produced in the case of Java source code. We test our representation for three tasks: code summarization , statement separation , and code search . We compare with the state-of-the-art non-autoregressive and end-to-end models for these tasks. We conclude that all tasks benefit from the proposed representation to boost their performance in terms of F1-score, accuracy, and mean reciprocal rank, respectively. Moreover, we show how models trained on code summarization and models trained on statement separation can be combined to address methods with tangled responsibilities, meaning that these models can be used to detect code misconduct.
REL-SAGAN: Relative Generation Adversarial Network Integrated With Attention Mechanism for Scene Data Augmentation of Remote Sensing

Deep learning shows a strong ability in target detection, scene classification, and change detection of remote sensing.However, the training process requires a large number of samples, and the production of most high-quality training samples requires a lot of time and human resources.With the resolution of remote sensing image gradually improved, and the scene information becoming more and more complicated, the sample augmentation methods at this stage have shown obvious defects, such as the loss of sample key information and the lack of diversity in augmented data sets.In order to solve these problems, this article proposes a new augmentation method for remote sensing scene data named Rel-SAGAN based on generative adversarial networks (GAN).First, combined with self-attention mechanism to improve the large-scale feature learning ability and reduce the calculation parameters of GAN.Second, using relativity adversarial loss function to improve the structural stability of GAN.Furthermore, increasing convolution kernel and deeper structural of GAN to improve the ability of global feature extraction and saving the training time.Finally, taking NWPU remote sensing image dataset as experimental data, the effectiveness of the method proposed in this article is verified through ResNet-18.The experimental results show that it can generate more diverse high-resolution remote sensing natural scene images, the overall classification accuracy of augmented training dataset of remote sensing used high-quality generated images selected based on inception score and Frechet inception distance evaluations is improved by more than 3% and the classification accuracy is generally higher than the traditional data augmentation methods.
Early Prediction of Mortality in Critical Care Setting in Sepsis Patients Using Structured Features and Unstructured Clinical Notes

Sepsis is an important cause of mortality, especially in intensive care unit(ICU) patients. Developing novel methods to identify early mortality is critical for improving survival outcomes in sepsis patients. Using the MIMIC-III database, we integrated demographic data, physiological measurements and clinical notes. We built and applied several machine learning models to predict the risk of hospital mortality and 30-day mortality in sepsis patients. From the clinical notes, we generated clinically meaningful word representations and embeddings. Supervised learning classifiers and a deep learning architecture were used to construct prediction models. The configurations that utilized both structured and unstructured clinical features yielded competitive F-measure of 0.512. Our results showed that the approaches integrating both structured and unstructured clinical features can be effectively applied to assist clinicians in identifying the risk of mortality in sepsis patients upon admission to the ICU.
BM3D adaptive TV filtering-based convolutional neural network for ESPI image denoising

Image denoising is a fundamental part of image processing. The real electronic speckle pattern interferometry (ESPI) contains a large amount of speckle noise, which affects the image quality and adversely affects subsequent studies. In this paper, a method based on an improved denoising convolutional neural network (CNN) has been proposed, with the goal of reducing noise while maintaining accurate information. The block matching 3D-based adaptive TV denoising CNN can protect the valid information while preventing the information of the original image itself from being corrupted. A two-channel model is used to improve the noise reduction effect of real images. The proposed method is compared with the conventional denoising algorithms and the deep-learning denoising algorithms. Experimental results show that the proposed method can maintain accuracy, integrity, and stability while preserving the details, texture, and edge information of the stripe pattern.
Hierarchical RNNs-Based transformers MADDPG for mixed cooperative-competitive environments

Structural models based on Attention can not only record the relationships between features’ position, but also can measure the importance of different features based on their weights. By establishing dynamically weighted parameters for choosing relevant and irrelevant features, the key information can be strengthened, and the irrelevant information can be weakened. Therefore, the efficiency of Deep Learning algorithms can be significantly elevated and improved. Although Transformer have been performed very well in many fields including Reinforcement Learning (RL). We tried to integrate Transformers into RL, however there are some challenge in this task. Especially, MARL (known as Multi-Agent Reinforcement Learning), which can be recognized as a set of independent agents trying to adapt and learn through their way to reach the goal. In order to emphasize the relationship between each MDP decision in a certain time period, we applied the hierarchical coding method and validated the effectiveness of this method. This paper proposed a Hierarchical Transformer MADDPG based on recurrent neural network(RNN) which we call it Hierarchical RNNs-Based Transformers MADDPG(HRTMADDPG). It consists of a lower level encoder based on RNNs that encodes multiple step sizes in each time sequence, and it also consists of an upper sequence level encoder based on Transformer for learning the correlations between multiple sequences. Then we can capture the causal relationship between sub-time sequences and make HRTMADDPG more efficient.
What are the Characteristics of Highly-Selected Packages? A Case Study on the NPM Ecosystem

Identifying high-quality and well-maintained packages from a large pool of packages to depend on is a basic and important problem, as it is beneficial for various applications, such as package recommendation and package search. However, no systematic and comprehensive work focuses on addressing this problem except in online discussions or informal literature and interviews. To fill this gap, in this paper, we conducted a mixed qualitative and quantitative analysis to understand how developers identify and select relevant open source packages. In particular, we started by surveying 118 JavaScript developers from the npm ecosystem to qualitatively understand the factors that make a package to be highly-selected within the npm ecosystem. Then, we conducted an experiment to quantitatively validate the developers’ perception of the factors that make a highly-selected package. In this analysis, we collected and mined historical data from 2,527 packages divided into highly-selected and not highly-selected packages. Next, we used regression analysis to quantitatively investigate which of the studied factors are the most important. Our regression analysis complements our survey results about highly-selected packages. In particular, the results showed that highly-selected packages tend to be correlated by the number of downloads, stars, and how large the package’s readme file is.
DEPICTER: A Design-Principle Guided and Heuristic-Rule Constrained Software Refactoring Approach

Software refactoring is one of the most significant practices in software maintenance as the quality of software design tends to deteriorate during software evolution. But, refactoring software is a very challenging task as it requires a holistic view of the entire software system. To this end, recent studies introduced search-based algorithms to facilitate software refactoring. However, they still have the following major limitations: 1) the searched solutions may violate the design principles as their fitness functions do not directly reflect the degree of software’s compliance with design principles; 2) most approaches start the searching process from a completely random initial population, which may lead to unoptimal solutions. In this article, we aim to develop effective search-based refactoring approach to recommend better refactoring activities for developers which can improve the degree of software’s compliance with design principles as well as the software design quality. We propose <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DEPICTER</monospace> , a design-principle guided and heuristic-rule constrained software refactoring recommendation approach. In particular, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DEPICTER</monospace> uses non-dominated sorting genetic algorithm (NSGA)-II genetic algorithm and employs design-principle metrics as fitness functions. Besides, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DEPICTER</monospace> leverages heuristic rules to improve the quality of initial population for subsequent generic evolution. Our evaluations, based on four widely used systems, show that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DEPICTER</monospace> is effective for guiding the development of better refactoring models in practice.
Anomalous Witnesses and Registrations Detection in the Italian Justice System Based on Big Data and Machine Learning Techniques

It is estimated that in 2020 the amount of data produced was about 44 zettabytes for a per capita daily production of about 16 gigabytes. These numbers make us think about how much knowledge is possible to extract from the data produced every day by every single inhabitant of the earth. Supported by the growing diffusion of frameworks and tools for the automatic analysis of Big Data, Machine Learning and Deep Learning, we can try to extract knowledge from all this information and use it to offer a greater definition to human knowledge. In this paper we present two techniques that exploit the knowledge provided by data analysis to identify anomalies in the Italian judicial system, in particular in the civil process. The first anomaly concerns the presence of “serial witnesses”, people who lend themselves to provide testimony of the facts occurred in different trial proceedings where places dates and events overlap highlighting a false testimony. The second anomaly relates to “multiple entries” by lawyers with the aim of being able to happen upon a judge “favorable” to the case. The two anomalies presented, but the possibilities are endless, are identified through the definition of Big Data pipelines for data aggregation, information extraction and data analysis.
Solar radiation forecasting using random forest

Power generation from renewable energy sources is key to a clean energy future and solar energy is world's fastest growing energy sector. Solar energy is renewable, CO2-free and with low operational cost. There are several advantages of using solar energy; however, it does have a few drawbacks such as hefty initial cost, high-priced storage, weather dependency, sizable space requirement, etc. As such, it is critical to predict solar radiation (SR) in an accurate and efficient way to install solar plants in optimal locations. Factors like global horizontal irradiance (GHI), temperature, humidity, cloud cover, wind speed, etc. make SR highly intermittent and variable. Accurate forecasting of SR is vital to finalize installed capacity of the proposed power plant but is extremely challenging due to unpredictability of sunlight. Even the world's best organizations such as the International Energy Agency (IEA) finds it difficult to accurately predict SR. In this work, we analyze global and diffuse SR data gathered from India Meteorological Department (IMD) Pune. This data is first analyzed for features, dominant features are identified, and several machine learning algorithms are employed. Random forest method provides the best results on this data based on several quantitative measures.