Reviewerly/exHarmony huggingface dataset:
Multiclass Object Detection in UAV Images Based on Rotation Region Network

The object detection in UAV application is a challenging task due to the diversity of target scales, variation of views, and complex backgrounds. To solve several challenges, including dense objects, objects with arbitrary orientation, and diversity of aspect ratios, this article proposes an end-to-end object detection method based on the convolutional neural network. In this article, the feature extraction performance is enhanced by utilizing a deep residual neural network. Multiscale feature maps are obtained through fusion with different convolutional layers, thus combining the high-level semantic information and low-level detail information. A rotation region proposal network is adopted to generate rotated regions, which makes the bounding box sensitive to dense objects in aerial images. Meanwhile, the RoIAlign is used and a convolution layer is appended in the classification stage, and focal loss is used in the classification stage. The proposed method focuses on arbitrary-oriented and dense objects in UAV images. After a comprehensive evaluation with several state-of-the-art object detection algorithms, the proposed method is proved to be effective to detect multiclass artificial objects in aerial images. Extensive experiments are conducted on the DOTA, VEDAI, and the VisDrone UAV image datasets, which demonstrate that the proposed method can obtain discriminative features through the improved multiscale feature extraction and the rotating region network. The results on the above datasets show that our method obtains gains in mean average precision compared with several state-of-the-art methods.
Responsible vulnerability disclosure in cryptocurrencies

Software weaknesses in cryptocurrencies create unique challenges in responsible revelations.
Noise Density Range Sensitive Mean-Median Filter for Impulse Noise Removal

A new noise density range sensitive algorithm for the restoration of images that are corrupted by impulse noise is proposed. The proposed algorithm replaces the noisy pixel by mean, median or pre-processed values based on noise density of the image. The proposed filter uses a unique approach for recovering images corrupted with very high noise densities (over 85%). It also provides significantly better image quality for different noise densities (10–90%). Simulation results show that the proposed filter outperforms in comparison with the other nonlinear filters. At very high noise densities, the proposed filter provides better visual representation with 6.5% average improvement in peak signal-to-noise ratio value when compared to state-of-the-art filters.
An Improved Air-Light Estimation Scheme for Single Haze Images Using Color Constancy Prior

Hazy environment attenuates the scene radiance and causes difficulty in distinguishing the color and texture of the scene. A crucial step in dehazing is the recovery of the global air-light vector. Traditional methods usually interpret the RGB value of the brightest region in haze images as the air-light. In this letter, a new prior called `color constancy prior' has been proposed to improve the robustness of air-light estimation when varicolored illumination exists. The prior utilizes the statistical observation that distant scenery objects become the most haze-opaque due to the pixel escalation towards the higher intensity side. The comparative evaluation on a variety of haze images manifests that the proposed prior perform better than existing air-light recovery methods and can be used for subsequent dehazing applications.
3D Orientation-Preserving Variational Models for Accurate Image Registration

The Beltrami coefficient from complex analysis has recently been found to provide a robust constraint for obtaining orientation-preserving and diffeomorphic transformations for registration of planar images.There exists no such a concept of Beltrami coefficient in three or higher dimensions, although a generalized theory of quasi-conformal maps in high dimensions exists.In this paper, we first propose a new algebraic measure in three dimensions (3D) that mimics the Beltrami concept in two dimensions (2D) and then propose a corresponding registration model based on it.We then establish the existence of solutions for the proposed model and further propose a converging generalized Gauss-Newton iterative method to solve the resulting nonlinear optimization problem.In addition, we also provide another two possible regularizers in 3D.Numerical experiments show that the new model can produce more accurate orientation-preserving transformations than competing state-of-the-art registration models.
Formalizing Tag-Based Metadata With the Brick Ontology

Current efforts establishing semantic metadata standards for the built environment span academia, industry and standards bodies. For these standards to be effective, they must be clearly defined and easily extensible, encourage consistency in their usage, and integrate cleanly with existing industrial standards, such as BACnet. There is a natural tension between informal tag-based systems that rely upon idiom and convention for meaning, and formal ontologies amenable to automated tooling. We present a qualitative analysis of Project Haystack, a popular tagging system for building metadata, and identify a family of inherent interpretability and consistency issues in the tagging model that stem from its lack of a formal definition. To address these issues, we present the design and implementation of the Brick+ ontology, a drop-in replacement for Brick with clear formal semantics that enables the inference of a valid Brick model from an informal Haystack model, and demonstrate this inference across five Haystack models.
AGNES-SMOTE: An Oversampling Algorithm Based on Hierarchical Clustering and Improved SMOTE

Aiming at low classification accuracy of imbalanced datasets, an oversampling algorithm—AGNES-SMOTE (Agglomerative Nesting-Synthetic Minority Oversampling Technique) based on hierarchical clustering and improved SMOTE—is proposed. Its key procedures include hierarchically cluster majority samples and minority samples, respectively; divide minority subclusters on the basis of the obtained majority subclusters; select “seed sample” based on the sampling weight and probability distribution of minority subcluster; and restrict the generation of new samples in a certain area by centroid method in the sampling process. The combination of AGNES-SMOTE and SVM (Support Vector Machine) is presented to deal with imbalanced datasets classification. Experiments on UCI datasets are conducted to compare the performance of different algorithms mentioned in the literature. Experimental results indicate AGNES-SMOTE excels in synthesizing new samples and improves SVM classification performance on imbalanced datasets.
Benchmark Dataset Selection of Web Services Technologies: A Factor Analysis

Web services have emerged as an accessible technology with the standard ’Extensible Mark Up’ (XML) language, which is known as ’Web Services Description Language’ WSDL. Web services have become a promising technology to promote the interrelationship between service providers and users. Web services users’ trust is measured by quality metrics. Web service quality metrics vary in many benchmark datasets used in the existing studies. The selection of a benchmark dataset is problematic to classify and retest web services. This paper proposes a method to rank web services quality metrics for the selection of benchmark web services datasets. To measure the diversity in quality metrics, factor analysis with Varimax rotation and scree plot is a well-established method. We use factor analysis to determine percentage variance among principal factors of four benchmark datasets. Our results showed that the two-factor solution explained 94.501, 76.524, and 45.009% variances in datasets A, B, and D, respectively. A three-factor solution explained 85.085% variance in dataset C. Reliability, and response time quality metrics were predicted as the most dominating quality metrics that contributed to explain the percentage variance in four datasets. Our proposed web metric ranking (WMR) method resulted in reliability as the top-most web metric with (57.62%) score and latency web metric at the bottom-most with (3.60%) score. The proposed WMR method showed a high (96.17%) ranking precision. Obtained results verified that factor solutions after reducing the dimensions could be generalized and used in the quality improvement of web services. In future works, the authors plan to focus on a dataset with dominating quality metrics to perform regression testing of web services.
Evaluating Surprise Adequacy for Question Answering

With the wide and rapid adoption of Deep Neural Networks (DNNs) in various domains, an urgent need to validate their behaviour has risen, resulting in various test adequacy metrics for DNNs. One of the metrics, Surprise Adequacy (SA), aims to measure how surprising a new input is based on the similarity to the data used for training. While SA has been evaluated to be effective for image classifiers based on Convolutional Neural Networks (CNNs), it has not been studied for the Natural Language Processing (NLP) domain. This paper applies SA to NLP, in particular to the question answering task: the aim is to investigate whether SA correlates well with the correctness of answers. An empirical evaluation using the widely used Stanford Question Answering Dataset (SQuAD) shows that SA can work well as a test adequacy metric for the question answering task.
Counterfactual learning for recommender system

Most commercial industrial recommender systems have built their closed feedback loops. Though it is helpful in item recommendation and model training, the closed feedback loop may lead to the so-called bias problems, including the position bias, selection bias and popularity bias. The recommendation models trained with biased may hurt the user experiences by recommending homogenous items. How to control the biases in the closed feedback loop has become one of major challenges in modern recommender systems. This talk discusses the counterfactual learning technologies for tackling the bias problem in recommendation.
Memory-Latency-Accuracy Trade-Offs for Continual Learning on a RISC-V Extreme-Edge Node

AI-powered edge devices currently lack the ability to adapt their embedded inference models to the ever-changing envi ronment. To tackle this issue, Continual Learning (CL) strategies aim at incrementally improving the decision capabilities based on newly acquired data. In this work, after quantifying memory and computational requirements of CL algorithms, we define a novel HW/SW extreme-edge platform featuring a low power RISC-V octa-core cluster tailored for on-demand incremental learning over locally sensed data. The presented multi-core HW/SW architecture achieves a peak performance of 2.21 and 1.70 MAC/cycle, respectively, when running forward and backward steps of the gradient descent. We report the trade-off between memory footprint, latency, and accuracy for learning a new class with Latent Replay CL when targeting an image classification task on the CORe50 dataset. For a CL setting that retrains all the layers, taking 5h to learn a new class and achieving up to 77.3% of precision, a more efficient solution retrains only part of the network, reaching an accuracy of 72.5% with a memory requirement of 300 MB and a computation latency of 1.5 hours. On the other side, retraining only the last layer results in the fastest (867 ms) and less memory hungry (20 MB) solution but scoring 58% on the CORe50 dataset. Thanks to the parallelism of the low-power cluster engine, our HW/SW platform results 25× faster than typical MCU device, on which CL is still impractical, and demonstrates an 11× gain in terms of energy consumption with respect to mobile-class solutions.
Role-Oriented Graph Auto-encoder Guided by Structural Information

Roles in a complex network usually represent the local connectivity patterns of nodes, which reflect the functions or behaviors of corresponding entities. Role discovery has great meaning for understanding the formation and evolution of networks. While the importance of role discovery in networks has been realized gradually, a variety of approaches of role-oriented network representation learning are proposed. Almost all the existing approaches are dependent on manual high-order structural properties which are always fragmentary. They suffer from unstable performances and poor generalization ability, because their hand-craft structural features sometimes miss the characteristics of different networks. In addition, graph neural networks (GNNs) have great potential to automatically capture structural properties, but it is hard to be given the rein to for the difficulty of designing role-oriented unsupervised loss. To overcome these challenges, we provide an idea that leverage low-dimensional extracted structural features as guidance to train graph neural networks. Based on the idea, we proposed GAS, a novel graph auto-encoder guided by structural information, to learn role-oriented representations for nodes. Results of extensive experiments show that GAS has better performance than other state-of-the-art approaches.
Virtual sensor forensics

Sensorial escalation, heterogeneity, pervasiveness, and increased connectivity have created a haven for cyber-criminals, where potential attacks are being exacerbated on a daily basis. In essence, most automated systems are not built to be able to dynamically detect potential security incidents or to provide digital forensic evidence that could be used for forensic hypothesis formation. Consequently, several ISO standards have also been proposed, however, they present a very generic view on investigation. Notably, in the recent past, sensorial proliferation has seen the increased usage of virtual sensors that have extended the functionalities and utilization of physical sensors-where sensing can be done using an abstraction/logical layers, which allows streamed sensor data to be collected dynamically in a proactive approach. The usage of virtual sensors has proved to be threat driven and the vulnerable threat landscape keeps being exploited. In this position paper, the author proposes a Virtual Sensor Forensic (VSF) architecture that may accurately give insights on event response processes, as a reactive technique. The proposed VSF architecture has dimensions that are application-specific, which shows that during the collection of streamed sensor data, post-event response may be of importance towards the detection of potential security incidents. The effort that has been put across in this position has presented the agenda of virtual sensor forensics based on a novel approach, that can be used as a post-event mechanism with a degree of acceptability.
The Good, The Bad, and The Ugly: Neural Networks Straight From JPEG

Over the past decade, convolutional neural networks (CNNs) have achieved state-of-the-art performance in many computer vision tasks. They can learn robust representations of image data by processing RGB pixels. Since image data are often stored in a compressed format, from which JPEG is the most widespread, a preliminary decoding process is demanded. Recently, the design of CNNs for processing JPEG compressed data has gained attention from the research community. They process DCT coefficients instead of RGB pixels, saving computation for decoding JPEG images, however, at the cost of increasing the computational complexity of the network. In this paper, we examine how spatial resolution and JPEG quality impacts on the performance of a state-of-the-art CNN designed to operate directly on the JPEG compressed domain. To alleviate its computational complexity, we propose a Frequency Band Selection (FBS) technique to select the most relevant DCT coefficients before feeding them to the network. Experiments were conducted on a subset of the ImageNet dataset considering both fine- and coarse-grained image classification tasks. Results show that such networks are resilient to JPEG quality but are susceptible to spatial resolution. Also, our FBS can reduce the computational complexity of the network while retaining a similar accuracy.
A Quantum Image Encryption Method Based on DNACNot

The image is one of the most important means to store information, and is widely used in every aspect of life. However, the characteristics of images enable them to be easily stolen, tampered with, and copied. Researchers have proposed many encryption methods, with most applying only to traditional digital images. There are few encryption methods for quantum images, and quantum image-oriented encryption technology has begun to attract researchers' attention. This study proposes a quantum image encryption method based on DNA Controlled-Not (DNACNot). Based on the quantum image information, encryption parameters are obtained and transferred as part of a chaotic initial key. Two natural DNA sequences are amplified to obtain the DNA coordinate sequence, and a modified chaos game representation is used to modify the DNA coordinate sequence, which is then used to correct the sequence generated by chaos. The resulting sequence is converted to an integer sequence to obtain DNACNot. A controlled-not operation is performed between DNACNot and the quantum image is scrambled by bit-plane to obtain the encrypted image. The encryption method has high security, a good encryption effect, and a large key space. The method can effectively resist exhaustive, statistical, and differential attacks. The algorithm is easy to implement at a low cost. The encryption time of our proposed method is satisfactory, and the method is suitable for real-time encryption. Moreover, the encryption results can be transmitted over the internet and stored in the cloud.
Anytime Integrated Task and Motion Policies for Stochastic Environments

In order to solve complex, long-horizon tasks, intelligent robots need to carry out high-level, abstract planning and reasoning in conjunction with motion planning. However, abstract models are typically lossy and plans or policies computed using them can be unexecutable. These problems are exacerbated in stochastic situations where the robot needs to reason about, and plan for multiple contingencies. We present a new approach for integrated task and motion planning in stochastic settings. In contrast to prior work in this direction, we show that our approach can effectively compute integrated task and motion policies whose branching structures encoding agent behaviors handling multiple execution-time contingencies. We prove that our algorithm is probabilistically complete and can compute feasible solution policies in an anytime fashion so that the probability of encountering an unresolved contingency decreases over time. Empirical results on a set of challenging problems show the utility and scope of our methods.
Hierarchical Coverage Path Planning in Complex 3D Environments

State-of-the-art coverage planning methods perform well in simple environments but take an ineffectively long time to converge to an optimal solution in complex three-dimensional (3D) environments. As more structures are present in the same volume of workspace, these methods slow down as they spend more time searching for all of the nooks and crannies concealed in three-dimensional spaces. This work presents a method for coverage planning that employs a multi-resolution hierarchical framework to solve the problem at two different levels, producing much higher efficiency than the state-of-the-art. First, a high-level algorithm separates the environment into multiple subspaces at different resolutions and computes an order of the subspaces for traversal. Second, a low-level sampling-based algorithm solves for paths within the subspaces for detailed coverage. In experiments, we evaluate our method using real-world datasets from complex three-dimensional scenes. Our method finds paths that are constantly shorter and converges at least ten times faster than the state-of-the-art. Further, we show results of a physical experiment where a lightweight UAV follows the paths to realize the coverage.
Research Progress of Zero-Shot Learning Beyond Computer Vision

Traditional machine learning techniques, including deep learning, most assume that the classes of testing samples belong to the subset of training samples. However, there are many scenarios that conflict with this assumption in the real world, that is, the classes of testing samples have never been seen in model training. To improve the generalization ability of the model in these cases, zero-shot learning (ZSL) was proposed, which can mine the mapping relationship between the features and the labels of the seen class samples and then transfer it to the prediction of unseen classes. Most of the existing ZSL algorithms or applications are concerned with computer vision problems. In fact, the above difficulties and the demand for ZSL also exist in other fields, but there is currently a lack of relevant research progress review. To make up for this gap, this paper reviews the latest research progress of ZSL beyond computer vision, introduces the general concepts of ZSL, classifies the mainstream models, and refines three issues worthy of study. This study is expected to provide ZSL-based solution guidance for researchers and engineers beyond the field of computer vision.
Satellite-Aided Consensus Protocol for Scalable Blockchains

In this work, we propose a satellite-aided permissionless consensus protocol for scalable space-terrestrial blockchains. We design its working principle and workflow by taking full advantage of satellites for extensive coverage and ubiquitous connectivity. Based on the proposed protocol, we demonstrate how such a space-terrestrial blockchain grows and evolves through several typical cases in the presence of adversarial nodes, user misbehavior, and transmission outage. Taking proof of work (PoW) as a benchmark, we assess the system security by considering both adversarial miners and possible colluding satellites. Then, we analyze the maximum blockchain throughput under network capacity limits and evaluate the impact of information propagation delay via a Markov model. Simulation results support that the proposed satellite-aided consensus protocol achieves higher throughput and exhibits greater scalability than PoW.
Identity-Preserving Realistic Talking Face Generation

Speech-driven facial animation is useful for a variety of applications such as telepresence, chatbots, etc. The necessary attributes of having a realistic face animation are 1) audiovisual synchronization (2) identity preservation of the target individual (3) plausible mouth movements (4) presence of natural eye blinks. The existing methods mostly address the audiovisual lip synchronization, and few recent works have addressed synthesis of natural eye blinks for overall video realism. In this paper, we propose a method for identity-preserving realistic facial animation from speech. We first generate person-independent facial landmarks from audio using DeepSpeech features for invariance to different voices, accents, etc. To add realism, we impose eye blinks on facial landmarks using unsupervised learning and retarget the person-independent landmarks to person-specific landmarks to preserve the identity-related facial structure which helps in generation of plausible mouth shapes of the target identity. Finally, we use LSGAN to generate the facial texture from person-specific facial landmarks, using an attention mechanism that helps to preserve identity-related texture. An extensive comparison of our proposed method with the current state-of-the-art methods demonstrate a significant improvement in terms of lip synchronization accuracy, image reconstruction quality, sharpness, and identity-preservation. A user study also reveals improved realism of our animation results over the state-of-the-art methods. To the best of our knowledge, this is the first work in speech-driven 2D facial animation that simultaneously addresses all the above-mentioned attributes of a realistic speech driven face animation.