Reviewerly/exHarmony huggingface dataset:
Learning Dynamic Scale Awareness and Global Implicit Functions for Continuous-Scale Super-Resolution of Remote Sensing Images

The mainstream remote sensing image (RSI) super-resolution (SR) algorithms treat tasks with different scale factors independently, and a single model can only process a fixed integer scale factor. However, in practical applications, it is important to continuously super-resolve RSIs to multiple resolutions, as different resolutions present various levels of detail. Retraining the model for each scale factor consumes huge computational resources and storage space. Existing continuous-scale SR models employ static convolutions, and most are designed for natural scenes, ignoring dynamic feature extraction needs for different scale factors and the inherent properties of RSIs. In addition, efficiently obtaining the continuous representation of RSIs and avoiding the artifacts of RSI SR results is still a challenging problem. To address the above problems, we propose a scale-aware dynamic network (SADN) for RSI continuous-scale SR. First, we devise a scale-aware dynamic convolutional (SAD-Conv) layer to handle the strong randomness of the RSI textural distribution and achieve dynamic feature extraction according to scale factors. Second, we devise a continuous-scale upsampling module (CSUM) with the multi-bilinear global implicit function (MBGIF) for any-scale upsampling. The CSUM constructs multiple feature spaces with asymptotic resolutions to approximate the continuous representation of an image, and then, the MBGIF makes full use of multiresolution features to map arbitrary coordinates to spectral values. We evaluate our SADN using various benchmarks, and the experimental results show that the CSUM can efficiently achieve continuous-scale upsampling while maintaining excellent objective and visual performance. Moreover, our SADN uses fewer parameters and even outperforms the state-of-the-art fixed-scale SR methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hanlinwu/SADN</uri> .
Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation

In this article, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline that includes material acquisition, ray-tracing-based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor in real time. We conduct real experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to the real-world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.
Deep Image Registration With Depth-Aware Homography Estimation

Image registration is a basic task in computer vision, for its wide potential applications in image stitching, stereo vision, motion estimation, and etc. Most current methods achieve image registration by estimating a global homography matrix between candidate images with point-feature-based matching or direct prediction. However, as real-world 3D scenes have point-variant photograph distances (depth), a unified homography matrix is not sufficient to depict the specific pixel-wise relations between two images. Some researchers try to alleviate this problem by predicting multiple homography matrixes for different patches or segmentation areas in images; in this letter, we tackle this problem with further refinement, i.e. matching images with pixel-wise, depth-aware homography estimation. Firstly, we construct an efficient convolutional network, the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DPH-Net</i> , to predict the essential parameters causing image deviation, the rotation ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$R$</tex-math></inline-formula> ) and translation ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T$</tex-math></inline-formula> ) of cameras. Then, we feed-in an image depth map for the calculation of initial pixel-wise homography matrixes, which are refined with an online optimization scheme. Finally, with the estimated pixel-specific homography parameters, pixel correspondences between candidate images can be easily computed for registration. Compared with state-of-the-art image registration algorithms, the proposed <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DPH-Net</i> has the highest performance of 0.912 EPE and 0.977 SSIM, demonstrating the effectiveness of adding depth information and estimating pixel-wise homography into the image registration process.
Deep learning in biomedical informatics

With the massive influx of multimodal data in the last decade, the role of data analytics in health informatics has grown rapidly. Deep learning (DL) is defined as a technology based on artificial neural networks (ANNs), which have recently emerged as a powerful ML tool. The rapid increase in computing power, fast data storage, and parallelization, as well as predictive capabilities and the ability to generate automatically optimized advanced functions and semantic interpretation from input data, all contribute to the rapid adoption of this technology. This chapter introduces the latest developments in the use of deep learning in health informatics and makes an important analysis of the relative advantages, potential shortcomings, and future prospects of the technology. This chapter mainly focuses on the key applications of DL in the fields of computational biology, drug design, medical imaging, pervasive sensing, medical informatics, and public health. Artificial intelligence (AI) has become a trend in recent years, opening up an entirely new era of research in various fields. The demand for AI in healthcare has increased in both the academia and industry; therefore, the potential benefits of its applications have been proven. Previous studies have attempted to implement AI methods on medical images, electronic health records, molecular characteristics, and a variety of lifestyles. Researchers used data aggregated from multiple data sources to train models that mimic what clinicians do when they see patients and help in decision-making through results and interpretations. It included how to read clinical images, predict results, discover the connection between genotype and phenotype or phenotype and disease, analyze treatment responses, and track lesions or structural changes (e.g., hippocampal volume reduction). In addition, predictive research (e.g., disease or readmission predictions) and correlation and pattern recognition research have been extended to early warning systems with risk scores and overall pattern research and population care (such as predictive care for an entire population).
Simultaneous Destriping and Image Denoising Using a Nonparametric Model With the EM Algorithm

Digital images often suffer from the common problem of stripe noise due to the inconsistent bias of each column. The existence of the stripe poses much more difficulties on image denoising since it requires another n parameters, where n is the width of the image, to characterize the total interference of the observed image. This paper proposes a novel EM-based framework for simultaneous stripe estimation and image denoising. The great benefit of the proposed framework is that it splits the overall destriping and denoising problem into two independent sub-problems, i.e., calculating the conditional expectation of the true image given the observation and the estimated stripe from the last round of iteration, and estimating the column means of the residual image, such that a Maximum Likelihood Estimation (MLE) is guaranteed and it does not require any explicit parametric modeling of image priors. The calculation of the conditional expectation is the key, here we choose a modified Non-Local Means algorithm to calculate the conditional expectation because it has been proven to be a consistent estimator under some conditions. Besides, if we relax the consistency requirement, the conditional expectation could be interpreted as a general image denoiser. Therefore other state-of-the-art image denoising algorithms have the potentials to be incorporated into the proposed framework. Extensive experiments have demonstrated the superior performance of the proposed algorithm and provide some promising results that motivate future research on the EM-based destriping and denoising framework.
Two families of self-adjusting spectral hybrid DL conjugate gradient methods and applications in image denoising

The study in this paper introduces two families of self-adjusting spectral hybrid DL conjugate gradient methods to solve unconstrained optimization problems. The search directions in the proposed methods are improved by integrating the negative spectral gradient and the last search direction that has convex combination structure. One component of composite conjugate parameter in the search direction is a hybridization of DL-like conjugate parameter and another flexible conjugate parameter. It is proved that the search directions in the two families of newly proposed methods always automatically satisfy the sufficient descent property, which is independent of choosing the specific conjugate parameter and line search. With general nonconvex objective functions, the convergence results under the weak Wolfe line search are obtained. Numerical experiments verify the efficiency and applicability of our developed methods by comparison with some existing methods for solving unconstrained optimization and image restoration problems.
Robust TS-ANFIS MPC of an Autonomous Racing Electrical Vehicle Considering the Battery State of Charge

In this work, the trajectory tracking problem of an autonomous racing electrical vehicle is addressed. Accordingly, a two-layer control scheme is developed, such that stability, recursive feasibility, and constraint satisfaction are guaranteed. The outer layer includes a zonotopic tube-based predictive control to ensure trajectory tracking while minimizing energy consumption considering the state of charge of the vehicle's battery. Meanwhile, the inner layer combines a linear quadratic zonotopic controller with a zonotopic Kalman filter to reduce the effect of exogenous disturbances and modeling errors. Moreover, for control and estimation purposes, a data-driven Takagi–Sugeno (TS) model trained by an adaptive neuro-fuzzy inference system (ANFIS) is employed. To illustrate the performance of the proposed control scheme, a simulated 1/10 Scale RC car is used.
Service Delay Minimization for Federated Learning Over Mobile Devices

Federated learning (FL) over mobile devices has fostered numerous intriguing applications/services, many of which are delay-sensitive. In this paper, we propose a service delay efficient FL (SDEFL) scheme over mobile devices. Unlike traditional communication efficient FL, which regards wireless communications as the bottleneck, we find that under many situations, the local computing delay is comparable to the communication delay during the FL training process, given the development of high-speed wireless transmission techniques. Thus, the service delay in FL should be computing delay + communication delay over training rounds. To minimize the service delay of FL, simply reducing local computing/communication delay independently is not enough. The delay trade-off between local computing and wireless communications must be considered. Besides, we empirically study the impacts of local computing control and compression strategies (i.e., the number of local updates, weight quantization, and gradient quantization) on computing, communication and service delays. Based on those trade-off observation and empirical studies, we develop an optimization scheme to minimize the service delay of FL over heterogeneous devices. We establish testbeds and conduct extensive emulations/experiments to verify our theoretical analysis. The results show that SDEFL reduces notable service delay with a small accuracy drop compared to peer designs.
RepHAR: Decoupling Networks With Accuracy-Speed Tradeoff for Sensor-Based Human Activity Recognition

Sensor-based human activity recognition (HAR) can provide users with more convenience and security than machine vision-based methods. The ideal HAR application would achieve high-accuracy and low-latency performance with minimal hardware consumption. However, so far, the speed-accuracy tradeoff in mobile device-based HAR has rarely been systematically analyzed and discussed. In contrast to the static networks of existing deep HAR models, this article aims to design a resource-efficient dynamic network, called RepHAR, for low-cost hardware-constrained HAR tasks. This article first combines multibranch (MB) topologies and structured reparameterization techniques for HAR. Specifically, RepHAR decouples the model into a training time and inference time architecture using the idea of structural reparameterization. That is to say, the model has a MB topology at training time, but it no longer has the same model structure at inference time and is transformed into a plain convolutional neural network (CNN) structure. As a result, RepHAR achieves a trade-off between speed and accuracy by simultaneously obtaining the accuracy gain of a MB network and the high-speed inference of a plain CNN. On four publicly available datasets (including UCI-HAR, PAMAP2, UNIMIB-SHAR, and OPPORTUNITY), RepHAR can achieve 0.83%–2.18% accuracy improvement and 12%–44% parameter reduction, which demonstrates the effectiveness of the proposed method. Finally, on the embedded Raspberry Pi, the RepHAR runs 72% faster than the MB CNN model, demonstrating its usefulness and practicality. Our model and codes will be released soon.
A Robustness-Assured White-Box Watermark in Neural Networks

Recently, stealing highly-valuable and large-scale deep neural network (DNN) models becomes pervasive. The stolen models may be re-commercialized, e.g., deployed in embedded devices, released in model markets, utilized in competitions, etc, which infringes the Intellectual Property (IP) of the original owner. Detecting IP infringement of the stolen models is quite challenging, even with the white-box access to them in the above scenarios, since they may have experienced fine-tuning, pruning, functionality-equivalent adjustment to destruct any embedded watermark. Furthermore, the adversaries may also attempt to extract the embedded watermark or forge a similar watermark to falsely claim ownership. In this article, we propose a novel DNN watermarking solution, named <inline-formula><tex-math notation="LaTeX">$HufuNet$</tex-math></inline-formula> , to detect IP infringement of DNN models against the above mentioned attacks. Furthermore, HufuNet is the first one theoretically proved to guarantee robustness against fine-tuning attacks. We evaluate HufuNet rigorously on four benchmark datasets with five popular DNN models, including convolutional neural network (CNN) and recurrent neural network (RNN). The experiments and analysis demonstrate that HufuNet is highly robust against model fine-tuning/pruning, transfer learning, kernels cutoff/supplement, functionality-equivalent attacks and fraudulent ownership claims, thus highly promising to protect large-scale DNN models in the real world.
Incorporating Deep Background Prior Into Model-Based Method for Unsupervised Moving Vehicle Detection in Satellite Videos

Background reconstruction is a key step of moving object detection in satellite videos. Most existing model-based methods exploit low-rank prior to recover background, which have achieved good performance but suffered degradation under complex and dynamic scenes. In this paper, we introduce a deep background prior into model-based methods for moving vehicle detection in satellite videos. Our deep background prior is obtained by a background reconstruction network, which can learn to reconstruct background from consecutive frames. By applying our deep background prior into model-based methods, a closed-form solution can be obtained via alternating direction method of multipliers (ADMM) and then detection results can be acquired through iterative optimization. More importantly, our background reconstruction network can be trained in an unsupervised way by introducing specifically designed loss, thus relieving the dependence on large-scale labeled dataset. Extensive experimental results demonstrate the efficiency and effectiveness of the proposed method.
Cross-modality salient object detection network with universality and anti-interference

Cross-modality salient object detection (SOD) mainly includes RGB-D salient object detection and RGB-T salient object detection. Depth or thermal infrared information is used to compensate for the RGB information. Although cross-modality salient object detection has achieved excellent results, the current methods need to be improved in terms of universality and anti-interference. Therefore, we propose a cross-modality salient object detection network with universality and anti-interference. First, we offer a feature extraction strategy to enhance the features in the feature extraction stage. It can promote the mutual improvement of different modal information and avoid the influence of interference on the subsequent process. Then we use the graph mapping reasoning module (GMRM) to infer the high-level semantics to obtain valuable information. It enables our proposed method to accurately locate the objects in different scenes and interference to improve the universality and anti-interference of the method. Finally, we adopt a mutual guidance fusion module (MGFM), including a modality adaptive fusion module (MAFM) and across-level mutual guidance fusion module (ALMGFM), to carry out an efficient and reasonable fusion of multi-scale and multi-modality information. To verify the universality and anti-interference of our proposed method, we conduct experiments on many RGB-D/T SOD datasets and compare our method with the current state-of-the-art methods. Experimental results show that our method performs well in universality and anti-interference.
A Model for Weather Forecasting Based on Parallel Calculations

With the rapid accumulation of processed data in the modeling of a problem, it is necessary to apply the parallelization of the computational process, in order to be able to make decisions in real-time. In particular, parallel calculations are considered a tool for modeling complex phenomena and processes, as well as in solving cumbersome scientific and engineering problems: in the study of atmospheric phenomena, weather forecasting, environmental pollution, geology, seismology; applied and nuclear physics, electrical engineering, particle physics, materials science, nanotechnology, microelectronics; research in the fields of biotechnology, ecology, population biology, virology, genetics, pharmacology, and molecular chemistry. The paper constructs a model for weather forecasting using a regression equation based on parallel calculations. The parallelism of the algorithm for finding the coefficients of the curvilinear regression equation is performed. At the same time, the advantage of the possibility of calculating the coefficients of curves of any order is obtained. For concurrent calculations, we use the built-in concurrent library in Python, as well as properties such as multithreading. To verify the reliability of the obtained results on the basis of numerical experiments, a theoretical estimation of parallel acceleration according to Gustafson's law was performed. The dependence of the execution time of the proposed algorithm on the curve order for the different numbers of processes is investigated. A comparison of predicted and actual temperature data is made, and the coefficient of determination is calculated. In order to obtain real benefits from the implementation of parallel calculations, the current date was expanded and indicators of acceleration and efficiency of the proposed algorithm were found. The advantages obtained indicate the significance and value of the article.
Incorporating BERT With Probability-Aware Gate for Spoken Language Understanding

Spoken language understanding (SLU) is an essential part of a task-oriented dialogue system, which mainly includes intent detection and slot filling. Some existing approaches obtain enhanced semantic representation by establishing the correlation between two tasks. However, those methods show little improvement when applied to BERT, since BERT has learned rich semantic features. In this paper, we propose a BERT-based model with the probability-aware gate mechanism, called PAGM ( <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">P</u> robability <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> ware <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">G</u> ated <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> odel). PAGM aims to learn the correlation between intent and slot from the perspective of probability distribution, which explicitly utilizes intent information to guide slot filling. Besides, in order to efficiently incorporate BERT with the probability-aware gate, we design the stacked fine-tuning strategy. This approach introduces a mid-stage before target model training, which enables BERT to get better initialization for final training. Experiments show that PAGM achieves significant improvement on two benchmark datasets, and outperforms the previous state-of-the-art results.
Permissioned Blockchain-Based Secure and Privacy-Preserving Data Sharing Protocol

In this Internet of Things era, privacy preserving is one of the most vital barriers for personal data sharing. In this article, we present a secure and privacy-preserving data sharing protocol over the permissioned blockchains which require to certificate the users before they submit the transactions. We use the structure-preserving Groth signature to construct the anonymous credentials for satisfying the requirement of permissioned blockchains, and the anonymous credentials does not disclose the real identities of data owners. We prove that the anonymous credential in our protocol achieves the ideal functionality. For the secure access control and privacy protection of the data accessors, we propose an efficiently anonymous authentication scheme which utilizes the ElGamal commitment and the one-out-of-many proof to ensure a data accessor is authorized, but any unauthorized entities cannot learn the real identity of the data accessor, and even the data owner does not know who (although in the access control list) and when downloads his/her data. The blockchain platform is used to record the data storing, access control list, and the storage addresses, which helps to enhance the security level of the protocol. We implement our protocol over the ThinkPad, the RaspBerry Pi, the Huawei cloud, and the Hyperledger Fabric, and the experiments show the good performances.