Reviewerly/exHarmony huggingface dataset:
Temporal Modulation Network for Controllable Space-Time Video Super-Resolution

Space-time video super-resolution (STVSR) aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Recently, deformable convolution based methods have achieved promising STVSR performance, but they could only infer the intermediate frame pre-defined in the training stage. Besides, these methods undervalued the short-term motion cues among adjacent frames. In this paper, we propose a Temporal Modulation Network (TMNet) to interpolate arbitrary intermediate frame(s) with accurate high-resolution reconstruction. Specifically, we propose a Temporal Modulation Block (TMB) to modulate deformable convolution kernels for controllable feature interpolation. To well exploit the temporal information, we propose a Locally-temporal Feature Comparison (LFC) module, along with the Bi-directional Deformable ConvLSTM, to extract short-term and long-term motion cues in videos. Experiments on three benchmark datasets demonstrate that our TMNet outperforms previous STVSR methods. The code is available at https://github.com/CS-GangXu/TMNet.
An Efficient Deep Learning Model for Automatic Modulation Recognition Based on Parameter Estimation and Transformation

Automatic modulation recognition (AMR) is a promising technology for intelligent communication receivers to detect signal modulation schemes. Recently, the emerging deep learning (DL) research has facilitated high-performance DL-AMR approaches. However, most DL-AMR models only focus on recognition accuracy, leading to huge model sizes and high computational complexity, while some lightweight and low-complexity models struggle to meet the accuracy requirements. This letter proposes an efficient DL-AMR model based on phase parameter estimation and transformation, with convolutional neural network (CNN) and gated recurrent unit (GRU) as the feature extraction layers, which can achieve high recognition accuracy equivalent to the existing state-of-the-art models but reduces more than a third of the volume of their parameters. Meanwhile, our model is more competitive in training time and test time than the benchmark models with similar recognition accuracy. Moreover, we further propose to compress our model by pruning, which maintains the recognition accuracy higher than 90% while has less than 1/8 of the number of parameters comparing with state-of-the-art models.
Universal-Prototype Enhancing for Few-Shot Object Detection

Few-shot object detection (FSOD) aims to strengthen the performance of novel object detection with few labeled samples. To alleviate the constraint of few samples, enhancing the generalization ability of learned features for novel objects plays a key role. Thus, the feature learning process of FSOD should focus more on intrinsical object characteristics, which are invariant under different visual changes and therefore are helpful for feature generalization. Unlike previous attempts of the meta-learning paradigm, in this paper, we explore how to enhance object features with intrinsical characteristics that are universal across different object categories. We propose a new prototype, namely universal prototype, that is learned from all object categories. Besides the advantage of characterizing invariant characteristics, the universal prototypes alleviate the impact of unbalanced object categories. After enhancing object features with the universal prototypes, we impose a consistency loss to maximize the agreement between the enhanced features and the original ones, which is beneficial for learning invariant object characteristics. Thus, we develop a new framework of few-shot object detection with universal prototypes (F SOD <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">up</sup> ) that owns the merit of feature generalization towards novel objects. Experimental results on PASCAL VOC and MS COCO show the effectiveness of F SOD <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">up</sup> . Particularly, for the 1-shot case of VOC Split2, FSOD <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">up</sup> outperforms the baseline by 6.8% in terms of mAP.
Exploring the impact of chatbots on consumer sentiment and expectations in retail

Retailers are increasingly using conversational AI (chatbots) for customer service due to the perceived benefits and reduced operational costs of this emerging technology. Yet our understanding of how consumers perceive interactions with chatbots, and how these interactions may influence other consumer service programs, remains limited. This paper investigates the differences in consumers' sentiments towards chatbots across retail sectors, and the influence chatbots have on consumers’ sentiments and expectations towards other service interactions with online human agents. Using a hybrid automated sentiment analysis approach, we identify that (1) overall sentiment towards bots are less negative than sentiment towards online human agents; (2) these sentiments differ across fashion and telecommunications sectors, and finally (3) sentiments towards online human agents in both sectors become more negative after a retailer implements a chatbot.
Facial recognition using Haar cascade and LBP classifiers

Facial Recognition is the biometric technique used in face detection. The task for validating or recognizing a face from the multi-media photographs is done using facial recognition technique. With the evolution of advanced society the requirement for face identification has been really important. Detection and identification of faces has been grown worldwide. It owes the demand for security such as authorization, national safety and other vital circumstances. There are number of algorithms for facial detection. This paper aspires to present the comparison of two face recognition techniques Haar Cascade and Local Binary Pattern edified for the classification. As a result the accuracy of Haar Cascade is more than the Local Binary Pattern but the execution time in Haar Cascade is more than Local Binary Pattern.
A comprehensive study of deep learning compiler bugs

There are increasing uses of deep learning (DL) compilers to generate optimized code, boosting the runtime performance of DL models on specific hardware. Like their traditional counterparts, DL compilers can generate incorrect code, resulting in unexpected model behaviors that may cause catastrophic consequences in mission-critical systems. On the other hand, the DL models processed by DL compilers differ fundamentally from imperative programs in that the program logic in DL models is implicit. As such, various characteristics of the bugs arising from traditional compilers need to be revisited in the context of DL compilers.
Challenges, Applications and Design Aspects of Federated Learning: A Survey

Federated Learning (FL) is a new technology that has been a hot research topic.It enables training an algorithm across multiple decentralized edge devices or servers holding local data samples, without exchanging them.There are many application domains where large amounts of properly labeled and complete data are not available in a centralized location, for example, doctors' diagnosis from medical image analysis.There are also growing concerns over data and user privacy as Artificial Intelligence is becoming ubiquitous in new application domains.As such, very recently, a lot of research has been conducted in several areas within the nascent field of FL.A variety of surveys on different subtopics exist in current literature, focusing on specific challenges, design aspects and application domains.In this paper, we review existing contemporary works in the related areas in order to understand the challenges and topics that are emphasized by each type of FL surveys.Furthermore, we categorize FL research in terms of challenges, design factors and applications, conducting a holistic review of each and outlining promising research directions.
TRAC: Traceable and Revocable Access Control Scheme for mHealth in 5G-Enabled IIoT

Mobile healthcare (mHealth) enables people to collect and share their personal health records (PHRs) and gain rapid medical treatment via mobile 5G-enabled Industrial Internet of Things (IIoT) devices, which also brings the challenge of keeping the PHRs confidentiality and preventing unauthorized access. By the emerging ciphertext-policy attribute-based encryption (CP-ABE), the PHR owner can encrypt his/her PHR data under self-defined access policies. However, existing CP-ABE schemes are suffering from either heavy computation cost and storage overhead or traitor tracing and direct revocation. In this article, we propose an efficient, traceable, and revocable access control scheme named TRAC for mHealth in 5G-enabled IIoT. In TRAC, the ciphertext is composed of the attribute-relevant ciphertext encrypted under an <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">and</small> -gate access structure and the identity-relevant ciphertext associated with some potential receivers. The malicious user who leaks his/her privilege to unauthorized entities will be precisely tracked and added in the revocation list, by which the cloud server can update the identity-relevant ciphertext by itself. The length of final ciphertext and the time of bilinear pairing operations used in decryption are constant. The security analysis and performance evaluation indicate the security, efficiency, and practicality of TRAC.
PEFL: Deep Privacy-Encoding-Based Federated Learning Framework for Smart Agriculture

Smart agriculture (SA) incorporates low-cost and low-energy-consuming sensors and devices to enhance quantitative and qualitative agricultural production. However, this device uses an open communication channel, i.e., Internet, and generates large amount of data in real time and, thus, has the potential to be misused. As a consequence, the major concern in the implementation of SA is minimizing the risk of security and data privacy violation (e.g., adversaries performing inference attacks). To address these challenges, we propose PEFL, a deep privacy-encoding-based federated learning (FL) framework that adopts a perturbation-based encoding and long short-term memory-autoencoder technique to achieve the target of privacy. Then, an FL-based gated recurrent unit neural network algorithm (FedGRU) is designed using the encoded data for intrusion detection. The experimental results based on the ToN-IoT data set reveal that the PEFL can efficiently identify normal and attack patterns after transformation over other non-FL and FL methods.
VQL: Efficient and Verifiable Cloud Query Services for Blockchain Systems

Despite increasingly emerging applications, a primary concern for blockchain to be fully practical is the inefficiency of data query. Direct queries on the blockchain take much time by searching every block, while indirect queries on a blockchain database greatly degrade the authenticity of query results. To conquer the authenticity problem, we propose a Verifiable Query Layer (VQL) that can be deployed in the cloud to provide both efficient and verifiable data query services for blockchain systems. The middleware layer extracts data from the underlying blockchain system and efficiently reorganizes them in databases. To prevent falsified data from being stored in the middleware, a cryptographic fingerprint is calculated based on each constructed database. The database fingerprint will be first verified by miners and then written into the blockchain. Moreover, public users can verify the entire databases or several databases that interest them in the middleware layer. We implement VQL together with the verification schemes and conduct extensive experiments based on a practical blockchain system. The evaluation results demonstrate that VQL can efficiently support various data query services and guarantee the authenticity of query results for blockchain systems.
Automated Drone Detection Using YOLOv4

Drones are increasing in popularity and are reaching the public faster than ever before. Consequently, the chances of a drone being misused are multiplying. Automated drone detection is necessary to prevent unauthorized and unwanted drone interventions. In this research, we designed an automated drone detection system using YOLOv4. The model was trained using drone and bird datasets. We then evaluated the trained YOLOv4 model on the testing dataset, using mean average precision (mAP), frames per second (FPS), precision, recall, and F1-score as evaluation parameters. We next collected our own two types of drone videos, performed drone detections, and calculated the FPS to identify the speed of detection at three altitudes. Our methodology showed better performance than what has been found in previous similar studies, achieving a mAP of 74.36%, precision of 0.95, recall of 0.68, and F1-score of 0.79. For video detection, we achieved an FPS of 20.5 on the DJI Phantom III and an FPS of 19.0 on the DJI Mavic Pro.
Organic Matter and Mineral Composition of Silicate Soils: FTIR Comparison Study by Photoacoustic, Diffuse Reflectance, and Attenuated Total Reflection Modalities

This study aims to compare photoacoustic (FTIR–PAS), diffuse reflectance (DRIFT), and attenuated total reflection (ATR) FTIR modalities in the wide wavenumber range from NIR (7500 cm−1) to FIR (150 cm−1) for the same silicate soil samples under the same conditions. The possibilities of non-destructive rapid qualitative analysis of soils by these modalities without comprehensive data treatment were compared. The assignment of more than 100 bands for the chernozem and sod-podzolic as common types of silicate types of soil was made. The following groups of bands of organic matter and inorganic matrix were reliably found in spectra of all or at least two modalities: 3690–3680 cm−1 (hydrogen-bonded SiO–H…H2O stretch, not ATR), 2930–2910 cm−1 and 2860–2850 cm−1 (methylene stretch), 1390–1380 cm−1, (symmetric stretch carboxylate, DRIFT and FTIR–PAS); 2000–1990 cm−1, 1885 cm−1, and 1790–1783 cm−1 (SiO2 overtones, DRIFT and FTIR–PAS), 1163–1153 cm−1, SiO2 lattice (not FTIR–PAS), 1037 cm−1 (Si–O or Al–O stretch), 796 cm−1 (lattice symmetrical Si–O–Si stretch); 697 cm−1, SiO2; and 256 cm−1 (not FTIR–PAS). Amide I, II, and III bands appear in DRIFT and FTIR–PAS spectra while not in ATR. Except for methylene and carboxylate groups, CH vibrations (3100–2900 cm−1) are not seen in ATR. Bands at 1640–1630 cm−1, 1620–1610 cm−1, 1600–1598 cm−1 (primary water bands and probably carboxylate) appear in the spectra of all three modalities but are unresolved and require data treatment. It is preferable to use all three modalities to characterize both soil organic matter and mineral composition. DRIFT provides the maximum number of bands in all three modalities and should be selected as a primary technique in the NIR and 4000–2000 cm−1 regions for hydrogen-bonding bands, CHX groups, and the silicate matrix. ATR–FTIR complements DRIFT and provides a good sensitivity for soil water and the matrix in 2000–400 cm−1. FTIR–PAS in 4000–1500 cm−1 reveals more bands than DRIFT and shows the highest sensitivity for absorption bands that do not appear in DRIFT or ATR-IR spectra. Thus, FTIR–PAS is expedient for supporting either DRIFT or ATR–FTIR. This modality comparison can be a basis for methodological support of IR spectroscopy of soils and similar organomineral complexes.
Challenges and Countermeasures for Adversarial Attacks on Deep Reinforcement Learning

Deep reinforcement learning (DRL) has numerous applications in the real world, thanks to its ability to achieve high performance in a range of environments with little manual oversight. Despite its great advantages, DRL is susceptible to adversarial attacks, which precludes its use in real-life critical systems and applications (e.g., smart grids, traffic controls, and autonomous vehicles) unless its vulnerabilities are addressed and mitigated. To address this problem, we provide a comprehensive survey that discusses emerging attacks on DRL-based systems and the potential countermeasures to defend against these attacks. We first review the fundamental background on DRL and present emerging adversarial attacks on machine learning techniques. We then investigate the vulnerabilities that an adversary can exploit to attack DRL along with state-of-the-art countermeasures to prevent such attacks. Finally, we highlight open issues and research challenges for developing solutions to deal with attacks on DRL-based intelligent systems.
Exploring Classification Equilibrium in Long-Tailed Object Detection

The conventional detectors tend to make imbalanced classification and suffer performance drop, when the distribution of the training data is severely skewed. In this paper, we propose to use the mean classification score to indicate the classification accuracy for each category during training. Based on this indicator, we balance the classification via an Equilibrium Loss (EBL) and a Memory-augmented Feature Sampling (MFS) method. Specifically, EBL increases the intensity of the adjustment of the decision boundary for the weak classes by a designed score-guided loss margin between any two classes. On the other hand, MFS improves the frequency and accuracy of the adjustment of the decision boundary for the weak classes through over-sampling the instance features of those classes. Therefore, EBL and MFS work collaboratively for finding the classification equilibrium in long-tailed detection, and dramatically improve the performance of tail classes while maintaining or even improving the performance of head classes. We conduct experiments on LVIS using Mask R-CNN with various backbones including ResNet-50-FPN and ResNet-101-FPN to show the superiority of the proposed method. It improves the detection performance of tail classes by 15.6 AP, and outperforms the most recent long-tailed object detectors by more than 1 AP. Code is available at https://github.com/fcjian/LOCE.
RGB-D Saliency Detection via Cascaded Mutual Information Minimization

Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. In this paper, we introduce a novel multistage cascaded learning framework via mutual information minimization to explicitly model the multi-modal information between RGB image and depth data. Specifically, we first map the feature of each mode to a lower dimensional feature vector, and adopt mutual information minimization as a regularizer to reduce the redundancy between appearance features from RGB and geometric features from depth. We then perform multi-stage cascaded learning to impose the mutual information minimization constraint at every stage of the network. Extensive experiments on benchmark RGB-D saliency datasets illustrate the effectiveness of our framework. Further, to prosper the development of this field, we contribute the largest (7× larger than NJU2K) COME15K dataset, which contains 15,625 image pairs with high quality polygon-/scribble-/object-/instance-/rank-level annotations. Based on these rich labels, we additionally construct four new benchmarks with strong baselines and observe some interesting phenomena, which can motivate future model design. Source code and dataset are available at https://github.com/JingZhang617/cascaded_rgbd_sod.
Dynamic Context-Sensitive Filtering Network for Video Salient Object Detection

The ability to capture inter-frame dynamics has been critical to the development of video salient object detection (VSOD). While many works have achieved great success in this field, a deeper insight into its dynamic nature should be developed. In this work, we aim to answer the following questions: How can a model adjust itself to dynamic variations as well as perceive fine differences in the real-world environment; How are the temporal dynamics well introduced into spatial information over time? To this end, we propose a dynamic context-sensitive filtering network (DCFNet) equipped with a dynamic context-sensitive filtering module (DCFM) and an effective bidirectional dynamic fusion strategy. The proposed DCFM sheds new light on dynamic filter generation by extracting location-related affinities between consecutive frames. Our bidirectional dynamic fusion strategy encourages the interaction of spatial and temporal information in a dynamic manner. Experimental results demonstrate that our proposed method can achieve state-of-the-art performance on most VSOD datasets while ensuring a real-time speed of 28 fps. The source code is publicly available at https://github.com/OIPLab-DUT/DCFNet.
Complementary Trilateral Decoder for Fast and Accurate Salient Object Detection

Salient object detection (SOD) has made great progress, but most of existing SOD methods focus more on performance than efficiency. Besides, the U-shape structure exists some drawbacks and there is still a lot of room for improvement. Therefore, we propose a novel framework to treat semantic context, spatial detail and boundary information separately in the decoder part. Specifically, we propose an efficient and effective Complementary Trilateral Decoder (CTD) for saliency detection with three branches: Semantic Path, Spatial Path and Boundary Path. These three branches are designed to solve the dilution of semantic information, loss of spatial information and absence of boundary information, respectively. These three branches are complementary to each other and we design three distinctive fusion modules to gradually merge them according to "coarse-fine-finer'' strategy, which significantly improves the region accuracy and boundary quality. To facilitate the practical application in different environments, we provide two versions: CTDNet-18 (11.82M, 180FPS) and CTDNet-50 (24.63M, 110FPS). Experiments show that our model performs better than state-of-the-art approaches on five benchmarks, which achieves a favorable balance between speed and accuracy.
SDWNet: A Straight Dilated Network with Wavelet Transformation for image Deblurring

Image deblurring is a classical computer vision problem that aims to recover a sharp image from a blurred image. To solve this problem, existing methods apply the Encode-Decode architecture to design the complex networks to make a good performance. However, most of these methods use repeated up-sampling and down-sampling structures to expand the receptive field, which results in texture information loss during the sampling process and some of them design the multiple stages that lead to difficulties with convergence. Therefore, our model uses dilated convolution to enable the obtainment of the large receptive field with high spatial resolution. Through making full use of the different receptive fields, our method can achieve better performance. On this basis, we reduce the number of up-sampling and down-sampling and design a simple network structure. Besides, we propose a novel module using the wavelet transform, which effectively helps the network to recover clear high-frequency texture details. Qualitative and quantitative evaluations of real and synthetic datasets show that our deblurring method is comparable to existing algorithms in terms of performance with much lower training requirements. The source code and pre-trained models are available at https://github.com/FlyEgle/SDWNet.