Reviewerly/exHarmony huggingface dataset:
Lightweight Deep Neural Network for Joint Learning of Underwater Object Detection and Color Conversion

Underwater image processing has been shown to exhibit significant potential for exploring underwater environments. It has been applied to a wide variety of fields, such as underwater terrain scanning and autonomous underwater vehicles (AUVs)-driven applications, such as image-based underwater object detection. However, underwater images often suffer from degeneration due to attenuation, color distortion, and noise from artificial lighting sources as well as the effects of possibly low-end optical imaging devices. Thus, object detection performance would be degraded accordingly. To tackle this problem, in this article, a lightweight deep underwater object detection network is proposed. The key is to present a deep model for jointly learning color conversion and object detection for underwater images. The image color conversion module aims at transforming color images to the corresponding grayscale images to solve the problem of underwater color absorption to enhance the object detection performance with lower computational complexity. The presented experimental results with our implementation on the Raspberry pi platform have justified the effectiveness of the proposed lightweight jointly learning model for underwater object detection compared with the state-of-the-art approaches.
RWF-2000: An Open Large Scale Video Database for Violence Detection

In recent years, surveillance cameras are widely deployed in public places, and the general crime rate has been reduced significantly due to these ubiquitous devices. Usually, these cameras provide cues and evidence after crimes are conducted, while they are rarely used to prevent or stop criminal activities in time. It is both time and labor consuming to manually monitor a large amount of video data from surveillance cameras. Therefore, automatically recognizing violent behaviors from video signals becomes essential. This paper summarizes several existing video datasets for violence detection and proposes the RWF-2000 database with 2,000 videos captured by surveillance cameras in real-world scenes. Also, we present a new method that utilizes both the merits of 3D-CNNs and optical flow, namely Flow Gated Network. The proposed approach obtains an accuracy of 87.25% on the test set of our proposed database. The database and source codes are currently open to access.
Image encryption algorithm based on 2D hyperchaotic map

It is well known that chaos is unpredictability and sensitivity to initial values and parameters. It is a best choice for cryptography application. However, the existing chaotic cryptography systems have many defects because of the chaotic maps that without complexity dynamic performances. To improve these weaknesses, we introduced a 2D hyperchaotic map through two 1D-chaotic maps, linear function and represents multiplier. The dynamic performances of the 2D hyperchaotic map are analyzed by attractor trajectory, 0–1 test, bifurcation diagram, Lyapunov exponents and Permutation entropy. The results illustrate that 2D hyperchaotic map has more complexity dynamic characteristics and randomness, which indicates that 2D hyperchaotic map is more suitable for image encryption algorithm. Therefore, an image encryption algorithm is introduced based on 2D hyperchaotic map. In encryption algorithm, based on 2D hyperchaotic map, the image is scrambled by row shift and column shift, then pixel values are diffused through forward and backward diffusion, finally, the ciphertext image is obtained. The simulation results show that the proposed image encryption algorithm is reliable. Moreover, the security analysis indicate that the introduced algorithm has a better security performances than several image encryption algorithms. This algorithm is not absolutely safe and efficient, and does not limit the improvement and optimization of the algorithm. However, this work provide a theoretical basis in chaotic theory and chaotic image encryption algorithm.
Unified Information Fusion Network for Multi-Modal RGB-D and RGB-T Salient Object Detection

The use of complementary information, namely depth or thermal information, has shown its benefits to salient object detection (SOD) during recent years. However, the RGB-D or RGB-T SOD problems are currently only solved independently, and most of them directly extract and fuse raw features from backbones. Such methods can be easily restricted by low-quality modality data and redundant cross-modal features. In this work, a unified end-to-end framework is designed to simultaneously analyze RGB-D and RGB-T SOD tasks. Specifically, to effectively tackle multi-modal features, we propose a novel multi-stage and multi-scale fusion network (MMNet), which consists of a cross-modal multi-stage fusion module (CMFM) and a bi-directional multi-scale decoder (BMD). Similar to the visual color stage doctrine in the human visual system (HVS), the proposed CMFM aims to explore important feature representations in feature response stage, and integrate them into cross-modal features in adversarial combination stage. Moreover, the proposed BMD learns the combination of multi-level cross-modal fused features to capture both local and global information of salient objects, and can further boost the multi-modal SOD performance. The proposed unified cross-modality feature analysis framework based on two-stage and multi-scale information fusion can be used for diverse multi-modal SOD tasks. Comprehensive experiments (<inline-formula> <tex-math notation="LaTeX">$\sim 92\text{K}$ </tex-math></inline-formula> image-pairs) demonstrate that the proposed method consistently outperforms the other 21 state-of-the-art methods on nine benchmark datasets. This validates that our proposed method can work well on diverse multi-modal SOD tasks with good generalization and robustness, and provides a good multi-modal SOD benchmark.
Blockchain and homomorphic encryption-based privacy-preserving data aggregation model in smart grid

In recent years, rapid advancements in smart grid technology and smart metering systems have raised serious privacy concerns about the collection of customers’ real-time energy usage behaviors. Due to cybersecurity attacks and threats, data aggregation operations in a smart grid are challenging. The majority of existing techniques have high computation and communication costs and are still vulnerable to various security and privacy concerns. This paper proposes a deep learning and homomorphic encryption-based privacy-preserving data aggregation model to mitigate the negative impact of a flash workload on the accuracy of prediction models. The model also ensures a secure data aggregation process with low computational overhead. The proposed model is 80% more effective than the traditional approach in detecting smart meter manipulation, and the computation cost is 20% to 80% less than existing techniques. Thus, the proposed blockchain and homomorphic encryption-based data aggregation (BHDA) scheme shows a significant improvement in performance and privacy preservation with minimal computation overhead for data aggregation in smart grids.
Activate or Not: Learning Customized Activation

We present a simple, effective, and general activation function we term ACON which learns to activate the neurons or not. Interestingly, we find Swish, the recent popular NAS-searched activation, can be interpreted as a smooth approximation to ReLU. Intuitively, in the same way, we approximate the more general Maxout family to our novel ACON family, which remarkably improves the performance and makes Swish a special case of ACON. Next, we present meta-ACON, which explicitly learns to optimize the parameter switching between non-linear (activate) and linear (inactivate) and provides a new design space. By simply changing the activation function, we show its effectiveness on both small models and highly optimized large models (e.g. it improves the ImageNet top-1 accuracy rate by 6.7% and 1.8% on MobileNet-0.25 and ResNet-152, respectively). Moreover, our novel ACON can be naturally transferred to object detection and semantic segmentation, showing that ACON is an effective alternative in a variety of tasks. Code is available at https://github.com/nmaac/acon.
A Survey of Modulation Classification Using Deep Learning: Signal Representation and Data Preprocessing

Modulation classification is one of the key tasks for communications systems monitoring, management, and control for addressing technical issues, including spectrum awareness, adaptive transmissions, and interference avoidance. Recently, deep learning (DL)-based modulation classification has attracted significant attention due to its superiority in feature extraction and classification accuracy. In DL-based modulation classification, one major challenge is to preprocess a received signal and represent it in a proper format before feeding the signal into deep neural networks. This article provides a comprehensive survey of the state-of-the-art DL-based modulation classification algorithms, especially the techniques of signal representation and data preprocessing utilized in these algorithms. Since a received signal can be represented by either features, images, sequences, or a combination of them, existing algorithms of DL-based modulation classification can be categorized into four groups and are reviewed accordingly in this article. Furthermore, the advantages as well as disadvantages of each signal representation method are summarized and discussed.
MobileDets: Searching for Object Detection Architectures for Mobile Accelerators

Inverted bottleneck layers, which are built upon depth-wise convolutions, have been the predominant building blocks in state-of-the-art object detection models on mobile devices. In this work, we investigate the optimality of this design pattern over a broad range of mobile accelerators by revisiting the usefulness of regular convolutions. We discover that regular convolutions are a potent component to boost the latency-accuracy trade-off for object detection on accelerators, provided that they are placed strategically in the network via neural architecture search. By incorporating regular convolutions in the search space and directly optimizing the network architectures for object detection, we obtain a family of object detection models, MobileDets, that achieve state-of-the-art results across mobile accelerators. On the COCO object detection task, MobileDets outperform MobileNetV3+SSDLite by 1.7 mAP at comparable mobile CPU inference latencies. MobileDets also outperform MobileNetV2+SSDLite by 1.9 mAP on mobile CPUs, 3.7 mAP on Google EdgeTPU, 3.4 mAP on Qualcomm Hexagon DSP and 2.7 mAP on Nvidia Jetson GPU without increasing latency. Moreover, MobileDets are comparable with the state-of-the-art MnasFPN on mobile CPUs even without using the feature pyramid, and achieve better mAP scores on both EdgeTPUs and DSPs with up to 2× speedup. Code and models are available in the TensorFlow Object Detection API [16]: https://github.com/tensorflow/models/tree/master/research/object_detection.
Exploring older adults’ perception and use of smart speaker-based voice assistants: A longitudinal study

Thanks to their conversational capabilities, smart speaker-based voice assistants are gaining attention for their potential to support the aging population, though the empirical evidence is still scarce. This paper aims to obtain empirical evidence on older adults' experiences with a voice assistant. We especially focused on how their perception and use change over time as they progress from novice to more experienced users through a longitudinal field deployment study. We deployed Google Home devices in the homes of twelve older adults aged 65 and above and studied their use for sixteen weeks. Results show that the benefits our participants perceived have incrementally changed from enjoying simplicity and convenience of operation in the early phase of the study to not worrying about making mistakes and building digital companionship as they got used to using it. Results also show that participants confronted several challenges that evolved from the unfamiliarity with a voice assistant in their first interactions to coping with the functional errors due to limited speech technology as they got used to using it. Based on the results, we discuss design implications that could foster better user experiences with a voice assistant among older adults.
Trust me, I'm a bot – repercussions of chatbot disclosure in different service frontline settings

Purpose Chatbots are increasingly prevalent in the service frontline. Due to advancements in artificial intelligence, chatbots are often indistinguishable from humans. Regarding the question whether firms should disclose their chatbots' nonhuman identity or not, previous studies find negative consumer reactions to chatbot disclosure. By considering the role of trust and service-related context factors, this study explores how negative effects of chatbot disclosure for customer retention can be prevented. Design/methodology/approach This paper presents two experimental studies that examine the effect of disclosing the nonhuman identity of chatbots on customer retention. While the first study examines the effect of chatbot disclosure for different levels of service criticality, the second study considers different service outcomes. The authors employ analysis of covariance and mediation analysis to test their hypotheses. Findings Chatbot disclosure has a negative indirect effect on customer retention through mitigated trust for services with high criticality. In cases where a chatbot fails to handle the customer's service issue, disclosing the chatbot identity not only lacks negative impact but even elicits a positive effect on retention. Originality/value The authors provide evidence that customers will react differently to chatbot disclosure depending on the service frontline setting. They show that chatbot disclosure does not only have undesirable consequences as previous studies suspect but can lead to positive reactions as well. By doing so, the authors draw a more balanced picture on the consequences of chatbot disclosure.
Identification and Authentication in Healthcare Internet-of-Things Using Integrated Fog Computing Based Blockchain Model

The healthcare Internet-of-Things (IoT) offers many benefits including data transmission in real-time mode, the ability to monitor the physiological state of the patient in a different interval of time. Devices such as blood-pressure monitors, glucose meters, heart monitoring implants, Electroencephalography (EEG), Electrocardiogram (ECG), and Electromyography (EMG) wearable devices allow health providers to collect the patient health information locally and make a real-time decision based on the Patient Health Data (PHD). Hospitals have been adopting the IoT for many years and now they have healthcare IoT devices in patients’ rooms and their bodies. However, the medical agencies, hospitals, and companies do not consider the security risk of healthcare IoT devices connected to a Local Area Network (LAN) or Wide Area Network (WAN). The IoT devices can be easily hacked and may lead to several potentially life-threatening risks due to poor authentication and encryption practices. Existing machine learning algorithms and blockchain approach working in the cloud computing environment are unable to meet the Quality of Service (QoS) like reliability, authentication, identification, and security requirements of healthcare IoT devices. Most of the traditional machine learning algorithms and techniques for healthcare IoT lacks the real-world implementation for secure data transmission. Therefore, blockchain is introduced for secure and reliable transaction in healthcare IoT. Whereas Fog Computing (FC) is introduced to extend the services of the cloud at the edge of networks. Integration of FC with blockchain can overcome the issue of healthcare IoT device identification, authentication, and verification for scalable frequent data transmission in a decentralized environment. Hence, a novel solution for the abovementioned problem is proposed using FC and blockchain. It includes an FC-based three-tier architecture, an analytical model, a mathematical framework, and an Advanced Signature-Based Encryption (ASE) algorithm for healthcare IoT device identification, verification, and Patient Health Data (PHD) authentication. The aim is to extend secure data transmission for healthcare IoT and end-users availing the real-time services. The proposed model and algorithm will be able to provide services for transaction and transmission near the edge in a secure manner. By analyzing the generated results from the proposed novel ASE algorithm for throughput, packet error, reliability, and malicious node detection accuracy; it is observed that the ASE algorithm in the FC environment easily outperforms the cloud and the other existing state of the art techniques such as FogBus, Femto cloud, Blockchain Fog-based Architecture Network (BFAN), and BeeKeeper. The malicious node detection accuracy of the ASE algorithm in the FC environment is 91% and in the cloud is 83%. Whereas the reliability percentage of the ASE algorithm in FC is 95% and in the cloud is 87%. The proposed approach is tested on simulators iFogSim (Net-Beans) and SimBlock.
Efficient Regional Memory Network for Video Object Segmentation

Recently, several Space-Time Memory based networks have shown that the object cues (e.g. video frames as well as the segmented object masks) from the past frames are useful for segmenting objects in the current frame. However, these methods exploit the information from the memory by global-to-global matching between the current and past frames, which lead to mismatching to similar objects and high computational complexity. To address these problems, we propose a novel local-to-local matching solution for semi-supervised VOS, namely Regional Memory Network (RMNet). In RMNet, the precise regional memory is constructed by memorizing local regions where the target objects appear in the past frames. For the current query frame, the query regions are tracked and predicted based on the optical flow estimated from the previous frame. The proposed local-to-local matching effectively alleviates the ambiguity of similar objects in both memory and query frames, which allows the information to be passed from the regional memory to the query region efficiently and effectively. Experimental results indicate that the proposed RM-Net performs favorably against state-of-the-art methods on the DAVIS and YouTube-VOS datasets.
Semantic-aware Knowledge Distillation for Few-Shot Class-Incremental Learning

Few-shot class incremental learning (FSCIL) portrays the problem of learning new concepts gradually, where only a few examples per concept are available to the learner. Due to the limited number of examples for training, the techniques developed for standard incremental learning cannot be applied verbatim to FSCIL. In this work, we introduce a distillation algorithm to address the problem of FSCIL and propose to make use of semantic information during training. To this end, we make use of word embeddings as semantic information which is cheap to obtain and which facilitate the distillation process. Furthermore, we propose a method based on an attention mechanism on multiple parallel embeddings of visual data to align visual and semantic vectors, which reduces issues related to catastrophic forgetting. Via experiments on MiniImageNet, CUB200, and CIFAR100 dataset, we establish new state-of-the-art results by outperforming existing approaches.
SwiftNet: Real-time Video Object Segmentation

In this work we present SwiftNet for real-time semisupervised video object segmentation (one-shot VOS), which reports 77.8% $\mathcal{J}\& \mathcal{F}$ and 70 FPS on DAVIS 2017 validation dataset, leading all present solutions in overall accuracy and speed performance. We achieve this by elaborately compressing spatiotemporal redundancy in matching-based VOS via Pixel-Adaptive Memory (PAM). Temporally, PAM adaptively triggers memory updates on frames where objects display noteworthy inter-frame variations. Spatially, PAM selectively performs memory update and match on dynamic pixels while ignoring the static ones, significantly reducing redundant computations wasted on segmentation-irrelevant pixels. To promote efficient reference encoding, light-aggregation encoder is also introduced in SwiftNet deploying reversed sub-pixel. We hope SwiftNet could set a strong and efficient baseline for real-time VOS and facilitate its application in mobile vision. The source code of SwiftNet can be found at https://github.com/haochenheheda/SwiftNet.
An image encryption algorithm based on hyperchaotic system and DNA coding

A chaotic image encryption scheme combining block scrambling and DNA coding is proposed. The chaotic sequences generated by Chen hyperchaotic system are used for inter-block index scrambling and intra-block Fisher-Yates scrambling. Then the index sequence is used to further scramble the image to further enhance the encryption effect. The pixel value of the image is initially changed by chaotic sequence, and diffused by different DNA coding rules and different operation rules corresponding to chaotic sequence. After decoding the DNA matrix, the final encrypted image is obtained. The initial values and parameters of the chaotic system are generated by SHA-512 hash function and plaintext image. The experimental results and various security analysis show that the algorithm has good encryption effect and can resist common attacks such as chosen-plaintext attacks, clipping attacks and noise attacks.
Deep Burst Super-Resolution

While single-image super-resolution (SISR) has attracted substantial interest in recent years, the proposed approaches are limited to learning image priors in order to add high frequency details. In contrast, multi-frame super-resolution (MFSR) offers the possibility of reconstructing rich details by combining signal information from multiple shifted images. This key advantage, along with the increasing popularity of burst photography, have made MFSR an important problem for real-world applications.We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. This is achieved by explicitly aligning deep embeddings of the input frames using pixel-wise optical flow. The information from all frames are then adaptively merged using an attention-based fusion module. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset, consisting of smartphone bursts and high-resolution DSLR ground-truth. We perform comprehensive experimental analysis, demonstrating the effectiveness of the proposed architecture.
Perturbation-based methods for explaining deep neural networks: A survey

Deep neural networks (DNNs) have achieved state-of-the-art results in a broad range of tasks, in particular the ones dealing with the perceptual data. However, full-scale application of DNNs in safety-critical areas is hindered by their black box-like nature, which makes their inner workings nontransparent. As a response to the black box problem, the field of explainable artificial intelligence (XAI) has recently emerged and is currently rapidly growing. The present survey is concerned with perturbation-based XAI methods, which allow to explore DNN models by perturbing their input and observing changes in the output. We present an overview of the most recent research focusing on the differences and similarities in the applications of perturbation-based methods to different data types, from extensively studied perturbations of images to the just emerging research on perturbations of video, natural language, software code, and reinforcement learning entities.