Reviewerly/exHarmony huggingface dataset:
Deep learning based solar radiation micro forecast by fusion of infrared cloud images and radiation data

Solar irradiance forecasting has been gaining paramount importance in recent years due to its impact on power grids. However, solar energy harvesting over shorter periods also brings new challenges due to its intermittent and uncertain attributes. Hence, accurate forecasting has become an indispensable aspect of the effective management of power system operations. The existing models focus on using only time-series data for solar radiation forecasting. But during cloudy time instances, it fails to quickly capture the nonlinear Spatio-temporal variations in the data for shorter periods. To bridge this gap, in this paper, a multi-modal fusion network is developed for studying solar irradiance micro forecasts by using both infrared images and past solar irradiance data. Here both spatial and temporal information is extracted parallelly and fused using a fully connected neural network. The solar forecasts of the proposed methods are evaluated against benchmark models in terms of Mean Absolute Percentage Error (MAPE) and other qualitative measures. The experimental results illustrate that the multi-modal fusion networks outperform the existing methods while predicting solar irradiance for cloudy days as well as mixed days (both cloudy and sunny days). Hence a transfer learning-based classifier with 99.23% accuracy is developed to categorize the cloudy days from sunny days. In the case of higher horizon forecasts, the proposed models show the optimum trade-off between performance and test time. Moreover, the Multiple Image Convolutional Long Short Term Memory Fusion Network (MICNN-L) shows a 46.42% improvement in MAPE whereas the Convolutional Long Short Term Memory Fusion Network (CNN-L) has a 42.02% increase when compared to the benchmark machine learning and deep learning models.
On the Unreasonable Effectiveness of Centroids in Image Retrieval

Image retrieval task consists of finding similar images to a query image from a set of gallery (database) images. Such systems are used in various applications e.g. person re-identification (ReID) or visual product search. Despite active development of retrieval models it still remains a challenging task mainly due to large intra-class variance caused by changes in view angle, lighting, background clutter or occlusion, while inter-class variance may be relatively low. A large portion of current research focuses on creating more robust features and modifying objective functions, usually based on Triplet Loss. Some works experiment with using centroid/proxy representation of a class to alleviate problems with computing speed and hard samples mining used with Triplet Loss. However, these approaches are used for training alone and discarded during the retrieval stage. In this paper we propose to use the mean centroid representation both during training and retrieval. Such an aggregated representation is more robust to outliers and assures more stable features. As each class is represented by a single embedding - the class centroid - both retrieval time and storage requirements are reduced significantly. Aggregating multiple embeddings results in a significant reduction of the search space due to lowering the number of candidate target vectors, which makes the method especially suitable for production deployments. Comprehensive experiments conducted on two ReID and Fashion Retrieval datasets demonstrate effectiveness of our method, which outperforms the current state-of-the-art. We propose centroid training and retrieval as a viable method for both Fashion Retrieval and ReID applications. Our code is available at https://github.com/mikwieczorek/centroids-reid.
Assessing IoT enabled cyber-physical attack paths against critical systems

Internet of Things (IoT) increase the interconnectivity and interoperability of systems in various critical sectors, such as industrial control, healthcare and smart transportation systems. At the same time, as IoT technologies enable systems to interact both in cyber and physical ways, they also act as enablers of complex attack paths against critical systems. In this paper we propose a novel risk-based methodology for identifying and assessing IoT-enabled attack paths against critical cyber-physical systems. While the majority of existing approaches focus on cyber system connectivity only, the proposed methodology models both cyber and physical interactions. In comparison to existing cyber physical approaches that grow exponentially, our approach is significantly more efficient, by utilizing an attack tree topology; the critical system is set as the root (target) of an attack tree that is recursively build, based on the identified cyber-physical system interactions. Our methodology uses well-known building blocks such Common Vulnerabilities and Exposures (CVE), Common Vulnerability Scoring System (CVSS) and threat modeling. Furthermore,we significantly reduce false positives by prioritizing the identified attack paths in a risk manner, which, in turn, can assist decision makers in effectively mitigating multi-hop attack paths. To validate our methodology, we developed a proof-of-concept implementation and tested it using a realistic scenario from the healthcare sector. Our results show that the proposed methodology can efficiently identify and assess hidden and/or underestimated cyber physical attack paths.
Traceability Transformed: Generating More Accurate Links with Pre-Trained BERT Models

Software traceability establishes and leverages associations between diverse development artifacts. Researchers have proposed the use of deep learning trace models to link natural language artifacts, such as requirements and issue descriptions, to source code; however, their effectiveness has been restricted by availability of labeled data and efficiency at runtime. In this study, we propose a novel framework called Trace BERT (T-BERT) to generate trace links between source code and natural language artifacts. To address data sparsity, we leverage a three-step training strategy to enable trace models to transfer knowledge from a closely related Software Engineering challenge, which has a rich dataset, to produce trace links with much higher accuracy than has previously been achieved. We then apply the T-BERT framework to recover links between issues and commits in Open Source Projects. We comparatively evaluated accuracy and efficiency of three BERT architectures. Results show that a Single-BERT architecture generated the most accurate links, while a Siamese-BERT architecture produced comparable results with significantly less execution time. Furthermore, by learning and transferring knowledge, all three models in the framework outperform classical IR trace models. On the three evaluated real-word OSS projects, the best T-BERT stably outperformed the VSM model with average improvements of 60.31% measured using Mean Average Precision (MAP). RNN severely underperformed on these projects due to insufficient training data, while T-BERT overcame this problem by using pretrained language models and transfer learning.
A Survey on Healthcare Data: A Security Perspective

With the remarkable development of internet technologies, the popularity of smart healthcare has regularly come to the fore. Smart healthcare uses advanced technologies to transform the traditional medical system in an all-round way, making healthcare more efficient, more convenient, and more personalized. Unfortunately, medical data security is a serious issue in the smart healthcare systems. It becomes a fundamental challenge that requires the development of efficient innovative strategies towards fulfilling the healthcare needs and supporting secure healthcare transfer and delivery. This article provides a comprehensive survey on state-of-the-art techniques for health data security and their new trends for solving challenges in real-world applications. We survey the various notable cryptography, biometrics, watermarking, and blockchain-based security techniques for healthcare applications. A comparative analysis is also performed to identify the contribution of reviewed techniques in terms of their objective, methodology, type of medical data, important features, and limitations. At the end, we discuss the open issues and research directions to explore the promising areas for future research.
An Empirical Review of Deep Learning Frameworks for Change Detection: Model Design, Experimental Frameworks, Challenges and Research Needs

Visual change detection, aiming at segmentation of video frames into foreground and background regions, is one of the elementary tasks in computer vision and video analytics. The applications of change detection include anomaly detection, object tracking, traffic monitoring, human machine interaction, behavior analysis, action recognition, and visual surveillance. Some of the challenges in change detection include background fluctuations, illumination variation, weather changes, intermittent object motion, shadow, fast/slow object motion, camera motion, heterogeneous object shapes and real-time processing. Traditionally, this problem has been solved using hand-crafted features and background modelling techniques. In recent years, deep learning frameworks have been successfully adopted for robust change detection. This article aims to provide an empirical review of the state-of-the-art deep learning methods for change detection. More specifically, we present a detailed analysis of the technical characteristics of different model designs and experimental frameworks. We provide model design based categorization of the existing approaches, including the 2D-CNN, 3D-CNN, ConvLSTM, multi-scale features, residual connections, autoencoders and GAN based methods. Moreover, an empirical analysis of the evaluation settings adopted by the existing deep learning methods is presented. To the best of our knowledge, this is a first attempt to comparatively analyze the different evaluation frameworks used in the existing deep change detection methods. Finally, we point out the research needs, future directions and draw our own conclusions.
Evolutionary Neural Architecture Search for High-Dimensional Skip-Connection Structures on DenseNet Style Networks

Convolutional neural networks hold state-of-the-art results for image classification, and many neural architecture search algorithms have been proposed to discover high performance convolutional neural networks. However, the use of neural architecture search for the discovery of skip-connection structures, an important element in modern convolutional neural networks, is limited within the literature. Furthermore, while many neural architecture search algorithms utilize performance estimation techniques to reduce computation time, empirical evaluations of these performance estimation techniques remain limited. This work focuses on utilizing evolutionary neural architecture search to examine the search space of networks, which follow a fundamental DenseNet structure, but have no fixed skip connections. In particular, a genetic algorithm is designed, which searches the space consisting of all networks between a standard feedforward network and the corresponding DenseNet. To design the algorithm, lower fidelity performance estimation of this class of networks is examined and presented. The final algorithm finds networks that are more accurate than DenseNets on CIFAR10 and CIFAR100, and have fewer trainable parameters. The structures found by the algorithm are examined to shed light on the importance of different types of skip-connection structures in convolutional neural networks, including the discovery of a simple skip-connection removal, which improves DenseNet performance on CIFAR10.
A Survey of FPGA-Based Robotic Computing

Recent researches on robotics have shown significant improvement, spanning from algorithms, mechanics to hardware architectures. Robotics, including manipulators, legged robots, drones, and autonomous vehicles, are now widely applied in diverse scenarios. However, the high computation and data complexity of robotic algorithms pose great challenges to its applications. On the one hand, CPU platform is flexible to handle multiple robotic tasks. GPU platform has higher computational capacities and easy-to-use development frameworks, so they have been widely adopted in several applications. On the other hand, FPGA-based robotic accelerators are becoming increasingly competitive alternatives, especially in latency-critical and power-limited scenarios. With specialized designed hardware logic and algorithm kernels, FPGA-based accelerators can surpass CPU and GPU in performance and energy efficiency. In this paper, we give an overview of previous work on FPGA-based robotic accelerators covering different stages of the robotic system pipeline. An analysis of software and hardware optimization techniques and main technical issues is presented, along with some commercial and space applications, to serve as a guide for future work.
Technological progress in electronic health record system optimization: Systematic review of systematic literature reviews

The recent, rapid development of digital technologies offers new possibilities for more efficient implementation of electronic health record (EHR) and personal health record (PHR) systems. A growing volume of healthcare data has been the hallmark of this digital transformation. The large healthcare datasets' complexity and their dynamic nature pose various challenges related to processing, analysis, storage, security, privacy, data exchange, and usability.We performed a systematic review of systematic reviews to assess technological progress in EHR and PHR systems. We searched MEDLINE, Cochrane, Web of Science, and Scopus for systematic literature reviews on technological advancements that support EHR and PHR systems published between January 1, 2010, and October 06, 2020.The searches resulted in a total of 2,448 hits. Of these, we finally selected 23 systematic reviews. Most of the included papers dealt with information extraction tools and natural language processing technology (n = 10), followed by studies that assessed the use of blockchain technology in healthcare (n = 8). Other areas of digital technology research included EHR and PHR systems in austere settings (n = 1), de-identification methods (n = 1), visualization techniques (n = 1), communication tools within EHR and PHR systems (n = 1), and methodologies for defining Clinical Information Models that promoted EHRs and PHRs interoperability (n = 1).Technological advancements can improve the efficiency in the implementation of EHR and PHR systems in numerous ways. Natural language processing techniques, either rule-based, machine-learning, or deep learning-based, can extract information from clinical narratives and other unstructured data locked in EHRs and PHRs, allowing secondary research (i.e., phenotyping). Moreover, EHRs and PHRs are expected to be the primary beneficiaries of the blockchain technology implementation on Health Information Systems. Governance regulations, lack of trust, poor scalability, security, privacy, low performance, and high cost remain the most critical challenges for implementing these technologies.
Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation

Domain adaptation deals with training models using large scale labeled data from a specific source domain and then adapting the knowledge to certain target domains that have few or no labels. Many prior works learn domain agnostic feature representations for this purpose using a global distribution alignment objective which does not take into account the finer class specific structure in the source and target domains. We address this issue in our work and propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. ILA-DA simultaneously accounts for intra-class clustering as well as inter-class separation among the categories, resulting in less noisy classifier boundaries, improved transferability and increased accuracy. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets and provide insights into the proposed alignment approach. Code will be made publicly available at https://github.com/astuti/ILA-DA.
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning

Modern computer vision applications suffer from catastrophic forgetting when incrementally learning new concepts over time. The most successful approaches to alleviate this forgetting require extensive replay of previously seen data, which is problematic when memory constraints or data legality concerns exist. In this work, we consider the high-impact problem of Data-Free Class-Incremental Learning (DFCIL), where an incremental learning agent must learn new concepts over time without storing generators or training data from past tasks. One approach for DFCIL is to replay synthetic images produced by inverting a frozen copy of the learner’s classification model, but we show this approach fails for common class-incremental benchmarks when using standard distillation strategies. We diagnose the cause of this failure and propose a novel incremental distillation strategy for DFCIL, contributing a modified cross-entropy training and importance-weighted feature distillation, and show that our method results in up to a 25.1% increase in final task accuracy (absolute difference) compared to SOTA DFCIL methods for common class-incremental benchmarks. Our method even outperforms several standard replay based methods which store a coreset of images. Our code is available at https://github.com/GT-RIPL/AlwaysBeDreaming-DFCIL
Federated learning for drone authentication

The ever-rising applications of drones in the Internet of Things (IoT) era is offering many opportunities and challenges. Owing to drone abilities (silent flying, capturing photos and videos, etc.), there is widespread concern about drone authentication and which drones allow to fly. In this regard, there are several machine learning (ML) proposals for authentication in IoT networks. Such ML-based models have drawbacks in data security, privacy-preserving, and scalability when applied in drone authentication. ML-based methods collect all data and centrally train the authentication model, exposing the model to adversarial situations. This paper proposes a federated learning-based drone authentication model with drones’ Radio Frequency (RF) features in IoT networks. In the proposed model, the Deep Neural Network (DNN) architecture is implemented for drone authentication with Stochastic Gradient Descent (SGD) optimization performed locally on drones. Also, Homomorphic encryption and the secure aggregation method are applied to secure model parameters. Experimental results show that the federated drone authentication model gains a high true positive rate (TPR) during drone authentication and better performance compared to other ML-based models.
Cross-domain secure data sharing using blockchain for industrial IoT

The Industrial Internet of Things (IIoT) enhances smart manufacturing process that escalates productivity through revolutionary techniques. The manufacturing process is sophisticated and complex because of various IoT domains (e.g. industries). A final product is an outcome of the efforts of several departments from different industries. However, this raises the cross-domain communication's privacy and security issues. Cross-domain data sharing for product manufacturing is a challenging research direction. This paper proposes a centralized cloud-based cross-domain data sharing platform using multiple security gateways. The security gateways use the blockchain to store the information into the centralized cloud. Once the application reported a malicious activity, the centralized cloud verifies the concern from the blockchain. Further, an action is taken against the party that performs malicious activity in the security gateways. The algorithms are designed for authentication and transaction of data. The proposed framework is able the secure data movement among different domains globally. The experiment result demonstrates that the proposed security and privacy framework helps to maintain trust among the industries that collaborate on manufacturing across the domains.
Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification

Video-based person re-identification (Re-ID) aims to automatically retrieve video sequences of the same person under non-overlapping cameras. To achieve this goal, it is the key to fully utilize abundant spatial and temporal cues in videos. Existing methods usually focus on the most conspicuous image regions, thus they may easily miss out fine-grained clues due to the person varieties in image sequences. To address above issues, in this paper, we propose a novel Global-guided Reciprocal Learning (GRL) framework for video-based person Re-ID. Specifically, we first propose a Global-guided Correlation Estimation (GCE) to generate feature correlation maps of local features and global features, which help to localize the high- and low-correlation regions for identifying the same person. After that, the discriminative features are disentangled into high-correlation features and low-correlation features under the guidance of the global representations. Moreover, a novel Temporal Reciprocal Learning (TRL) mechanism is designed to sequentially enhance the high-correlation semantic information and accumulate the low-correlation sub-critical clues. Extensive experiments are conducted on three public benchmarks. The experimental results indicate that our approach can achieve better performance than other state-of-the-art approaches. The code is released at https://github.com/flysnowtiger/GRL.
Quantifying VR cybersickness using EEG

Abstract Current techniques for characterizing cybersickness (visually induced motion sickness) in virtual environments rely on qualitative questionnaires. For interactive graphics to create visual experiences that enhance the illusion of presence while mitigating cybersickness, interactive measures are needed to characterize cybersickness. In this paper, we acquire EEG signals from participants as they experience vection-induced cybersickness and compare those signals to a baseline. Our study shows that there is a correlation between the participant-reported cybersickness (as measured by movements of a joystick) and brain EEG signals. Through independent component analysis, we separate those signals which are a result of cybersickness from other sources (such as eye blinks). Our user study finds that there is a highly correlative and statistically significant Delta- (1.0–4.0 Hz), Theta- (4.0–7.0 Hz), and Alpha-wave (7.0–13.0 Hz) increase associated with cybersickness in immersive virtual environments across participants. Establishing a strong correlation between cybersickness and EEG-measured brain activity provides us with the first step toward interactively characterizing and mitigating cybersickness in virtual environments.