Reviewerly/exHarmony huggingface dataset:
Estimating the distance to an object from grayscale stereo images using deep learning

This article presents an innovative proposal for estimating the distance between an autonomous vehicle and an object in front of it.Such information can be used, for example, to support the process of controlling an autonomous vehicle.The primary source of information in research is monochrome stereo images.The images were made in compliance with the laws of the canonical order.The developed convolutional neural network model was used for the estimation.A proprietary dataset was developed for the experiments.The analysis was based on the phenomenon of disparity in stereo images.As a result of the research, a correctly trained model of the CNN network was obtained in six variants.High accuracy of distance estimation was achieved.This publication describes an original proposal for a hybrid blend of digital image analysis, stereo-vision, and deep learning for engineering applications.
3DG-STFM: 3D Geometric Guided Student-Teacher Feature Matching

We tackle the essential task of finding dense visual correspondences between a pair of images. This is a challenging problem due to various factors such as poor texture, repetitive patterns, illumination variation, and motion blur in practical scenarios. In contrast to methods that use dense correspondence ground-truths as direct supervision for local feature matching training, we train 3DG-STFM: a multi-modal matching model (Teacher) to enforce the depth consistency under 3D dense correspondence supervision and transfer the knowledge to 2D unimodal matching model (Student). Both teacher and student models consist of two transformer-based matching modules that obtain dense correspondences in a coarse-to-fine manner. The teacher model guides the student model to learn RGB-induced depth information for the matching purpose on both coarse and fine branches. We also evaluate 3DG-STFM on a model compression task. To the best of our knowledge, 3DG-STFM is the first student-teacher learning method for the local feature matching task. The experiments show that our method outperforms state-of-the-art methods on indoor and outdoor camera pose estimations, and homography estimation problems. Code is available at: https://github.com/Ryan-prime/3DG-STFM .
Neuromorphic Event-based Face Identity Recognition

Faces in motion contain rich information about a myriad of social cues, including identity. However, is still unknown how much information about a person’s identity is conveyed by different kinds of facial movements and the precise role of facial motion as a cue for identity is still under debate. Open questions include how much information about identity is embedded in facial movements, and how efficiently a perceptual system extracts those cues toward identity. We address these questions by exploring the potential of Neuromorphic Vision Sensors to perform identity recognition through facial dynamics derived from speech and simultaneously validate how contributory facial motion is for human face identity categorization. As far as we know, this is the first work that evaluates the aptitude of the technology offered by neuromorphic cameras for completing the task of facial identity recognition. A one-stream inflated 3D convnet was used to evaluate the ability of different frame-based events encoding strategies in classifying identity through Short-Time Facial Motion Tokens (ST-FAMOTO). A consistent set of evaluations was conducted using two small datasets of individuals engaged in speech tasks, specifically text reading and speech-driven visual passcode. Results reported consistently consolidate the contribution of facial dynamics in the process of identity recognition and the usefulness of neuromorphic sensor devices in acquiring identity discriminatory facial micro-movements. Additionally, it is also proposed the first Neuromorphic Vision Dataset for Speech Induced Facial Dynamics (NVSFD Dataset), created in the scope of this work and available to facilitate future third-party investigation on the topic.
Efficient hardware architectures of Lilliput lightweight algorithm for image encryption

With the advancement of communication networks, information security has become extremely crucial in storage and transmission. As images are used in most of the networks, hence image security is becoming a challenging task. This paper proposes two hardware architectures of Lilliput lightweight block cipher. These hardware architectures are implemented on FPGA and ASIC platforms compared with state-of-the-art designs. In these architectures, first, 8-bit and then 16-bit serial structures are designed to implement. The serialised designs of 8-bit and 16-bit imply low area by consuming less number of slices. Finally, these serialised architectures are utilised for image encryption with the help of a controller. The simulation results and security analysis for hardware generated encrypted images show the better performance of proposed architectures and stronger resistance against entropy attack, differential attack, and statistical attacks.
CyberTOMP: A Novel Systematic Framework to Manage Asset-Focused Cybersecurity From Tactical and Operational Levels

Currently different reference models are used to manage cybersecurity, although practically none are applicable “as is” to lower levels as they do not detail specific procedural aspects for them. However, they urge organizations to develop a methodological foundation to manage cybersecurity at those levels. Although they allow organizations to adhere to a recognized standard at the strategic level, this advantage vanishes when organizations must define specific low-level procedures, allowing the appearance of inconsistency at tactical and operational levels between departments of the same organization or between organizations. The design of these elements with the required holism and homogeneity is difficult, and this is why generic processes focused on getting certified regarding a standard are usually originated, but they are insufficient to obtain effective cybersecurity because they are not focused on dealing with real cyber threats. Because of the great responsibility of lower levels to achieve effective cybersecurity, this lack of methodological definition makes it difficult to adapt cybersecurity to the highly dynamic cyber context with the required holism and strategic alignment. Our proposal provides CyberTOMP, a process for managing cybersecurity at lower levels, as well as a set of methodological elements that support it. The novelty of these contributions is that they complement the strategic standard selected by the organization, providing it with a set of procedural elements ready to be used out of the box, contributing those aspects required by high-level frameworks to manage cybersecurity at lower levels, for which there is no alternative with a managerial approach.
Molecular Dynamics Workflow Decomposition for Hybrid Classic/Quantum Systems

Since we are entering the Post-Moore Law era and consequently the limit of Von Neumann's architecture, the scientific community is looking for alternatives to satisfy the growing computing power demands of scientific applications. Quantum computing promises to achieve a computational advantage over the classic Von Neumann architecture. However, the limited capabilities of current noisy intermediate-scale quantum (NISQ) devices require quantum computers to interoperate with classic systems, forming the so-called hybrid quantum systems. Research on hybrid quantum systems led to the design of Variational Quantum Algorithms, currently the most promising way to move towards quantum advantage. However, execution time and accuracy of variational quantum algorithms are affected by different hyperparameters, including selected cost functions and parametrized quantum circuits. Consequently, providing developers with methods to select the right set of parameters is of paramount importance. In this work, we provide a formal method for the selection of hyperparameters in variational quantum algorithms, which will support quantum algorithms developers in the design of quantum applications, and evaluate it on a real-world scientific application, showing a reduction of error up to 31%.
Weakly Supervised Object Localization Through Inter-class Feature Similarity and Intra-class Appearance Consistency

Weakly supervised object localization (WSOL) aims at detecting objects through only image-level labels. Class activation maps (CAMs) are the commonly used features for WSOL. However, existing CAM-based methods tend to excessively pursue discriminative features for object recognition and hence ignore the feature similarities among different categories, thereby leading to CAMs incomplete for object localization. In addition, CAMs are sensitive to background noise due to over-dependence on the holistic classification. In this paper, we propose a simple but effective WSOL model (named ISIC) through Inter-class feature Similarity and Intra-class appearance Consistency. In practice, our ISIC model first proposes the inter-class feature similarity (ICFS) loss against the original cross entropy loss. Such an ICFS loss sufficiently leverages the shared features together with the discriminative features between different categories, which significantly reduces the model over-fitting risk to background noise and brings more complete object masks. Besides, instead of CAMs, a non-negative matrix factorization mask module is applied to extract object masks from multiple intra-class images. Thanks to intra-class appearance consistency, the achieved pseudo masks are more complete and robust. As a result, extensive experiments confirm that our ISIC model achieves state-of-the-art on both CUB-200 and ImageNet-1K benchmarks i.e., 97.3% and 70.0% GT-Known localization accuracy, respectively.
A Dual-Fusion Semantic Segmentation Framework with Gan for SAR Images

Deep learning based semantic segmentation is one of the popular methods in remote sensing image segmentation. In this paper, a network based on the widely used encoder-decoder architecture is proposed to accomplish the synthetic aperture radar (SAR) images segmentation. With the better representation capability of optical images, we propose to enrich SAR images with generated optical images via the generative adversative network (GAN) trained by numerous SAR and optical images. These optical images can be used as expansions of original SAR images, thus ensuring robust result of segmentation. Then the optical images generated by the GAN are stitched together with the corresponding real images. An attention module following the stitched data is used to strengthen the representation of the objects. Experiments indicate that our method is efficient compared to other commonly used methods.
Decentralized Federated Learning: A Comprehensive Survey and a New Blockchain-based Data Evaluation Scheme

Blockchain and Deep Learning (DL) are two of the most revolutionary concepts in the field of Computer Science. Both have made astounding leaps in research and application areas such as Finance, Healthcare, Internet of Things, and many more. Federated Learning (FL) is a type of distributed Deep Learning framework, in which the model is trained locally on each device and the trained gradients are sent to a central server which aggregates them and creates a global model. This helps ensure the data privacy of the user as the data never leaves the local device. However, this dependency on the central server can lead to various issues such as lack of transparency and communication bottleneck. Making this process decentralized can help address these issues. In this review, a detailed survey on using blockchain in federated learning is presented. This review also focuses on how can we use blockchain to make federated learning more transparent and decentralized to protect the privacy of the user. We also discuss the major strengths and drawbacks of each approach and further present a few ideas of our own, regarding some of these challenges and ways on how can these be improved. A new scheme to evaluate data using miners as well as methods to reduce storage overhead in decentralized federated learning are discussed in this paper.
Towards a Dynamic Composability Approach for using Heterogeneous Systems in Remote Sensing

Influenced by the advances in data and computing, the scientific practice increasingly involves machine learning and artificial intelligence driven methods which requires specialized capabilities at the system-, science- and service-level in addition to the conventional large-capacity supercomputing approaches. The latest distributed architectures built around the composability of data-centric applications led to the emergence of a new ecosystem for container coordination and integration. However, there is still a divide between the application development pipelines of existing supercomputing environments, and these new dynamic environments that disaggregate fluid resource pools through accessible, portable and re-programmable interfaces. New approaches for dynamic composability of heterogeneous systems are needed to further advance the data-driven scientific practice for the purpose of more efficient computing and usable tools for specific scientific domains. In this paper, we present a novel approach for using composable systems in the intersection between scientific computing, artificial intelligence (AI), and remote sensing domain. We describe the architecture of a first working example of a composable infrastructure that federates Expanse, an NSF-funded supercomputer, with Nautilus, a Kubernetes-based GPU geo-distributed cluster. We also summarize a case study in wildfire modeling, that demonstrates the application of this new infrastructure in scientific workflows: a composed system that bridges the insights from edge sensing, AI and computing capabilities with a physics-driven simulation.
Accelerating Score-Based Generative Models with Preconditioned Diffusion Sampling

Score-based generative models (SGMs) have recently emerged as a promising class of generative models. However, a fundamental limitation is that their inference is very slow due to a need for many (e.g., 2000) iterations of sequential computations. An intuitive acceleration method is to reduce the sampling iterations which however causes severe performance degradation. We investigate this problem by viewing the diffusion sampling process as a Metropolis adjusted Langevin algorithm, which helps reveal the underlying cause to be ill-conditioned curvature. Under this insight, we propose a model-agnostic preconditioned diffusion sampling (PDS) method that leverages matrix preconditioning to alleviate the aforementioned problem. Crucially, PDS is proven theoretically to converge to the original target distribution of a SGM, no need for retraining. Extensive experiments on three image datasets with a variety of resolutions and diversity validate that PDS consistently accelerates off-the-shelf SGMs whilst maintaining the synthesis quality. In particular, PDS can accelerate by up to $$29\times $$ on more challenging high resolution (1024 $$\times $$ 1024) image generation.
Robustness Analysis of Deep Learning-Based Lung Cancer Classification Using Explainable Methods

Deep Learning (DL) based classification algorithms have been shown to achieve top results in clinical diagnosis, namely with lung cancer datasets. However, the complexity and opaqueness of the models together with the still scant training datasets call for the development of explainable modeling methods enabling the interpretation of the results. To this end, in this paper we propose a novel interpretability approach and demonstrate how it can be used on a malignancy lung cancer DL classifier to assess its stability and congruence even when fed a low amount of image samples. Additionally, by disclosing the regions of the medical images most relevant to the resulting classification the approach provides important insights to the correspondent clinical meaning apprehended by the algorithm. Explanations of the results provided by ten different models against the same test sample are compared. These attest the stability of the approach and the algorithm focus on the same image regions.
GeoSS: Geographic Segmentation Security Barriers for Virtual Emotion Detection With Discriminative Priorities in Intelligent Cooperative Vehicular System

A development of the integrated vehicular system through ground and aerial cooperation using intelligent mobile robots and smart UAVs is required to support various applications including intelligent transportation, secure service, surveillance reinforcement. Also, it is highly anticipated that the applicability of virtual emotion is expanded continuously in smart systems. In this paper, we introduce a geographic segmentation security barrier system for the purpose of secure surveillance through virtual emotion detection in intelligent cooperative vehicular area with mobile robots and UAVs. The proposed system is necessary to provide discriminative priorities in the requested cooperative vehicular area consisting of mobile robots and UAVs with rapid construction. Moreover, with ILP formulation, we formally defined a problem whose goal is to maximize a total number of geographic segmentation security barriers such that the completed establishment of those barriers is done by a specific group of mobile robots and UAVs within the requested various districts where they are divided according to differential security priorities. To solve the problem, we devise novel schemes to return the maximum number of geographic segmentation security barriers so that it pursues the maximization of system lifetime consequently. Then, the proposed approaches are implemented through expansive simulations and their performances based on numerical results are evaluated clearly.
Interacting with Virtual Reality with a Controller Instead of the Body Benefits Performance and Perceptions

Methods of interaction within virtual environments, such as virtual reality (VR), typically include hand-held controllers, but recent advances in technology are making body-based interactions possible, such as eye and hand tracking. These modern interaction methods may prove to be more natural and intuitive for users and improve their experience by reducing the mental effort associated with learning a new interaction. In this within-subjects experiment, we compared users’ performance on a speeded button selection task and ratings of their experience in five different interaction methods: controller-based (Trigger, Push, and Raycasting) and body-based (Eyetracking and Handtracking). We found that participants performed best with controller-based interactions and rated their experiences more favorably than the body-based interactions. Participants reported higher mental and physical effort in the body-based conditions than the controller-based conditions. Implications for VR system design and future research are discussed.
Sublinear Secure Computation from New Assumptions

Secure computation enables mutually distrusting parties to jointly compute a function on their secret inputs, while revealing nothing beyond the function output. A long-running challenge is understanding the required communication complexity of such protocols—in particular, when communication can be sublinear in the circuit representation size of the desired function. For certain functions, such as Private Information Retrieval (PIR), this question extends to even sublinearity in the input size. We develop new techniques expanding the set of computational assumptions for sublinear communication in both settings:
Detection Beyond What and Where: A Benchmark for Detecting Occlusion State

Object detection is a computer vision technique that provides the most fundamental information for computer vision applications: “What objects are where?”, which has achieved significant progress thanks to the great success of deep learning in the past decade. With this kind of identification and localization, object detection is able to identify objects and track their precise locations in a scene. However, information beyond what and where is fairly desirable for more advanced applications, such as scene understanding, autonomous driving, and service robots, in which knowing the behavior or state or attribute of the objects is important. In this paper, we concern about the occlusion state of the target as well as its identification and localization, which is crucial for a service robot to keep track of a target or to grasp an object, for instance. We present a Dataset for Detecting Occlusion State of Objects (DDOSO). This benchmark aims to encourage research in developing novel and accurate methods for this challenging task. DDOSO contains 10 categories of bounding-box annotations collected from 6959 images. Based on this well-annotated dataset, we build baselines over two state-of-the-art algorithms. By releasing DDOSO, we expect to facilitate future researches on detecting the occlusion state of objects and draw more attention to object detection beyond what and where.
Holistic Parameter Optimization for Software Defect Prediction

A software defect prediction (SDP) model identifies the defect-prone modules. Setting appropriate parameters in an SDP model is critical because it affects the model performance. In a recent study, parameters were automatically explored using an optimization algorithm. However, such studies did not explore all the parameters that could be handled in the SDP process from preprocessing to model building, but only optimized parameters in some modeling process steps, such as feature selection or model building. Our goal is to improve the model performance by optimizing parameters across the entire SDP process. For this, we propose a cost-sensitive decision tree based on harmony search (HS-CSDT). HS-CSDT uses a harmony search algorithm to simultaneously identify the optimal feature set, regularization technique, class weight, and decision tree hyperparameters. We compared HS-CSDT against the methods in related studies in terms of probability of detection, probability of false alarm, G-measure, and file inspection reduction in the evaluation of 28 open-source projects. The results of the effect size using Cohen’s d reveal that HS-CSDT provides a statistically better performance than methods in related work. Experimental results show that optimizing the identified parameters throughout the entire SDP modeling process by using the optimization algorithm helps improve the model performance. In summary, HS-CSDT shows excellent defect prediction performance by automatically allocating an appropriate parameter set according to the software project. Thus, the model can help effectively allocate limited quality assurance resources.